June 15, 2019
June 15, 2019
June 19, 2019
Educational Research and Methods
This Work-in-Progress paper presents the development of a grading process model for open-ended engineering problems with the assistance of rubrics across many graders. As many problems in engineering are open-ended, best practices dictate the use of authentic open-ended performance tasks for assessment in engineering courses. Unfortunately, evaluation of open-ended work inherently requires at least some degree of subjective judgment. However, for assessment to be considered valid, it is imperative that evaluation is fair, which includes holding all students to the same, transparently communicated expectations and standards. In large, multi-section courses, which are common in first-year engineering programs, achieving this fairness of assessment evaluation across all students becomes a significant challenge. Recognizing that the grading process consists of humans interacting with objects and making decisions, this study treats the grading process as a complex socio-technical system and describes the process of developing a model using the Functional Resonance Analysis Method (FRAM). The paper describes how the FRAM will be applied to identify sources that contribute to variability of system performance and possible corresponding control mechanisms.
Hicks, N. M., & Douglas, K. A., & Diefes-Dux, H. A. (2019, June), Understanding Grader Reliability through the Lens of Cognitive Modeling Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--33477
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2019 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015