June 15, 2019
June 15, 2019
June 19, 2019
Electrical and Computer
This will be a complete research paper. In the assessment of closed-ended engineering problem solving tasks, a challenge often faced by the instructor is to ensure delivery of consistent and reliable feedback to students. Increasing the level of detail in a grading scheme is a common technique cited in the “assessment for learning” literature to increase assessment reliability in grading and feedback delivery. This is referred to as the grading grain or granularity, and the use of this approach motivated us to examine whether a finer grained grading scheme actually improves the observable quality of feedback given by teaching assistants on electrical engineering tests.
Graded student test papers from three electrical engineering courses were used for this study. For each course, 7 unique problems were selected. For each problem, we randomly selected between 15 and 26 graded student solutions that represented the full range of performance levels. We also acquired from the instructor the associated solution guide (i.e. the correct solution) and grading scheme. Every problem was characterized in terms of its granularity and complexity.
Granularity and complexity served as the independent variables in this work. The definitions for these are derived from literature in the field and described in detail in the full paper. Our hypothesis was that increasing the granularity of the solution guide, and/or increasing the complexity of the problem would improve the quality of the feedback. The goal was to analyze the quality of feedback across test problems having varying grading granularity and level of problem complexity.
The feedback marks on the student papers were classified and coded. The categories included: • Validating: e.g. check-marks • Flagging: e.g. error identification such as cross-marks • Penalizing: e.g. grade deductions • Tagging: e.g. reference to a feedback framework, scheme, or rubric • Elaborating: e.g. free-form textual feedback
The data shows a positive correlation between problem complexity and feedback quality, but no relationship between the granularity of the solution guide and the feedback quality. The findings of this study are substantial as they reveal that it is not always the case that the level of grain in a grading scheme improves assessors’ feedback quality, as the literature presumes. The findings also demonstrate that the problem’s complexity can significantly influence the way assessors provide feedback to students in the context of electrical engineering problem solving tasks. The results may suggest that to improve the quality of feedback to students in electrical engineering courses, more time and attention should be spent on the quality of the problems given to the students rather than developing finer grained grading schemes.
Memarian, B., & McCahan, S. (2019, June), Systematic Analysis of Formative Feedback, Focus on Electrical Engineering Assessments Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--33334
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2019 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015