Asee peer logo

Grader Consistency in using Standards-based Rubrics

Download Paper |

Conference

2017 ASEE Annual Conference & Exposition

Location

Columbus, Ohio

Publication Date

June 24, 2017

Start Date

June 24, 2017

End Date

June 28, 2017

Conference Session

Tips and Tricks for Assessing Student Performance

Tagged Division

New Engineering Educators

Page Count

19

Permanent URL

https://peer.asee.org/28416

Download Count

53

Request a correction

Paper Authors

biography

Nathan M. Hicks Purdue University, West Lafayette (College of Engineering) Orcid 16x16 orcid.org/0000-0003-2512-8484

visit author page

Nathan M. Hicks is a Ph.D. student in Engineering Education at Purdue University. He received his B.S. and M.S. degrees in Materials Science and Engineering at the University of Florida and taught high school math and science for three years.

visit author page

biography

Heidi A. Diefes-Dux Purdue University, West Lafayette (College of Engineering) Orcid 16x16 orcid.org/0000-0003-3635-1825

visit author page

Heidi A. Diefes-Dux is a Professor in the School of Engineering Education at Purdue University. She received her B.S. and M.S. in Food Science from Cornell University and her Ph.D. in Food Process Engineering from the Department of Agricultural and Biological Engineering at Purdue University. She is a member of Purdue’s Teaching Academy. Since 1999, she has been a faculty member within the First-Year Engineering Program, teaching and guiding the design of one of the required first-year engineering courses that engages students in open-ended problem solving and design. Her research focuses on the development, implementation, and assessment of modeling and design activities with authentic engineering contexts.

visit author page

Download Paper |

Abstract

Standards-based grading offers several advantages over the traditional norm-referenced approach. These advantages include improving the quality of feedback provided to the students, facilitating graders’ abilities to assign grades consistently and objectively, and enhancing the meaning, trustworthiness, and integrity of grades. However, for large-scale classes that span several sections and include multiple instructional teams, evidence suggests that, even with standards-based grading, assigned grades vary from section to section more than one would expect for randomly distributed populations. While differences in strengths and weaknesses of each instructor may contribute to this variation, so too may tendencies of each grader. This latter cause may be reduced through the use of rubrics; however, the inherent subjectivity of text-based rubrics to assess open-ended tasks makes natural variability of interpretation potentially unavoidable. As engineering courses often tackle complex and open-ended problems that may elicit a variety of responses exhibiting a wide range of quality, inconsistency of interpretation or application of rubrics threatens to undermine grade integrity—the very reason for using standards-based grading and rubrics in the first place.

This exploratory study shows the results of implementing a standards-based grading system in a large-scale first-year engineering course. These results highlight the challenge of achieving consistency of rubric interpretation and application. Through analysis of specific examples of student work and grader scoring, this paper demonstrates potential causes for such inconsistencies, including lack of grading discipline and ambiguities in assignments and rubrics. These causes, in conjunction with the literature regarding best-practices of standards-based grading and rubrics, illustrate two of the most important considerations we recommend for standards-based courses with multiple graders: extensive training and oversight of graders; and careful construction and revision of assignments and rubrics.

Hicks, N. M., & Diefes-Dux, H. A. (2017, June), Grader Consistency in using Standards-based Rubrics Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. https://peer.asee.org/28416

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2017 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015