Asee peer logo

Efforts to Improve Undergraduate Grader Consistency: A Qualitative Analysis

Download Paper |

Conference

2018 ASEE Annual Conference & Exposition

Location

Salt Lake City, Utah

Publication Date

June 23, 2018

Start Date

June 23, 2018

End Date

July 27, 2018

Conference Session

Practice II: Curricular Innovations

Tagged Division

Educational Research and Methods

Page Count

19

Permanent URL

https://peer.asee.org/30366

Download Count

10

Request a correction

Paper Authors

biography

Nathan M. Hicks Purdue University, West Lafayette (College of Engineering) Orcid 16x16 orcid.org/0000-0003-2512-8484

visit author page

Nathan M. Hicks is a Ph.D. student in Engineering Education at Purdue University. He received his B.S. and M.S. degrees in Materials Science and Engineering at the University of Florida and taught high school math and science for three years.

visit author page

biography

Kerrie A. Douglas Purdue University, West Lafayette (College of Engineering)

visit author page

Dr. Douglas is an Assistant Professor in the Purdue School of Engineering Education. Her research is focused on improving methods of assessment in large learning environments to foster high-quality learning opportunities.

visit author page

Download Paper |

Abstract

In this research paper, we explore the difficult decisions faced by large-scale, multi-section courses in early undergraduate engineering education regarding fair and consistent assessment of student learning across sections. Our previous analysis of grading patterns of undergraduate graders in a first-year engineering course revealed that divergent decisions likely stemmed from two sources: insufficient grader training and ambiguities in rubrics and assignments. After revising rubrics and implementing grader training for a semester, we conducted think-aloud interviews with 17 undergraduate graders regarding grading, rubrics, and training. Qualitative analysis identified four technical aspects of rubrics that led to divergent grading decisions (wordiness, redundancies, unexpected solutions, and grade misfit) and five aspects that limited training effectiveness (length, misalignment, insufficient feedback, limited consequences, and philosophical misunderstanding). These findings contribute nuance to and extend upon aspects of rubric design and undergraduate grader training that have been previously identified in the literature. Recommendations related to issues identified are provided.

Hicks, N. M., & Douglas, K. A. (2018, June), Efforts to Improve Undergraduate Grader Consistency: A Qualitative Analysis Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. https://peer.asee.org/30366

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015