June 15, 2019
June 15, 2019
June 19, 2019
An assessment of bias and precision in instructor grading was undertaken at several private and public institutions with civil engineering programs. At five institutions, undergraduate civil engineering majors completed a concept inventory at the conclusion of their first course in geotechnical engineering. The ten-question instrument focused on fundamental concepts in geotechnical engineering to assess students’ knowledge gained throughout the course. A random sample of ten surveys was collected from each of the five institutions, leading to a dataset of n = 50 concept inventories encompassing a breadth of student populations. A team of geotechnical engineering professors from nine institutions (including four institutions not included in the dataset) independently graded the 50 concept inventories, using an established solution to the instrument. The end result was a distribution of nine instructor scores for each question within the dataset of student responses. The objectives of this study were (1) to quantify instructor bias in grading concept inventories by examining whether there are differences between instructors' grading of their own students and instructors' grading of anonymous surveys, and (2) to quantify instructor precision (instructor-to-instructor variability) in grading. Statistical analyses were performed on the scores of the concept inventories to quantify the distributions of instructor grading within the distributions of student scores. In the context of an undergraduate geotechnical engineering course, this paper discusses the concept inventory, grading criteria, institutional contexts, statistical analyses, and suggestions for future research. Recommendations are provided for reducing bias and increasing precision in instructor grading in undergraduate civil engineering courses.
Ghanat, S. T., & Kaklamanos, J., & Kunberger, T., & Walton-Macaulay, C., & Immanuel, S., & Saftner, D. A., & Barry, B. E., & Griffiths, S., & Shillaber, C. M., & Swan, C. (2019, June), Bias and Precision in Instructor Grading of Concept Inventories in Geotechnical Engineering Courses Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. https://peer.asee.org/32154
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2019 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015