June 15, 2019
June 15, 2019
June 19, 2019
Rubrics have often been touted as an effective tool to communicate expectations and save time grading. One area of study is the improvement of rubrics to increase inter-rater reliability; that is, creating consistent rubrics such that multiple assessors are likely to assign the same score or designation to the same work. The number of columns or “standard levels” within a rubric is often up to debate: for a given criteria, should work be assessed on a three-, four-, or five-point scale, or should another strategy be adopted altogether? One drawback to increasing the number of levels in a rubric is that it may become more likely for multiple assessors to use the rubric to assign different ratings. Another task that comes with increased levels is the writing of descriptions that accurately communicate the kind of work that merits each level. Could it be effective to structure rubrics using only two levels? In this work, we will summarize some of the literature on the development of rubrics, and then describe our process of creating a “two-column” rubric – one that only describes excellent and minimally acceptable performances. We will share examples of how we apply these two-column rubrics in our junior- and senior-level chemical engineering laboratory courses. We explain our algorithm for using the two-column rubrics, including how faculty, teaching assistants, and students are trained to apply the algorithm. Finally, we conducted inter-rater reliability analysis for an example assignment and found modest improvement in agreement between assessors compared to previous evaluation methods. We conclude with our next steps in our development and revision of these rubrics.
Enszer, J. A. (2019, June), Developing Reliable Lab Rubrics Using Only Two Columns Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. https://peer.asee.org/32644
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2019 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015