Salt Lake City, Utah
June 23, 2018
June 23, 2018
July 27, 2018
Engineering Design Graphics
The creative and ill-structured nature of open-ended design problems has contributed to notoriously unreliable and difficult to implement methods of assessment (Bartholomew, 2017; Pollitt, 2004, 2012). However, an innovative form of assessment called adaptive comparative judgment (ACJ) has recently been developed and validated (Pollitt, 2012). Rather than traditional forms of assessment such as rubrics, ACJ relies on a judge simply looking at two pieces of student work and picking the better of the two. Herein resides the strength of ACJ, as judgments between two items are significantly more reliable than rubric or criterion-based judgments (Pollitt, 2004, 2012). The judges are not asked to provide a grade for each piece of work but rather asked to provide a holistic decision as to which artifact is better based on their own professional opinion. In ACJ, this comparative judgment process is repeated, and, utilizing a devised algorithm, each piece of student work is compared with others until a rank-order is produced for all items. This method of assessment has proven more reliable than traditional forms of assessment (Kimbell, 2012; Pollitt, 2004; 2012) and has been used in a variety of settings and content-areas (Bartholomew & Yoshikawa, 2017). In addition to improved reliability through the ACJ assessment process, judges also have the opportunity to provide feedback, or justification, in conjunction with each judgment. This feedback becomes a valuable source of feedback and guidance for students and, although ACJ was originally designed as a tool for professional assessors and teachers, this paper will examine a new approach to ACJ with students acting as the judges of their peer work in the ACJ process. Situated in a university-level engineering graphics course this research will study the impact on student performance when students are engaged in ACJ as judges—both providing and receiving feedback to and from their peers—on an open-ended design problem. Additionally, this paper will examine the impacts of resituating the ACJ process in the midst of an open-ended design project, as opposed to the conclusion, with a specific emphasis on using ACJ as a formative assessment and feedback tool.
Bartholomew, S. R., & Strimel, G. J., & Garcia, E., & Yoshikawa, E., & Zhang, L. (2018, June), Formative Feedback For Improved Student Performance Through Adaptive Comparative Judgment Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. https://peer.asee.org/30531
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015