Salt Lake City, Utah
June 23, 2018
June 23, 2018
July 27, 2018
Design in Engineering Education
Adequately preparing students for future careers, learning, and civic opportunities is at the forefront of discussions around best practices for education (Sharples, de Roock, Ferguson, Gaved, Herodotou, Koh,...& Weller, 2016). However, despite an emphasis on this preparation, research shows there is not always alignment between student, teacher, and industry perceptions of skills, traits, and competencies necessary for this preparation (Adecco, 2014; Deloitte, 2015). Further, much of this preparation revolves around difficult-to-assess-and-teach 21st century skills such as critical thinking, problem-solving, creativity, innovative thinking, and collaboration (NRC, 2011). Despite the challenges associated with these issues, the adequate preparation of students for future success is paramount (Partnership, 2017). This research will investigate the use of a relatively-new form of assessment, Adaptive Comparative Judgment, as a tool for not only improving the assessment practices of educators in open-ended situations, but also identifying the potential disconnects between student, teacher, and industry perceptions of quality. Rather than traditional forms of assessment such as rubrics, ACJ is an assessment approach which relies on a judge simply looking at two pieces of student work and picking the better of the two. The judges are not asked to provide a grade for each piece of work but rather asked to provide a holistic decision as to which artifact is better based on their own professional opinion. In ACJ, this comparative judgment process is repeated, and, using a devised algorithm, each piece of student work is compared with others until a rank-order is produced for all items. This method of assessment has proven more reliable than traditional forms of assessment (Kimbell, 2012; Pollitt, 2004; 2012) and has been used in a variety of settings and content-areas (Bartholomew & Yoshikawa, 2017). Significantly, researchers (Bartholomew, Hartell, & Strimel, 2017) have also used ACJ assessment to identify similarities and differences in design values amongst diverse groups with success. This project entails first-year university engineering students, engineering instructors, and industry professionals collaborating to assess student work. Each group (students, instructors, and industry professionals) used ACJ to rank the final projects from a first-year engineering course at a public land-grant institution. The similarities, differences, and rationale statements for assessing the students work were examined to identify the assessment values and perceptions of quality of each group. A better understanding of these similarities and differences can inform education practice and may assist educators in ensuring alignment between current practice and industry-needs towards future student success.
Bartholomew, S. R., & Strimel, G. J., & Purzer, S., & Zhang, L., & Yoshikawa, E. (2018, June), Differences and Similarities in Student, Instructor, and Professional Perceptions of "Good Engineering Design" through Adaptive Comparative Judgment Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. https://peer.asee.org/30333
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015