June 14, 2015
June 14, 2015
June 17, 2015
Design in Engineering Education
26.692.1 - 26.692.23
Evaluating the Pre-Professional Engineer: Project Team and Individual PerformanceAt a large midwestern research university, teams of 5-7 engineers from multiple disciplines worktogether on multi-term engineering design projects. Annually, the program enrolls approximately220 students on engineering design project teams. As we prepare to enter the fifth year of thisprogram, we are studying the effectiveness of scalable administration of effective peer reviewfeedback. Currently, we collect, anonymize and disseminate performance feedback utilizing theCATME team peer evaluation tool (Loughry, Ohland, & DeWayne, 2007) twice during thetenure of each team’s project. We observed that our students were unable to sufficientlyinterpret the feedback data, and their reactions to the feedback from their peers ranged fromcomplete dismissal to an inability to integrate the feedback into their performance. As a result,this has not generated our desired outcomes: improved project-specific engineering design skills,professional behavior, or evidence of self-reflection. To address this, we have created andimplemented an additional avenue for peer-to-peer anonymized feedback: a qualitative surveyutilizing coded competencies drawn from various industry-based best practices. In thissupplemental qualitative survey, students select two performance competencies (one positive andone developmental) from a predefined list for each of their peers. Students then support theselected competencies by citing specific behavioral examples.We coded these statements into three skill constructs, professionalism, teamwork, and core skills,using a unique combination of structural, attribute, provisional, and holistic qualitative codingtechniques. The coded data is analyzed against semi-anonymized demographic data in order todetermine how various identities affect the ways in which students evaluate and are evaluated bytheir peers. Utilizing action research methodology, we made relevant changes to the surveysprior to the second round of evaluations to improve students’ understanding of the surveyquestions.Student growth is measured via peers’ quantitative (CATME) and qualitative evaluations ofengineering design skills and professional behavior, as well as the students’ own ability to craftconstructive feedback statements. Because specific instances of behavior are cited in thequalitative survey, students are better equipped to leverage their strengths and addressshortcomings. We identify trends in peer evaluations with regard to self-identified gender,residency status, academic year, and other categories. The research team documents the ways inwhich students evaluate one another in an experiential team setting, the roles that certainacademic and social identities play in that evaluation process, and how students’ evaluationabilities change over time.
Adams, J. M., & DeGraaff, M. D., & Hohner, G. S. (2015, June), Evaluating the Pre-Professional Engineer: Exploring the Peer Review Process Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24029
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015