A Comparison of Peer Evaluation Methods in Capstone Design

Joshua A Enszer; Mariajose Castellanos

Download Paper | Permalink

Conference: 2013 ASEE Annual Conference & Exposition
Location: Atlanta, Georgia
Publication Date: June 23, 2013
Start Date: June 23, 2013
End Date: June 26, 2013
ISSN: 2153-5965
Conference Session: "How Do We Compare?" - Students, Case Studies, and Learning Approaches
Tagged Division: Chemical Engineering
Page Count: 9
Page Numbers: 23.32.1 - 23.32.9
DOI: 10.18260/1-2--19046
Permanent URL: https://peer.asee.org/19046
Download Count: 583

Paper Authors

biography

Joshua A Enszer University of Maryland Baltimore County

visit author page

Dr. Joshua Enszer is a full-time lecturer in Chemical Engineering at the University of Maryland Baltimore County. He has taught core and elective courses across the curriculum, from introduction to engineering science and material and energy balances to process control and modeling of chemical and environmental systems. His research interests include technology and learning in various incarnations: electronic portfolios as a means for assessment and professional development, implementation of computational tools across the chemical engineering curriculum, and game-based learning.

visit author page

author page

Mariajose Castellanos University of Maryland, Baltimore County

Download Paper | Permalink

Abstract

A Comparison of Peer Evaluation Methods in Capstone DesignThe chemical engineering capstone design course at our university consists of a semester-longproject where students work in predetermined groups. The emphasis of the experience is to allowstudents to develop and explore their engineering creativity while at the same time cultivate theimportance of communication and teamwork skills. Students receive an industrially relevantopen-ended problem statement and face a rigorous schedule with specific weekly deliverables.We concentrate on building teamwork skills, ownership of the project, accountability,communication and self-assessment. We use the projects to incorporate the skills our graduatesneed to join the workforce or graduate schools while inspiring students to evaluate, defend, andtreasure their creations. Measuring group dynamics, division of labor, engagement, group versusindividual experiences is a challenging task. Student grades are highly dependent on peerevaluation, as the team does not receive the same grade for group assignments, but instead, groupgrades are multiplied by a peer assessment factor (for example if student’s X peer assessment is50 percent, all group grades for that student will be multiplied by 0.5. In the spring of 2012, twoseparate methods were employed as part of peer assessment: a traditional “point division”method [1] and CATME, the Comprehensive Assessment of Team Member Effectiveness [2].The point division method instructs students to divide 100 points between the group members; ifthere are five team members in a group and a student believes all the students in the groupparticipated equivalently in the project then he/she will assign 20 points to every student in thegroup. CATME allows the students to evaluate team members in more behavioral areas:contribution to the team, interaction with peers, keeping team on track, quality of work,proficiency in the discipline. Because the point division method is effectively a zero-sum gamewhose philosophy may contradict the cooperative nature of the course, we are interested inreplacing it, and therefore have explored correlations between it and CATME. Additionally, weused the data from both methods to evaluate how students rated themselves compared to theratings of their peers. In this report we share the results of our analysis. We have determined thatthe two methods have a statistically significant positive correlation and that students on averageself assess higher than their peers assess them.[1] Michaelsen, L. K., A. B. Knight, and L. D. Fink (eds.), Team-Based Learning: ATransformative Use of Small Groups in College Teaching. Sterling, Va.: Stylus, 2004.[2] Ohland, Matthew W., H. R. Pomeranz, and H. W. Feinstein, “The ComprehensiveAssessment of Team Member Effectiveness: A New Peer Evaluation Instrument,” Proceedingsof the American Society of Engineering Education Annual Conference, Chicago, IL, June 2006.

Citation
Format

Enszer, J. A., & Castellanos, M. (2013, June), A Comparison of Peer Evaluation Methods in Capstone Design Paper presented at 2013 ASEE Annual Conference & Exposition, Atlanta, Georgia. 10.18260/1-2--19046

TY - CPAPER
AB - A Comparison of Peer Evaluation Methods in Capstone DesignThe chemical engineering capstone design course at our university consists of a semester-longproject where students work in predetermined groups. The emphasis of the experience is to allowstudents to develop and explore their engineering creativity while at the same time cultivate theimportance of communication and teamwork skills. Students receive an industrially relevantopen-ended problem statement and face a rigorous schedule with specific weekly deliverables.We concentrate on building teamwork skills, ownership of the project, accountability,communication and self-assessment. We use the projects to incorporate the skills our graduatesneed to join the workforce or graduate schools while inspiring students to evaluate, defend, andtreasure their creations. Measuring group dynamics, division of labor, engagement, group versusindividual experiences is a challenging task. Student grades are highly dependent on peerevaluation, as the team does not receive the same grade for group assignments, but instead, groupgrades are multiplied by a peer assessment factor (for example if student’s X peer assessment is50 percent, all group grades for that student will be multiplied by 0.5. In the spring of 2012, twoseparate methods were employed as part of peer assessment: a traditional “point division”method [1] and CATME, the Comprehensive Assessment of Team Member Effectiveness [2].The point division method instructs students to divide 100 points between the group members; ifthere are five team members in a group and a student believes all the students in the groupparticipated equivalently in the project then he/she will assign 20 points to every student in thegroup. CATME allows the students to evaluate team members in more behavioral areas:contribution to the team, interaction with peers, keeping team on track, quality of work,proficiency in the discipline. Because the point division method is effectively a zero-sum gamewhose philosophy may contradict the cooperative nature of the course, we are interested inreplacing it, and therefore have explored correlations between it and CATME. Additionally, weused the data from both methods to evaluate how students rated themselves compared to theratings of their peers. In this report we share the results of our analysis. We have determined thatthe two methods have a statistically significant positive correlation and that students on averageself assess higher than their peers assess them.[1] Michaelsen, L. K., A. B. Knight, and L. D. Fink (eds.), Team-Based Learning: ATransformative Use of Small Groups in College Teaching. Sterling, Va.: Stylus, 2004.[2] Ohland, Matthew W., H. R. Pomeranz, and H. W. Feinstein, “The ComprehensiveAssessment of Team Member Effectiveness: A New Peer Evaluation Instrument,” Proceedingsof the American Society of Engineering Education Annual Conference, Chicago, IL, June 2006.
AU - Joshua A Enszer
AU - Mariajose Castellanos
CY - Atlanta, Georgia
DA - 2013/06/23
PB - ASEE Conferences
TI - A Comparison of Peer Evaluation Methods in Capstone Design
UR - https://peer.asee.org/19046
DO - 10.18260/1-2--19046
ER -