Bias in First-Year Engineering Student Peer Evaluations

Lea Wittie; James Bennett; Carly Merrill; Jove Graham; Troy Schwab

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: First-Year Programs: Student Perceptions and Perspectives
Tagged Division: First-Year Programs
Tagged Topic: Diversity
Page Count: 9
DOI: 10.18260/1-2--36748
Permanent URL: https://peer.asee.org/36748
Download Count: 360

Request a correction

Paper Authors

biography

Lea Wittie Bucknell University

visit author page

Lea Wittie is an Associate Professor in the department of Computer Science in the Engineering College at Bucknell University. She has spent the past 4 years coordinating the first year Engineering student Introduction to Engineering and over a decade participating in the program before that.

visit author page

biography

James Bennett Cornell University

visit author page

James Bennett is a biomedical engineer specializing in medical device design and development. He has earned a Bachelor of Science Degree in Biomedical Engineering from Bucknell University and is currently pursuing a Master's of Engineering in Biomedical Engineering at Cornell University.

visit author page

biography

Carly Merrill Bucknell University

visit author page

Carly Merrill is currently working in the healthcare industry where she is pursing a career in strategic product development. She has recently earned a Bachelor of Science Degree in Biomedical Engineering from Bucknell University.

visit author page

biography

Jove Graham Geisinger

visit author page

Jove Graham, PhD is an Associate Professor in the Center for Pharmacy Innovation and Outcomes at Geisinger, a nonprofit integrated health system in Pennsylvania.

visit author page

biography

Troy Schwab Bucknell University

visit author page

Troy Schwab is a computer scientist currently working as a federal consultant, specifically concerning data engineering. He received undergraduate degrees in computer science and contemporary music composition from Bucknell University.

visit author page

Download Paper | Permalink

Abstract

This Complete Research paper describes a study on race, gender, and self bias in first year engineering student’s team peer evaluations.

Motivation: Our institution runs a first year introduction to engineering course with approximately 200 students that uses team projects over the course of the semester. Each project has 2-5 students per team and incorporates peer and self evaluations into each student’s individual project grades. The researchers began this study to see how racial, gender, and self bias impact these peer evaluations.

Background: Peer evaluations are often employed in instances of group work, particularly in the undergraduate setting. These peer evaluations can present important information regarding team dynamic and distribution of workload. However, this method of assessment is also susceptible to both explicit and implicit biases, specifically in regard to race, gender, and self bias. After identifying possible biases in our peer evaluation procedure, the researchers plan to examine methods to mitigate these biases.

Methods and Assessment: For each project, students submitted peer evaluations of themselves and each of their team members and were required to split 100 points amongst all team members including themselves with an optional written rationale for scores. The 1725 peer evaluation scores collected by this study were double-key entried into a database. Participants also were asked to self identify as one of 6 gender options and 8 race options. If participants selected multiple races, they were assigned to the less common one. Two generalized linear regression models (GLMs) were used, one to estimate self-bias within each gender/race group (i.e. whether students give higher scores to themselves), and one to estimate how members in each gender/race group scored members of other groups, excluding the self-scores. Model coefficients significantly different from zero at the p<0.05 level indicated differences between groups and therefore possible evidence of biases.

Results: There were 160 participants, all of whom identified as either male or female. Due to small numbers, participants were combined into 3 race categories (White, Asian, or ‘Other’) for a total of 6 gender/race groups. Results showed that the students were significantly more likely to give themselves a higher score than other students on average (around 8%), even after accounting for gender/race, and this self-bias was consistent in both genders. While males gave scores to Other males that were significantly lower than what they gave to all other groups (8-15% lower, p<0.05) and significantly lower than what Other males received from White females or Asian males, suggesting a possible negative bias from White males to Other males. Similarly, Asian males gave scores to Other females that were significantly higher than what they gave to all other groups (53-77% higher, p<0.0001) and significantly higher that what Other females received from all other groups suggesting a possible positive bias from Asian males to Other females.

Citation
Format

Wittie, L., & Bennett, J., & Merrill, C., & Graham, J., & Schwab, T. (2021, July), Bias in First-Year Engineering Student Peer Evaluations Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36748

TY - CPAPER
AB - This Complete Research paper describes a study on race, gender, and self bias in first year engineering student’s team peer evaluations.

Motivation: Our institution runs a first year introduction to engineering course with approximately 200 students that uses team projects over the course of the semester. Each project has 2-5 students per team and incorporates peer and self evaluations into each student’s individual project grades. The researchers began this study to see how racial, gender, and self bias impact these peer evaluations.

Background: Peer evaluations are often employed in instances of group work, particularly in the undergraduate setting. These peer evaluations can present important information regarding team dynamic and distribution of workload. However, this method of assessment is also susceptible to both explicit and implicit biases, specifically in regard to race, gender, and self bias. After identifying possible biases in our peer evaluation procedure, the researchers plan to examine methods to mitigate these biases.

Methods and Assessment: For each project, students submitted peer evaluations of themselves and each of their team members and were required to split 100 points amongst all team members including themselves with an optional written rationale for scores. The 1725 peer evaluation scores collected by this study were double-key entried into a database. Participants also were asked to self identify as one of 6 gender options and 8 race options. If participants selected multiple races, they were assigned to the less common one. Two generalized linear regression models (GLMs) were used, one to estimate self-bias within each gender/race group (i.e. whether students give higher scores to themselves), and one to estimate how members in each gender/race group scored members of other groups, excluding the self-scores. Model coefficients significantly different from zero at the p&amp;lt;0.05 level indicated differences between groups and therefore possible evidence of biases.

Results: There were 160 participants, all of whom identified as either male or female. Due to small numbers, participants were combined into 3 race categories (White, Asian, or ‘Other’) for a total of 6 gender/race groups. Results showed that the students were significantly more likely to give themselves a higher score than other students on average (around 8%), even after accounting for gender/race, and this self-bias was consistent in both genders. While males gave scores to Other males that were significantly lower than what they gave to all other groups (8-15% lower, p&amp;lt;0.05) and significantly lower than what Other males received from White females or Asian males, suggesting a possible negative bias from White males to Other males. Similarly, Asian males gave scores to Other females that were significantly higher than what they gave to all other groups (53-77% higher, p&amp;lt;0.0001) and significantly higher that what Other females received from all other groups suggesting a possible positive bias from Asian males to Other females.
AU - Lea Wittie
AU - James Bennett
AU - Carly Merrill
AU - Jove Graham
AU - Troy Schwab
CY - Virtual Conference
DA - 2021/07/26
PB - ASEE Conferences
TI - Bias in First-Year Engineering Student Peer Evaluations
UR - https://peer.asee.org/36748
DO - 10.18260/1-2--36748
ER -