June 14, 2015
June 14, 2015
June 17, 2015
Educational Research and Methods
26.813.1 - 26.813.15
Gender and Personality Type Bias in Peer EvaluationTeamwork is an expected experience in an undergraduate engineering program, particularly indesign and project courses. The teamwork experiences are often accompanied by peerevaluation activities in which team members score or rate one another on contributions to theteam functioning and project success. The reliability and validity of peer evaluation results areimportant considerations for both students and educators, especially in cases where the resultscontribute to student grades.Diversity is generally viewed as a beneficial characteristic of student teams; however, it ispossible that team diversity could introduce biases that affect the validity and reliability of peerevaluation scores. The influence of factors such as social style bias, gender, and ethnicity onpeer evaluation outcomes have been studied, particularly in business and management domains.The current work examines the influence of gender on peer evaluation outcomes from theperspective of an intense, seven-week sophomore engineering design course. In addition, theinfluence of personality type (as measured through the Myers-Briggs Type Indicator) on peerevaluation scores received is also examined. Each of the four Myers-Briggs domains(Introversion/Extraversion, Sensing/iNtuition, Thinking/Feeling, and Judging/Perceiving) isconsidered separately in the analysis.This study was conducted in a team-based second year mechanical engineering design course.Data from eight different cohorts from 2007 to 2014, with roughly 120 students per cohort, wereconsidered. Diverse course teams were formed by maximizing heterogeneity in GPA,personality type, hands-on skills, communication ability, and other factors. Each cohortcompleted six online peer evaluation exercises. From 2007 to 2010, students used an allocation-of-points method for conducting the peer evaluations. Starting in 2011, in a crossover study,students completed half of their evaluations using an allocation-of-points method and half usinga behaviourally anchored rating scale (BARS) method. The influences of evaluatee gender andMyers-Briggs preferences were considered in determining average peer evaluation scorereceived. The possible influence of academic ability on evaluation scores received wasaccounted for by also considering evaluatee course grades (which have been reported to have asignificant positive correlation with intrinsic motivation and hence peer evaluation scorereceived).The preliminary analysis suggests there are both gender and personality type biases thatinfluence peer evaluation. For the eight cohorts considered, female students received slightlyhigher (+0.3%) peer evaluation scores on average over the eight years in spite of the fact thatthey received lower (-3.1%) average test grades in the course. In terms of Myers-Briggspersonality preferences, a statistically significant difference in peer evaluation score (1%) wasobserved for the judging/perceiving domain (judging higher), negatively correlated with averageexam scores. Statistical differences were not observed for the other three Myers-Briggsdomains. Although the observed effects are small (≤ 1%) they suggest peer evaluation outcomesused to determine students’ grades should be applied carefully. More pronounced differenceswere observed with the BARS-type peer evaluation instrument over a point-allocation one.
Ostafichuk, P. M., & Sibley, J., & d'Entremont, A. G., & Shirzad, N. (2015, June), Gender and Personality Type Influence in Peer Evaluation Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24150
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015