Asee peer logo

Reliability, Validity, And Bias In Peer Evaluations Of Self Directed Interdependent Work Teams

Download Paper |


2001 Annual Conference


Albuquerque, New Mexico

Publication Date

June 24, 2001

Start Date

June 24, 2001

End Date

June 27, 2001



Page Count


Page Numbers

6.845.1 - 6.845.37

Permanent URL

Download Count


Request a correction

Paper Authors

author page

Robert Thompson

Download Paper |

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Session 1330

Reliability, Validity, and Bias in Peer Evaluations of Self-Directed Interdependent Work Teams

Robert S. Thompson Colorado School of Mines

I. Introduction

Teamwork education has become increasingly important over the last decade. In a recent survey conducted at the Purdue School of Engineering, over 76% of the students responded that they had been involved as members of student work teams (486 out of 1,953 responded) 1. This emphasis on teamwork skills stems from the widespread use of teams in industry.

Peer evaluations are being used as a source of information for improving team performance 2,3 and accounting for individual contributions to a group project 4. Peer evaluations as a source of information for small self-directed group work have an appeal because the team members are in the best position to observe the team skills of their fellow team members. Despite this advantage, concerns have been levied against the use of peer evaluations. Abson 5, for example, suggested that peer evaluations can be abused and have undesirable effects on individuals in the group. Mathews 6 studied peer assessment of small group work in a management studies program. He noted patterns of response included giving all group members the same score, collusion between group members, and potential ganging-up on one member. Mathews also noted that perceptions can vary between people accounting for some of the variability. Mathews comments were based on his observations. He did not report any statistical data to support his claims.

Engineering design projects are a common source of teamwork in engineering education. Teamwork in these settings is characterized by three attributes: team members having a common goal, dependence on each other to achieve their goal, and intense work over an extended period of time. Because of the extended nature of the group work and the interdependence among team members, friendships have time to develop over the duration of the project. In many cases, friendships were formed prior to the group work. With this nature of the teamwork and the corresponding use of peer evaluations, there is a need for a better understanding of the reliability, validity, and bias in peer assessments of students working on these interdependent, self-directed, problem-solving teams. This research focuses on the specific problem of the reliability, validity, bias, and user acceptance of peer evaluations in small interdependent self-directed problem- solving teams in an educational setting. Generalizability theory techniques, structured interviews, and survey data were used to answer the following research questions: 1. What is the user reaction to peer evaluations? 2. What is the level of consensus in peer evaluations? 3. What is the level of bias in peer evaluations?

“Proceedings of the 2001 American Society for Engineering Education Annual Conference & Exposition Copyright 2001, American Society for Engineering Education

Thompson, R. (2001, June), Reliability, Validity, And Bias In Peer Evaluations Of Self Directed Interdependent Work Teams Paper presented at 2001 Annual Conference, Albuquerque, New Mexico.

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2001 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015