Paper Authors

Abstract

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Session 1330

Reliability, Validity, and Bias in Peer Evaluations of Self-Directed Interdependent Work Teams

Robert S. Thompson Colorado School of Mines

I. Introduction

Teamwork education has become increasingly important over the last decade. In a recent survey conducted at the Purdue School of Engineering, over 76% of the students responded that they had been involved as members of student work teams (486 out of 1,953 responded) 1. This emphasis on teamwork skills stems from the widespread use of teams in industry.

Peer evaluations are being used as a source of information for improving team performance 2,3 and accounting for individual contributions to a group project 4. Peer evaluations as a source of information for small self-directed group work have an appeal because the team members are in the best position to observe the team skills of their fellow team members. Despite this advantage, concerns have been levied against the use of peer evaluations. Abson 5, for example, suggested that peer evaluations can be abused and have undesirable effects on individuals in the group. Mathews 6 studied peer assessment of small group work in a management studies program. He noted patterns of response included giving all group members the same score, collusion between group members, and potential ganging-up on one member. Mathews also noted that perceptions can vary between people accounting for some of the variability. Mathews comments were based on his observations. He did not report any statistical data to support his claims.

Engineering design projects are a common source of teamwork in engineering education. Teamwork in these settings is characterized by three attributes: team members having a common goal, dependence on each other to achieve their goal, and intense work over an extended period of time. Because of the extended nature of the group work and the interdependence among team members, friendships have time to develop over the duration of the project. In many cases, friendships were formed prior to the group work. With this nature of the teamwork and the corresponding use of peer evaluations, there is a need for a better understanding of the reliability, validity, and bias in peer assessments of students working on these interdependent, self-directed, problem-solving teams. This research focuses on the specific problem of the reliability, validity, bias, and user acceptance of peer evaluations in small interdependent self-directed problem- solving teams in an educational setting. Generalizability theory techniques, structured interviews, and survey data were used to answer the following research questions: 1. What is the user reaction to peer evaluations? 2. What is the level of consensus in peer evaluations? 3. What is the level of bias in peer evaluations?

“Proceedings of the 2001 American Society for Engineering Education Annual Conference & Exposition Copyright 2001, American Society for Engineering Education

• Citation

• Format
  • APA
  • APA - LaTeX bibitem
  • MLA
  • MLA - LaTeX bibitem
  • Bibtex
  • EndNote - RIS

Thompson, R. (2001, June), Reliability, Validity, And Bias In Peer Evaluations Of Self Directed Interdependent Work Teams Paper presented at 2001 Annual Conference, Albuquerque, New Mexico. 10.18260/1-2--9729