Paper Authors

Abstract

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

I:\Submission\ASEE-393.doc

Building Better Teamwork Assessments: A Process for Improving the Validity and Sensitivity of Self/Peer Ratings Eric Van Duzer and Flora McMartin

Abstract: A process employing both quantitative and qualitative methods was developed to improve the validity and sensitivity of self/peer ratings in assessing teamwork skills. Preliminary results indicate a dramatic improvement in the sensitivity of scales in measuring differences between student skill levels. The data also indicate that the process improves the validity of the ratings in measuring what the developers intended.

Introduction

Traditional engineering education emphasized individualism, in contrast current practice increasingly involves team projects, cooperative learning and an emphasis on the synergy possible through group processes. Most faculty who interact regularly with their students have a general sense of a student’s teamwork skills. However, in order for faculty to develop effective interventions, it is necessary for them to measure the underlying skills that contribute to successful teamwork. Building on a number of existing instruments (e.g., the Foundation Coalition’s Team Evaluation Sheet) the Synthesis Coalition’s assessment team developed a self/peer assessment instrument that incorporated the best practices in engineering and other disciplines. This instrument was initially pilot tested in the Fall of 1997, revised in the following spring and a follow-up pilot test was conducted in the summer of 1998. The revision process outlined in this paper significantly improved the sensitivity and validity of the teamwork instrument. It provides a model process for developing local self/peer assessment instruments.

Developing the Instrument

In order to determine which characteristics of self/peer assessment instruments would improve their validity and reliability, a review of the literature on self-assessment and peer evaluation was conducted. That literature revealed two primary types of bias, self enhancement and downward comparison, that can distort self-ratings on assessment instruments (Mabe & West, 1982; Groeger & Grande, 1996).

Self enhancement is the unreasonably optimistic self appraisal that may be triggered by threats to self-esteem, for example, by asking nursing students to rate their level of competence in treating their patients. Downward comparison, is a general tendency for positive self-bias and negative other bias when self-evaluation involves social comparisons, such as “Compared to other freshman”. The literature indicates that the effects of these biases can be reduced and thus enhance response reliability and validity by: • Using explicit language shared by respondents and testers in defining the traits and the criteria.

OVER

• Citation

• Format
  • APA
  • APA - LaTeX bibitem
  • MLA
  • MLA - LaTeX bibitem
  • Bibtex
  • EndNote - RIS

McMartin, F. (1999, June), Building Better Teamwork Assessments: A Process For Improving The Validity And Sensitivity Of Self/Peer Ratings Paper presented at 1999 Annual Conference, Charlotte, North Carolina. 10.18260/1-2--7962