Charlotte, North Carolina
June 20, 1999
June 20, 1999
June 23, 1999
4.112.1 - 4.112.12
Building Better Teamwork Assessments: A Process for Improving the Validity and Sensitivity of Self/Peer Ratings Eric Van Duzer and Flora McMartin
Abstract: A process employing both quantitative and qualitative methods was developed to improve the validity and sensitivity of self/peer ratings in assessing teamwork skills. Preliminary results indicate a dramatic improvement in the sensitivity of scales in measuring differences between student skill levels. The data also indicate that the process improves the validity of the ratings in measuring what the developers intended.
Traditional engineering education emphasized individualism, in contrast current practice increasingly involves team projects, cooperative learning and an emphasis on the synergy possible through group processes. Most faculty who interact regularly with their students have a general sense of a student’s teamwork skills. However, in order for faculty to develop effective interventions, it is necessary for them to measure the underlying skills that contribute to successful teamwork. Building on a number of existing instruments (e.g., the Foundation Coalition’s Team Evaluation Sheet) the Synthesis Coalition’s assessment team developed a self/peer assessment instrument that incorporated the best practices in engineering and other disciplines. This instrument was initially pilot tested in the Fall of 1997, revised in the following spring and a follow-up pilot test was conducted in the summer of 1998. The revision process outlined in this paper significantly improved the sensitivity and validity of the teamwork instrument. It provides a model process for developing local self/peer assessment instruments.
Developing the Instrument
In order to determine which characteristics of self/peer assessment instruments would improve their validity and reliability, a review of the literature on self-assessment and peer evaluation was conducted. That literature revealed two primary types of bias, self enhancement and downward comparison, that can distort self-ratings on assessment instruments (Mabe & West, 1982; Groeger & Grande, 1996).
Self enhancement is the unreasonably optimistic self appraisal that may be triggered by threats to self-esteem, for example, by asking nursing students to rate their level of competence in treating their patients. Downward comparison, is a general tendency for positive self-bias and negative other bias when self-evaluation involves social comparisons, such as “Compared to other freshman”. The literature indicates that the effects of these biases can be reduced and thus enhance response reliability and validity by: • Using explicit language shared by respondents and testers in defining the traits and the criteria.
McMartin, F. (1999, June), Building Better Teamwork Assessments: A Process For Improving The Validity And Sensitivity Of Self/Peer Ratings Paper presented at 1999 Annual Conference, Charlotte, North Carolina. https://peer.asee.org/7962
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 1999 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015