Paper Authors

Abstract

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Session 3230

Assignment and Quality Control of Peer Reviewers Edward F. Gehringer North Carolina State University efg@ncsu.edu

Abstract

Much work has been performed on assessing the validity and usefulness of peer assessment in the classroom. Much less effort has been invested in enumerating and classifying strategies for assigning reviewers, encouraging good feedback, and preventing clustering of grades. This study reviews the different approaches that have been taken to those problems. Several of these strategies are employed in PG, our Web-based application for peer review and peer grading.

Usually, students are assigned randomly to review other students’ work. Often, students work in teams, with each member of the team reviewing the other members. Or, students or teams may choose from a list of topics to work on. In this case, it is helpful to assign students to review others who have chosen the same topic. To encourage students to provide adequate feedback to their reviewees, several approaches can be taken. Students can be denied credit for the assignment unless they do the required reviews. Or, they can be prevented from seeing feedback on their work until they provide feedback to others. Multiple review periods may be employed, with students required to give some feedback in each period. A formula may be devised to allow reviewers to share in good grades received by their reviewees. Or students may be assigned to review each other’s reviews.

To improve the accuracy of grading, students can be required to pass a pre-certification test before being allowed to serve as peer graders. The instructor can supply a set of grading criteria, and discuss it with the students, either in advance or after the students complete their first round of review. Reviewer mappings can be constrained to assure that each student will review one paper from each quartile (etc.) of the class.

1. Introduction

Peer review in the classroom is a technique that is becoming increasingly popular, with over 100 papers published on the topic in the past ten years. Much work has been performed on assessing usefulness of the technique (students generally like it, and learn well from it) and its validity (students do in general rate better work more highly, though some effort needs to be invested in the assessment procedure to assure this). However, very few published reports discuss appropriate strategies for matching reviewers with reviewees, how students can be induced to give good feedback, or how student-assigned grades can be prevented from clustering closely around the mean. These topics are the focus of this review paper.

2. Strategies for assigning reviewers

In his 1998 survey paper [Topp 98], Topping says, “How peer assessors and assessees should best be matched ... is discussed surprisingly little in the literature.” In most cases, he says, a

Proceedings of the 2001 American Society for Engineering Education Annual Conference & Exposition Copyright  2001, American Society for Engineering Education

• Citation

• Format
  • APA
  • APA - LaTeX bibitem
  • MLA
  • MLA - LaTeX bibitem
  • Bibtex
  • EndNote - RIS

Gehringer, E. (2001, June), Assignment And Quality Control Of Peer Reviewers Paper presented at 2001 Annual Conference, Albuquerque, New Mexico. 10.18260/1-2--8941