Albuquerque, New Mexico
June 24, 2001
June 24, 2001
June 27, 2001
6.224.1 - 6.224.8
Assignment and Quality Control of Peer Reviewers Edward F. Gehringer North Carolina State University firstname.lastname@example.org
Much work has been performed on assessing the validity and usefulness of peer assessment in the classroom. Much less effort has been invested in enumerating and classifying strategies for assigning reviewers, encouraging good feedback, and preventing clustering of grades. This study reviews the different approaches that have been taken to those problems. Several of these strategies are employed in PG, our Web-based application for peer review and peer grading.
Usually, students are assigned randomly to review other students’ work. Often, students work in teams, with each member of the team reviewing the other members. Or, students or teams may choose from a list of topics to work on. In this case, it is helpful to assign students to review others who have chosen the same topic. To encourage students to provide adequate feedback to their reviewees, several approaches can be taken. Students can be denied credit for the assignment unless they do the required reviews. Or, they can be prevented from seeing feedback on their work until they provide feedback to others. Multiple review periods may be employed, with students required to give some feedback in each period. A formula may be devised to allow reviewers to share in good grades received by their reviewees. Or students may be assigned to review each other’s reviews.
To improve the accuracy of grading, students can be required to pass a pre-certification test before being allowed to serve as peer graders. The instructor can supply a set of grading criteria, and discuss it with the students, either in advance or after the students complete their first round of review. Reviewer mappings can be constrained to assure that each student will review one paper from each quartile (etc.) of the class.
Peer review in the classroom is a technique that is becoming increasingly popular, with over 100 papers published on the topic in the past ten years. Much work has been performed on assessing usefulness of the technique (students generally like it, and learn well from it) and its validity (students do in general rate better work more highly, though some effort needs to be invested in the assessment procedure to assure this). However, very few published reports discuss appropriate strategies for matching reviewers with reviewees, how students can be induced to give good feedback, or how student-assigned grades can be prevented from clustering closely around the mean. These topics are the focus of this review paper.
2. Strategies for assigning reviewers
In his 1998 survey paper [Topp 98], Topping says, “How peer assessors and assessees should best be matched ... is discussed surprisingly little in the literature.” In most cases, he says, a
Proceedings of the 2001 American Society for Engineering Education Annual Conference & Exposition Copyright 2001, American Society for Engineering Education
Gehringer, E. (2001, June), Assignment And Quality Control Of Peer Reviewers Paper presented at 2001 Annual Conference, Albuquerque, New Mexico. https://peer.asee.org/8941
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2001 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015