Paper Authors

Abstract

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Main Menu Session 2793

An Effective Strategy for Dynamic Mapping of Peer Reviewers Edward F. Gehringer Yun Cui North Carolina State University {efg@ncsu.edu, ycui4@unity.ncsu.edu}

Abstract

Although much work has been done on peer review in an academic setting, virtually no work has been done on strategies for mapping reviewers. In most cases, an instructor merely collects papers, shuffles them, and passes them out to students for peer review. When peer review is electronic and asynchronous, better strategies are needed. A random mapping, arranged statically in advance, is frustrated by students who drop the course during the assignment period, or who do not submit their assignments. The result is that some students get reviewed by more reviewers than other students, and some students do not get reviewed at all.

We have developed a dynamic strategy that adapts to situations like students dropping the course or not doing their assigned reviews, to ensure that all students’ work is reviewed by approximately the same number of other students, and that no students are assigned to review their own work. The strategy is very similar to the Banker’s Algorithm for allocating resources in a computer system. It assigns reviewers in such a way as to maximize the chance that all future mapping assignments will be able to be assigned successfully (i.e., without assigning a student to review him/herself) The strategy has been extended to second-level reviews—that is, where one student reviews another student’s review, to give students an incentive to do a careful job of reviewing It has also been extended to the situation where team projects are reviewed by individual students This strategy offers great advantages for peer review of student work, and opportunity for extension to other problems like asynchronous review periods, where students themselves negotiate review deadlines.

1. Introduction

Peer review in the classroom is a technique that is becoming increasingly popular, with over 100 papers published on the topic in the past ten years. Much work has been performed on assessing usefulness of the technique (students generally like it, and learn well from it) and its validity (students do in general rate better work more highly, though some effort needs to be invested in the assessment procedure to assure this). However, very few published reports discuss appropriate strategies for matching reviewers with reviewees. In his 1998 survey paper [Topp 98], Topping says, “How peer assessors and assessees should best be matched ... is discu ssed surprisingly little in the literature.” In most cases, he says, a single assessor was matched with an assessee. Most papers omit entirely any indication of how reviewers are chosen; others just say they are matched “randomly.” In other cases, multiple assessors were used.

Random matching is relatively easy to do. In an offline environment where papers are collected at the start of a class period and then shuffled, it can be guaranteed that each student will get a paper to review, and each student will be reviewed by another student. When review takes place

Proceedings of the 2002 American Society for Engineering Education Annual Conference & Exposition Copyright Ó 2002, American Society for Engineering Education

Main Menu

• Citation

• Format
  • APA
  • APA - LaTeX bibitem
  • MLA
  • MLA - LaTeX bibitem
  • Bibtex
  • EndNote - RIS

Cui, Y., & Gehringer, E. (2002, June), An Effective Strategy For Dynamic Mapping Of Peer Reviewers Paper presented at 2002 Annual Conference, Montreal, Canada. 10.18260/1-2--10645