Asee peer logo

Nine Years of Calibrated Peer Review in Rhetoric and Engineering Design

Download Paper |


2011 ASEE Annual Conference & Exposition


Vancouver, BC

Publication Date

June 26, 2011

Start Date

June 26, 2011

End Date

June 29, 2011



Conference Session

NSF Grantees Poster Session

Tagged Topic

NSF Grantees

Page Count


Page Numbers

22.1102.1 - 22.1102.10



Permanent URL

Download Count


Request a correction

Paper Authors


Patricia A. Carlson Rose-Hulman Institute of Technology

visit author page

Patricia A. Carlson received the B.A. from the College of William and Mary and the M.A. and Ph.D. from Duke University. She came to Rose-Hulman early in her teaching career and has taught a wide variety of courses. She is currently pursuing research interests in educational applications for Commmunication and Information Technology (CIT) Pat has held a number of American Society for Engineering Education summer fellowships that have taken her to NASA-Goddard, NASA-Langley, the Army Research Laboratory in Aberdeen, Maryland, and NASA’s Classroom of the Future in Wheeling, WV. She was on loan to the Air Force Human Resources Laboratory from 1989 to 1995, managing a project to transition advanced instructional technologies to ten different middle schools located in five states. She is on the editorial board of three professional publications and has served as National Research Council Senior Fellow assigned to the Air Force Human Resources Laboratory. In her spare time, Pat enjoys reading and gardening.

visit author page

Download Paper |


The paper (and accompanying poster) draws from our extended experience in using CalibratedPeer Review in the classroom. The work is based upon findings from three NSF grants(#9980867, #0404923, and #0816849), spanning a period from 2002 – 2010.Developed by the Division of Molecular Sciences at UCLA (through an NSF grant), CPR™ is anexcellent "learning environment" that creates an electronic, asynchronous, discipline-independent platform for creating, implementing, and evaluating communication assignments(both written and visual), without significantly increasing the instructor’s workload. Theextensive data collected by the "environment" can be used to measure learning outcomes. WhereCPR™ is used in multi-sectioned courses, data can be merged. The flexibility and versatility ofthe platform make it very appropriate as a data collection tool for ABET accreditation criteria.We report on lessons learned and make both strategic and tactical suggestions for implementingthis versatile tool in the engineering classroom. We focus on three categories:Designing Assignments: As a guide in considering the types of communication that might beused to teach aspects of engineering, we present a loose taxonomy of potential assignmentsranging from the more exploratory to the more codified.Using CPR Data: CPR’s built-in data collection provides a range of in-situ observations onstudent performance at eleven separate points during the four workspaces. We give illustrationsusing both descriptive and inferential statistical methods on these data.Nurturing Commentary: During both the calibration phase and the peer review phase,assignments can be constructed so that students must provide a narrative comment to supporttheir quantitative answers to the various elements of a rubric. We suggest ways in which thismore qualitative feedback can be used to foster collaboration and mature communication.Our experience with Calibrated Peer Review™ (CPR™) in several courses at Rose-HulmanInstitute of Technology suggests that this robust instructional technology partners both with theinstructor and with the student to -- Increase competence, creativity, and confidence in exploratory inquiry and reasoning. Promote and sustain interest in engineering practice. Engage the user and transfer powerful strategies for problem-solving. Bridge the gap between process (thinking) and product (writing). Improve quality of writing by improving quality of thinking.

Carlson, P. A. (2011, June), Nine Years of Calibrated Peer Review in Rhetoric and Engineering Design Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. 10.18260/1-2--18974

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2011 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015