Asee peer logo

Improving Efficacy of Peer-Evaluation in Team Project Scenarios

Download Paper |

Conference

2011 ASEE Annual Conference & Exposition

Location

Vancouver, BC

Publication Date

June 26, 2011

Start Date

June 26, 2011

End Date

June 29, 2011

ISSN

2153-5965

Conference Session

They're Not "Soft" Skills!

Tagged Division

Educational Research and Methods

Page Count

16

Page Numbers

22.832.1 - 22.832.16

Permanent URL

https://peer.asee.org/18113

Download Count

30

Request a correction

Paper Authors

biography

Eckehard Doerry Northern Arizona University

visit author page

Eck Doerry is an Associate Professor of Computer Science at Northern Arizona University. His research interests fall within the broad area on “Groupware support for Online Groups”, with active research in portal-based tools to support distributed scientific communities, groupware tools to support small, distributed engineering design teams, and distance education tools and environments. He has been a long-time advocate of realistic, interdisciplinary team design projects as a key element in engineering education, and has been managing advanced project teams in the Design4Practice program at NAU for nearly ten years.

visit author page

biography

James Dean Palmer Northern Arizona University

visit author page

Dr. Palmer is an Assistant Professor at Northern Arizona University where his research interests include undergraduate computer science education, language design, and computational storytelling.

visit author page

Download Paper |

Abstract

Tracking Individual Performance in Team Project ScenariosThe past decades have brought a growing awareness of the value of integrating training in abroad range of “soft skills” – including teaming, project management, and oral and writtencommunication – into the modern engineering curriculum. As a result, the vast majority ofengineering programs now include one or more project courses in the curricular core, in whichstudents work in teams to solve realistic engineering challenges.This renewed emphasis on team-based project work, along with recent ABET guidelinesrequiring programs to document learning outcomes at the individual level, has exacerbated anage-old problem with team-based projects: how to reliably measure individual contributions togroup performance in a team-based project scenario. In the worst case (often called “coat-tailing”), one or more team members contribute little or nothing, forcing their teammates to pickup the slack — and yet all team members end up receiving the same “team grade” when theproject concludes.In general, the problem is that, although team projects undoubtedly have strong potential fordeveloping critical teaming skills (e.g., project management; task decomposition, distribution,and monitoring; and team communication), individual performance in these skill areas is noteasily accessible to the instructor; monitoring and documenting the extent to which individuallearning actually occurs (e.g., for grading, for ABET) in team project contexts represents adifficult challenge.In this paper, we review existing approaches to evaluating student performance in project teams,then build on this to describe our own experiences in our team-based Capstone course over thepast decade. In particular, we describe an effective system we have developed to gain insightinto internal team dynamics and individual performance, based on structured task reports andpeer evaluations. Earlier evolutionary variants of our system and their weaknesses are describedas well, to rationalize and motivate the development of our current approach. Discussion issupported by examples and analyses from data collected over the years; by juxtaposing thesedata with actual problems encountered in corresponding teams, we are even able to identifyseveral typical “trouble profiles” for dysfunctional teams. Benefits include early detection ofinternal team difficulties, a low-overhead system for documenting individual contributions inteam contexts, and a strong mechanism for differentiating student grades based on individualperformance. Using our system, it is quite possible for one team member to receive a D, whileanother receives an A and team deliverables as a whole receive a B.We conclude by describing our current grant-funded effort to implement our project teammanagement system within a secure web portal linked to existing project course websites. Ourgoal is to fully-automate the team management process, further reducing the overhead on theinstructor while allowing real-time feedback to team members and automatically generatingdocumentation on contributions/performance for individual team members. This will also allowus to explore the intriguing potential for fuzzy detection of the “trouble profiles” mentionedabove, allowing the system to automatically flag developing problems for instructor attention.

Doerry, E., & Palmer, J. D. (2011, June), Improving Efficacy of Peer-Evaluation in Team Project Scenarios Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. https://peer.asee.org/18113

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2011 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015