Evaluating the Effects of Non-Anonymity on Student Team-Member Evaluations

Taylor Robert Smith; Rollin H. Hotchkiss

Download Paper | Permalink

Conference: 2014 ASEE Annual Conference & Exposition
Location: Indianapolis, Indiana
Publication Date: June 15, 2014
Start Date: June 15, 2014
End Date: June 18, 2014
ISSN: 2153-5965
Conference Session: Engineering Leadership Development Constituent Committee Division Poster Session
Tagged Division: Engineering Leadership Development Division
Page Count: 15
Page Numbers: 24.543.1 - 24.543.15
DOI: 10.18260/1-2--20434
Permanent URL: https://peer.asee.org/20434
Download Count: 489

Paper Authors

biography

Taylor Robert Smith Brigham Young University

visit author page

Taylor Smith has a Bachelor of Science and a Master's degree from the civil and environmental engineering program at Brigham Young University. For his graduate studies Taylor's classwork and technical emphasis were in geotechnical engineering; however, his Master's Thesis was non-technical and he examined and tested ways to improve performance through the use of peer feedback. More particularly, he evaluated the effect that having students conduct team member evaluations non-anonymously had upon personal and team effectiveness. His research is an original work (initiated by himself), which was funded by the BYU college of engineering. The contents of this paper are a condensed version of his thesis. Taylor, his wife Judy, and five children recently moved to the Washington, DC area where he is currently employed with Clark Construction.

visit author page

author page

Rollin H. Hotchkiss P.E.,D.WRE, F.ASCE Brigham Young University

Download Paper | Permalink

Abstract

ABSTRACT Evaluating the Effects of Non-Anonymity on Student Team-Member Evaluations Taylor SmithThis paper presents the findings of a study on the effects of non-anonymity upon student team-member evaluations; this is in contrast to the traditional method which has students rate theirfellow group members anonymously. More specifically, this study looked at the effects ofincreased accountability, and how to create conditions of openness and honesty in which studentswill readily give and receive authentic constructive criticism to and from their fellow groupmembers.The central hypothesis was that if students are taught and prepared to properly give and receiveconstructive criticism, and have multiple opportunities to do so over the course of a project, non-anonymous feedback is the most effective and desirable. In order to gauge the effects of non-anonymity, eight specific hypotheses relating to different aspects of the feedback process weretested. Predictions were made as to the effects upon the self-awareness and defensiveness ofthose who received feedback, the honesty and candor of those who provided it, as well as theeffect upon teams’ levels of trust and unity, and levels of performance.The statistical analysis showed that non-anonymity had no significant effect upon self-awareness, trust and unity, and performance. Significant differences were observed for honestyand candor, as well as defensiveness. Although some of these differences were in favor, otherswere contrary to the assumptions that were made. One of the results showed that at the beginningof the procedure, non-anonymous ratings were more lenient, but at the end of the process therewas no difference, as was expected. In regards to the overall process, non-anonymous studentsperceived ratings to be less honest and candid. A second conclusion was that non-anonymousstudents were actually more defensive towards negative feedback.In the end, there was no strong statistical evidence for or against non-anonymity, and thus itappears that there was no major treatment effect. There are two justifications as to why this maybe the case. These are based upon insights gained from the free-response section of a follow-upsurvey which the participants took. First, if non-anonymous feedback does indeed producepositive outcomes it may take a longer period of time for these differences to be noticed. Thisprocess took place over only about a three-month period, and feedback was received at intervalsof only 3-4 weeks apart. Secondly, when teams are small (i.e., only 3-5 members), it is difficultto maintain anonymity, which essentially removes the treatment.From these observations, the final recommendation of this report is that for students working insmall teams, non-anonymous feedback is preferable. This is because, as just noted, anonymity isdifficult to maintain even if it is a required condition. It seems that "pretending" that anonymityexists, when in fact it does not, actually hinders transparency and trust. Also, it seems that givingfeedback non-anonymously will more effectively prepare students for working on teams in theircareers, as this is more reflective of the way that feedback will be communicated in theworkforce.

Citation
Format

Smith, T. R., & Hotchkiss, R. H. (2014, June), Evaluating the Effects of Non-Anonymity on Student Team-Member Evaluations Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--20434

TY  - CPAPER
AB  -                                         ABSTRACT        Evaluating the Effects of Non-Anonymity on Student Team-Member Evaluations                                        Taylor SmithThis paper presents the findings of a study on the effects of non-anonymity upon student team-member evaluations; this is in contrast to the traditional method which has students rate theirfellow group members anonymously. More specifically, this study looked at the effects ofincreased accountability, and how to create conditions of openness and honesty in which studentswill readily give and receive authentic constructive criticism to and from their fellow groupmembers.The central hypothesis was that if students are taught and prepared to properly give and receiveconstructive criticism, and have multiple opportunities to do so over the course of a project, non-anonymous feedback is the most effective and desirable. In order to gauge the effects of non-anonymity, eight specific hypotheses relating to different aspects of the feedback process weretested. Predictions were made as to the effects upon the self-awareness and defensiveness ofthose who received feedback, the honesty and candor of those who provided it, as well as theeffect upon teams’ levels of trust and unity, and levels of performance.The statistical analysis showed that non-anonymity had no significant effect upon self-awareness, trust and unity, and performance. Significant differences were observed for honestyand candor, as well as defensiveness. Although some of these differences were in favor, otherswere contrary to the assumptions that were made. One of the results showed that at the beginningof the procedure, non-anonymous ratings were more lenient, but at the end of the process therewas no difference, as was expected. In regards to the overall process, non-anonymous studentsperceived ratings to be less honest and candid. A second conclusion was that non-anonymousstudents were actually more defensive towards negative feedback.In the end, there was no strong statistical evidence for or against non-anonymity, and thus itappears that there was no major treatment effect. There are two justifications as to why this maybe the case. These are based upon insights gained from the free-response section of a follow-upsurvey which the participants took. First, if non-anonymous feedback does indeed producepositive outcomes it may take a longer period of time for these differences to be noticed. Thisprocess took place over only about a three-month period, and feedback was received at intervalsof only 3-4 weeks apart. Secondly, when teams are small (i.e., only 3-5 members), it is difficultto maintain anonymity, which essentially removes the treatment.From these observations, the final recommendation of this report is that for students working insmall teams, non-anonymous feedback is preferable. This is because, as just noted, anonymity isdifficult to maintain even if it is a required condition. It seems that &quot;pretending&quot; that anonymityexists, when in fact it does not, actually hinders transparency and trust. Also, it seems that givingfeedback non-anonymously will more effectively prepare students for working on teams in theircareers, as this is more reflective of the way that feedback will be communicated in theworkforce.
AU  - Taylor Robert Smith
AU  - Rollin H. Hotchkiss P.E.,D.WRE, F.ASCE
CY  - Indianapolis, Indiana
DA  - 2014/06/15
PB  - ASEE Conferences
TI  - Evaluating the Effects of Non-Anonymity on Student Team-Member Evaluations
UR  - https://peer.asee.org/20434
DO  - 10.18260/1-2--20434
ER  -