Asee peer logo

Work in Progress: Automating Anonymous Processing of Peer Evaluation Comments

Download Paper |

Conference

2020 ASEE Virtual Annual Conference Content Access

Location

Virtual On line

Publication Date

June 22, 2020

Start Date

June 22, 2020

End Date

June 26, 2021

Conference Session

First-year Programs: Teams and Teamwork

Tagged Division

First-Year Programs

Page Count

8

DOI

10.18260/1-2--35615

Permanent URL

https://peer.asee.org/35615

Download Count

163

Request a correction

Paper Authors

biography

Siqing Wei Purdue University, West Lafayette Orcid 16x16 orcid.org/0000-0002-7086-5953

visit author page

Siqing Weir received both bachelor’s and master’s degrees in electrical and Computer Engineering from Purdue University. He is currently pursuing a Ph.D. degree in Engineering Education at Purdue University. After years of experience of serving a peer teacher and a graduate teaching assistant in first-year-engineering courses, he is a research assistant at CATME research group studying the existence, causes and interventions on international engineering teamwork behaviors, the integration and implementation of team-based assignments and projects into STEM course designs and using mixed-method, especially natural language processing to student written research data, such as peer-to-peer comments. Siqing also works as the technical support manager at CATME research group.

visit author page

biography

Rui Wang Purdue University, West Lafayette

visit author page

Rui Wang is an undergraduate student at School of Electrical and Computer Engineering at Purdue University. His research interests include interpretable machine learning, robust computer vision and natural language processing.

visit author page

biography

Matthew W. Ohland Purdue University, West Lafayette Orcid 16x16 orcid.org/0000-0003-4052-1452

visit author page

Matthew W. Ohland is Associate Head and Professor of Engineering Education at Purdue University. He has degrees from Swarthmore College, Rensselaer Polytechnic Institute, and the University of Florida. His research on the longitudinal study of engineering students, team assignment, peer evaluation, and active and collaborative teaching methods has been supported by the National Science Foundation and the Sloan Foundation and his team received for the best paper published in the Journal of Engineering Education in 2008, 2011, and 2019 and from the IEEE Transactions on Education in 2011 and 2015. Dr. Ohland is an ABET Program Evaluator for ASEE. He was the 2002–2006 President of Tau Beta Pi and is a Fellow of the ASEE, IEEE, and AAAS.

visit author page

biography

Gaurav Nanda Purdue University, West Lafayette Orcid 16x16 orcid.org/0000-0003-1240-8639

visit author page

Dr. Gaurav Nanda is an Assistant Professor of Practice in the School of Engineering Technology at Purdue University. He completed his Ph.D. in Industrial Engineering from Purdue University and Masters and Bachelors from Indian Institute of Technology Kharagpur, India.

His research interests include application of text mining and machine learning methods to analyze real-world data. Currently, he is studying learner experiences in online courses by applying text mining approaches on user generated data such as discussion forums and open-ended feedback.

visit author page

Download Paper |

Abstract

De-identifying original qualitative datasets is time-consuming and expensive but required to protect the private information of studied subjects. Peer-to-peer comments are very likely to be collected by instructors as one form of peer evaluation in team-based learning courses. Those comments are essential and valuable research data for educational study to investigate but they usually contain identifiable entities, such as names. In this work in progress, we study and propose a pipeline tool to identify all names appearing in CATME team peer evaluation comments and replacing those names with pseudonyms such as Rater 1 and Rater 2. We explored several natural language processing techniques empowered by machine learning methods and then optimized to the final algorithm. At its core, the algorithm combines the long short-term memory (LSTM) and conditional random field (CRF) approaches most common in the field of named entity recognition. The current algorithm performs well, with a high recall of 0.8 with reasonable precision scores resulting in 76 of F_1 score as we want to put an emphasis on recalls. We also propose this as a tool to make a large amount of data available for research that would otherwise be sensitive due to personally identifiable information.

Wei, S., & Wang, R., & Ohland, M. W., & Nanda, G. (2020, June), Work in Progress: Automating Anonymous Processing of Peer Evaluation Comments Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--35615

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2020 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015