Asee peer logo

Pilot Study on Applying Natural Language Processing Techniques to Classify Student Responses to Open-Ended Problems to Improve Peer Review Assignments

Download Paper |


2015 ASEE Annual Conference & Exposition


Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015





Conference Session

Innovative Use of Technology II

Tagged Division

Computers in Education

Tagged Topic


Page Count


Page Numbers

26.1227.1 - 26.1227.10



Permanent URL

Download Count


Request a correction

Paper Authors


Matthew A. Verleger Embry-Riddle Aeronautical University, Daytona Beach

visit author page

Matthew Verleger is an Assistant Professor of Engineering Fundamentals at Embry-Riddle Aeronautical University in Daytona Beach, Florida. His research interests are focused on using action research methodologies to develop immediate, measurable improvements in classroom instruction and the use of Model-Eliciting Activities (MEAs) in teaching students about engineering problem solving. Dr. Verleger is an active member of ASEE. He also serves as the developer and site manager for the Model-Eliciting Activities Learning System (, a site designed for implementing, managing, and researching MEAs in large classes.

visit author page

Download Paper |


Pilot Study on Applying Natural Language Processing to Classify Student Responses to Open-Ended ProblemsThe use of peer review is an essential part of the engineering design process. The AmericanSociety of Civil Engineers maintains an official policy, formally supporting the use of peerreview in engineering. As an educational tool, peer review can be a valuable way to providestudents feedback without a significant increase in instructor workload. Despite all that iscurrently known about our students, the best mechanism for assigning reviewers to reviewees ina peer review of artifacts is still considered to be blind, random assignment. The underlyingconjecture of this research project is that “there has to be a better way”. Specifically, if amechanism can be identified to accurately predict reviewer quality and reviewee need,complementing matches can be made – assigning high quality reviewers to high need revieweesand vice-versa.This paper represents a follow-up to XXXXX. That study presented the results of an attempt todevelop a classification schema using a large archival database of student work. This papertakes the resulting algorithms produced from that archival dataset and applies them to newstudent work, identifying how well the archival-based classification works on a similar data set.The archival data set came from a 2008 offering of a large (n = 1164 students split into sectionscapped at 120 students) “Introduction to Engineering” course at a large, public, mid-westuniversity. The new data set used the same problem with minimal changes, but was given in2014 to a small (n = 27 students) “Introduction to Engineering” course at a small, private,STEM+Business-only institution in the southeast.The paper will focus on the how the archive-based algorithm was designed, how it was appliedto the 2014 dataset, and the results of that application. The implications of that application onfuture algorithm design will be discussed as well as the next steps for the research.

Verleger, M. A. (2015, June), Pilot Study on Applying Natural Language Processing Techniques to Classify Student Responses to Open-Ended Problems to Improve Peer Review Assignments Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24564

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015