June 14, 2015
June 14, 2015
June 17, 2015
Computers in Education
26.1227.1 - 26.1227.10
Pilot Study on Applying Natural Language Processing to Classify Student Responses to Open-Ended ProblemsThe use of peer review is an essential part of the engineering design process. The AmericanSociety of Civil Engineers maintains an official policy, formally supporting the use of peerreview in engineering. As an educational tool, peer review can be a valuable way to providestudents feedback without a significant increase in instructor workload. Despite all that iscurrently known about our students, the best mechanism for assigning reviewers to reviewees ina peer review of artifacts is still considered to be blind, random assignment. The underlyingconjecture of this research project is that “there has to be a better way”. Specifically, if amechanism can be identified to accurately predict reviewer quality and reviewee need,complementing matches can be made – assigning high quality reviewers to high need revieweesand vice-versa.This paper represents a follow-up to XXXXX. That study presented the results of an attempt todevelop a classification schema using a large archival database of student work. This papertakes the resulting algorithms produced from that archival dataset and applies them to newstudent work, identifying how well the archival-based classification works on a similar data set.The archival data set came from a 2008 offering of a large (n = 1164 students split into sectionscapped at 120 students) “Introduction to Engineering” course at a large, public, mid-westuniversity. The new data set used the same problem with minimal changes, but was given in2014 to a small (n = 27 students) “Introduction to Engineering” course at a small, private,STEM+Business-only institution in the southeast.The paper will focus on the how the archive-based algorithm was designed, how it was appliedto the 2014 dataset, and the results of that application. The implications of that application onfuture algorithm design will be discussed as well as the next steps for the research.
Verleger, M. A. (2015, June), Pilot Study on Applying Natural Language Processing Techniques to Classify Student Responses to Open-Ended Problems to Improve Peer Review Assignments Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24564
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015