San Antonio, Texas
June 10, 2012
June 10, 2012
June 13, 2012
2153-5965
Educational Research and Methods
12
25.379.1 - 25.379.12
10.18260/1-2--21137
https://peer.asee.org/21137
505
Luanna Prevost is a Postdoctoral Research Associate at the Center for Engineering Education Research (CEER) at Michigan State University. She is a member of the Automated Analysis of Constructed Responses program, an NSF-funded cross-institutional collaboration of interdisciplinary science, technology, engineering, and mathematics (STEM) education researchers interested in exploring the use of automated text analysis to evaluate constructed-response assessments. Her research activities focus on instructional material development, learning assessment, and investigating student cognition STEM disciplines.
Deciphering student ideas on thermodynamics using computerized lexical analysis of student writingConstructed responses, in which students describe their understanding in their own language,provide better insight into their thinking than do multiple-choice assessments. However,constructed responses are not often employed in large enrollment courses due to the time andfinancial constraints involved in grading these assessments. In this study, we examined studentunderstanding of thermodynamics using computerized lexical analysis of constructed responsesin a large enrollment course (N=384). Students were asked to interpret a graph depicting changesin free energy during the course of a reaction using both multiple-choice and open-endedresponses. Opened ended responses were analyzed using SPSS Text Analytics for Surveys(TAFS). The software extracts scientific terms from the students’ writing and places them intocategories using custom dictionaries of science terms. We validated the automated lexicalanalysis by using the categories created by TAFS as independent variables in discriminantanalysis to predict expert scoring of the students’ writing. Our findings to date reveal i) thatstudents hold a mix of correct and incorrect ideas about thermodynamics, and ii) that this resultis masked by multiple-choice testing. Over 50% of the students answering multiple-choicecorrectly displayed incorrect, or both correct and incorrect conceptualizations in their open-ended responses. Our results support previous studies that have uncovered students’heterogeneous ideas about matter and energy conservation and acid-base chemistry using lexicalanalysis. These findings suggest that computerized lexical analysis can improve instructors’understanding of the heterogeneity of ideas that student harbor about key concepts in STEMdisciplines and inform assessment practices.
Prevost, L. B., & Haudek, K. C., & Merrill, J. E., & Urban-Lurain, M. (2012, June), Deciphering Student Ideas on Thermodynamics Using Computerized Lexical Analysis of Student Writing Paper presented at 2012 ASEE Annual Conference & Exposition, San Antonio, Texas. 10.18260/1-2--21137
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2012 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015