Does Neatness Count?  What the Organization of Student Work Says About Understanding

Timothy S. Van Arsdale; Thomas Stahovich

Download Paper | Permalink

Conference: 2012 ASEE Annual Conference & Exposition
Location: San Antonio, Texas
Publication Date: June 10, 2012
Start Date: June 10, 2012
End Date: June 13, 2012
ISSN: 2153-5965
Conference Session: Understanding Our Students II
Tagged Division: Educational Research and Methods
Page Count: 12
Page Numbers: 25.478.1 - 25.478.12
DOI: 10.18260/1-2--21236
Permanent URL: https://peer.asee.org/21236
Download Count: 726

Paper Authors

biography

Timothy S. Van Arsdale University of California, Riverside

visit author page

Timothy Van Arsdale earned his B.S. in engineering from Walla Walla University in 2010. He is currently a Ph.D. student in mechanical wngineering at the University of California, Riverside.

visit author page

biography

Thomas Stahovich University of California, Riverside

visit author page

Thomas Stahovich received a B.S. in mechanical engineering from the University of California, Berkeley, in 1988. He received a M.S. and Ph.D. in mechanical engineering from the Massachusetts Institute of Technology in 1990 and 1995, respectively. He is currently Chair and professor in the Mechanical Engineering Department at the University of California, Riverside.

visit author page

Download Paper | Permalink

Abstract

Does Neatness Count? What the Organization of Student Work Says About UnderstandingStudents have long been taught that neatness counts. But does it? In this project, we seek tounderstand how the organization of a student’s solution to a homework problem relates to thequality. More precisely, we seek to understand how the history of the solution constructionprocess correlates with the correctness of the work. Understanding this relationship will enableus to create software to provide early warnings to students who may be struggling in a course.We have conducted a study in which students in an undergraduate statics course completed all oftheir work, including homework, quizzes, and exams, using LivescribeTM “Smartpens.” TheSmartpens record the solutions as time-stamped pen strokes, enabling us to see not only the finalink on the page, but also the order in which it was written. We characterize the solution historieswith a number of quantitative features describing the temporal and spatial organization of thework. For example, there are features that describe the order in which various problem solvingtasks, such as the construction of free body diagrams or equilibrium equations, are performed,and the amount of time spent on each task. Because Smartpens use ink, students cannot erasetheir errors and must cross them out. We characterize cross-outs by the delay between when theink was written and when it was crossed out. The spatial organization of the work ischaracterized by the extent to which a student revisits earlier parts of a solution to add additionalink. None of the features consider the actual correctness of the work.We have developed automatic techniques for computing these features. Many can be computeddirectly from a student’s raw pen stroke data. Others require the ink to be semantically labeled,for example, as free body diagram strokes or equation strokes. (In other work, we are developingtechniques for automatically identifying the correct semantic label for a pen stroke.) In oneanalysis, we have found that 45% of the variation in student grades can be explained using thefeatures of the solution history. The correlation is only slightly less if features that rely onsemantic labeling of the strokes are excluded. We are also using our features to train a classifierto predict student grades on individual assignments. Our initial results are quite promising.

Citation
Format

Van Arsdale, T. S., & Stahovich, T. (2012, June), Does Neatness Count? What the Organization of Student Work Says About Understanding Paper presented at 2012 ASEE Annual Conference & Exposition, San Antonio, Texas. 10.18260/1-2--21236

TY - CPAPER
AB - Does Neatness Count? What the Organization of Student Work Says About UnderstandingStudents have long been taught that neatness counts. But does it? In this project, we seek tounderstand how the organization of a student’s solution to a homework problem relates to thequality. More precisely, we seek to understand how the history of the solution constructionprocess correlates with the correctness of the work. Understanding this relationship will enableus to create software to provide early warnings to students who may be struggling in a course.We have conducted a study in which students in an undergraduate statics course completed all oftheir work, including homework, quizzes, and exams, using LivescribeTM “Smartpens.” TheSmartpens record the solutions as time-stamped pen strokes, enabling us to see not only the finalink on the page, but also the order in which it was written. We characterize the solution historieswith a number of quantitative features describing the temporal and spatial organization of thework. For example, there are features that describe the order in which various problem solvingtasks, such as the construction of free body diagrams or equilibrium equations, are performed,and the amount of time spent on each task. Because Smartpens use ink, students cannot erasetheir errors and must cross them out. We characterize cross-outs by the delay between when theink was written and when it was crossed out. The spatial organization of the work ischaracterized by the extent to which a student revisits earlier parts of a solution to add additionalink. None of the features consider the actual correctness of the work.We have developed automatic techniques for computing these features. Many can be computeddirectly from a student’s raw pen stroke data. Others require the ink to be semantically labeled,for example, as free body diagram strokes or equation strokes. (In other work, we are developingtechniques for automatically identifying the correct semantic label for a pen stroke.) In oneanalysis, we have found that 45% of the variation in student grades can be explained using thefeatures of the solution history. The correlation is only slightly less if features that rely onsemantic labeling of the strokes are excluded. We are also using our features to train a classifierto predict student grades on individual assignments. Our initial results are quite promising.
AU - Timothy S. Van Arsdale
AU - Thomas Stahovich
CY - San Antonio, Texas
DA - 2012/06/10
PB - ASEE Conferences
TI - Does Neatness Count? What the Organization of Student Work Says About Understanding
UR - https://peer.asee.org/21236
DO - 10.18260/1-2--21236
ER -