Asee peer logo

Does Neatness Count? What the Organization of Student Work Says About Understanding

Download Paper |


2012 ASEE Annual Conference & Exposition


San Antonio, Texas

Publication Date

June 10, 2012

Start Date

June 10, 2012

End Date

June 13, 2012



Conference Session

Understanding Our Students II

Tagged Division

Educational Research and Methods

Page Count


Page Numbers

25.478.1 - 25.478.12



Permanent URL

Download Count


Request a correction

Paper Authors


Timothy S. Van Arsdale University of California, Riverside

visit author page

Timothy Van Arsdale earned his B.S. in engineering from Walla Walla University in 2010. He is currently a Ph.D. student in mechanical wngineering at the University of California, Riverside.

visit author page


Thomas Stahovich University of California, Riverside

visit author page

Thomas Stahovich received a B.S. in mechanical engineering from the University of California, Berkeley, in 1988. He received a M.S. and Ph.D. in mechanical engineering from the Massachusetts Institute of Technology in 1990 and 1995, respectively. He is currently Chair and professor in the Mechanical Engineering Department at the University of California, Riverside.

visit author page

Download Paper |


Does Neatness Count? What the Organization of Student Work Says About UnderstandingStudents have long been taught that neatness counts. But does it? In this project, we seek tounderstand how the organization of a student’s solution to a homework problem relates to thequality. More precisely, we seek to understand how the history of the solution constructionprocess correlates with the correctness of the work. Understanding this relationship will enableus to create software to provide early warnings to students who may be struggling in a course.We have conducted a study in which students in an undergraduate statics course completed all oftheir work, including homework, quizzes, and exams, using LivescribeTM “Smartpens.” TheSmartpens record the solutions as time-stamped pen strokes, enabling us to see not only the finalink on the page, but also the order in which it was written. We characterize the solution historieswith a number of quantitative features describing the temporal and spatial organization of thework. For example, there are features that describe the order in which various problem solvingtasks, such as the construction of free body diagrams or equilibrium equations, are performed,and the amount of time spent on each task. Because Smartpens use ink, students cannot erasetheir errors and must cross them out. We characterize cross-outs by the delay between when theink was written and when it was crossed out. The spatial organization of the work ischaracterized by the extent to which a student revisits earlier parts of a solution to add additionalink. None of the features consider the actual correctness of the work.We have developed automatic techniques for computing these features. Many can be computeddirectly from a student’s raw pen stroke data. Others require the ink to be semantically labeled,for example, as free body diagram strokes or equation strokes. (In other work, we are developingtechniques for automatically identifying the correct semantic label for a pen stroke.) In oneanalysis, we have found that 45% of the variation in student grades can be explained using thefeatures of the solution history. The correlation is only slightly less if features that rely onsemantic labeling of the strokes are excluded. We are also using our features to train a classifierto predict student grades on individual assignments. Our initial results are quite promising.

Van Arsdale, T. S., & Stahovich, T. (2012, June), Does Neatness Count? What the Organization of Student Work Says About Understanding Paper presented at 2012 ASEE Annual Conference & Exposition, San Antonio, Texas. 10.18260/1-2--21236

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2012 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015