June 15, 2014
June 15, 2014
June 18, 2014
Educational Research and Methods
24.26.1 - 24.26.10
Development of a Cognitive Model for Automatic Student Assessment: Classification of Student Errors in Engineering Statics and DynamicsThe taxonomy of human errors is critical for professions dealing with public safety such asengineering and healthcare. Once errors are identified, preventative and corrective measures canbe sought. From a pedagogical perspective, detection of error is also important. When studentsmake an error it is important to determine why the error was made and then begin the process ofcorrecting it by providing them with individualized feedback and assignments. For anexperienced educator, student assessment can be a fairly straightforward task. However, during atypical course, when a student is in the learning stage of development, assessment of thestudent's work is typically performed by a marker. This undergraduate or graduate student oftenhas teaching skills that are not fully developed and therefore may have difficulty properlyassessing the students' work. Guidelines or rules can be provided to the marker in the form of arubric but can be either too general or assignment specific. In addition, with decreasing educationbudgets and increasing popularity of distance learning in the form of massive open onlinecourses, automatic assessment methods are becoming increasingly important.This paper summarizes the errors that students made in two first year engineering final exams:engineering statics and dynamics. A cognitive model is then developed based on an errorclassification scheme rooted in Action Theory, which serves to classify errors as either mistakesor slips in logic. Simple rules are then developed into an algorithm for a computer to categorizeerrors based on a priori concepts such as: student experience level and priority level given on thecourse concept inventory as well as a posteriori concepts such as the frequency of the error. Theassessment algorithm is then “trained” using two final exams. The coefficients in the algorithmare optimized by comparing the results from experienced educators and that of the algorithm.The optimized algorithm is then tested on a third independent final exam and the resulting marksare compared. The cognitive model used is found to be general and easily extendible for use inother STEM courses. Results of the study show that the model performs well for probabilityregions where there is a clear distinction between slips and mistakes, but becomes moredependent on the model coefficients when the probability of the mistake, P(M)=0.5. Extension ofthis model is shown to be useful for automatically assessing a students' work as well as providingtimely and meaningful feedback.
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015