Asee peer logo

A Cognitive Model for Automatic Student Assessment: Classification of Errors in Engineering Dynamics

Download Paper |

Conference

2014 ASEE Annual Conference & Exposition

Location

Indianapolis, Indiana

Publication Date

June 15, 2014

Start Date

June 15, 2014

End Date

June 18, 2014

ISSN

2153-5965

Conference Session

Student Learning, Problem Solving, and Critical Thinking 1

Tagged Division

Educational Research and Methods

Page Count

10

Page Numbers

24.26.1 - 24.26.10

Permanent URL

https://peer.asee.org/19918

Download Count

30

Request a correction

Paper Authors

biography

Jeffrey A. Davis Grant MacEwan University

visit author page

With degrees in both civil and mechanical engineering, Jeff went on to obtain a Ph.D. from the Institute of Energy Technology at ETH Zurich in 2004. His past research includes dispersion of pollutants in rivers, and turbulent and multiphase flow modeling from a numerical perspective. Currently, Jeff is a first-year engineering instructor at MacEwan University. With a passion for teaching, his focus on research has turned to understanding and automating student assessment techniques as well as looking at the socioeconomic sustainability of educational institutions.

visit author page

biography

Shelley Lorimer P.Eng. Grant MacEwan University

visit author page

Dr. Shelley Lorimer, P.Eng., is chair of the bachelor of science in engineering transfer Program (BSEN) at Grant MacEwan University in Edmonton, Alberta. She teaches undergraduate courses in statics and dynamics, as well as courses in engineering professionalism. She currently is participating in a research project with Alberta Innovates – Technology Futures in the oil sands and hydrocarbon recovery group doing reservoir simulation of enhanced oil-recovery processes. She has a Ph.D. in numerical modeling from the University of Alberta, also in Edmonton.

visit author page

Download Paper |

Abstract

Development of a Cognitive Model for Automatic Student Assessment: Classification of Student Errors in Engineering Statics and DynamicsThe taxonomy of human errors is critical for professions dealing with public safety such asengineering and healthcare. Once errors are identified, preventative and corrective measures canbe sought. From a pedagogical perspective, detection of error is also important. When studentsmake an error it is important to determine why the error was made and then begin the process ofcorrecting it by providing them with individualized feedback and assignments. For anexperienced educator, student assessment can be a fairly straightforward task. However, during atypical course, when a student is in the learning stage of development, assessment of thestudent's work is typically performed by a marker. This undergraduate or graduate student oftenhas teaching skills that are not fully developed and therefore may have difficulty properlyassessing the students' work. Guidelines or rules can be provided to the marker in the form of arubric but can be either too general or assignment specific. In addition, with decreasing educationbudgets and increasing popularity of distance learning in the form of massive open onlinecourses, automatic assessment methods are becoming increasingly important.This paper summarizes the errors that students made in two first year engineering final exams:engineering statics and dynamics. A cognitive model is then developed based on an errorclassification scheme rooted in Action Theory, which serves to classify errors as either mistakesor slips in logic. Simple rules are then developed into an algorithm for a computer to categorizeerrors based on a priori concepts such as: student experience level and priority level given on thecourse concept inventory as well as a posteriori concepts such as the frequency of the error. Theassessment algorithm is then “trained” using two final exams. The coefficients in the algorithmare optimized by comparing the results from experienced educators and that of the algorithm.The optimized algorithm is then tested on a third independent final exam and the resulting marksare compared. The cognitive model used is found to be general and easily extendible for use inother STEM courses. Results of the study show that the model performs well for probabilityregions where there is a clear distinction between slips and mistakes, but becomes moredependent on the model coefficients when the probability of the mistake, P(M)=0.5. Extension ofthis model is shown to be useful for automatically assessing a students' work as well as providingtimely and meaningful feedback.

Davis, J. A., & Lorimer, S. (2014, June), A Cognitive Model for Automatic Student Assessment: Classification of Errors in Engineering Dynamics Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. https://peer.asee.org/19918

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015