Asee peer logo

WIP: Detection of Student Misconceptions of Electrical Circuit Concepts in a Short Answer Question Using NLP

Download Paper |

Conference

2021 ASEE Virtual Annual Conference Content Access

Location

Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

Electrical and Computer Division Poster Session

Tagged Division

Electrical and Computer

Page Count

13

Permanent URL

https://peer.asee.org/38076

Download Count

59

Request a correction

Paper Authors

biography

James P Becker Montana State University, Bozeman

visit author page

James Becker is a Professor of electrical and computer engineering at Montana State University. His professional interests include microwave circuits, radio frequency electronics, nanoelectronics, pedagogical research, and distance education.

visit author page

biography

Indika Kahanda University of North Florida Orcid 16x16 orcid.org/0000-0002-4536-6917

visit author page

Dr. Indika Kahanda is an Assistant Professor in the School of Computing at the University of North Florida, where he directs the bioinformatics, biomedical informatics and medical informatics lab. Prior to that Dr. Kahanda worked as an Assistant Professor in the Gianforte School of Computing at Montana State University. He received his Ph.D. in Computer Science from Colorado State University in 2016 in the area of Bioinformatics, a Master of Science in Computer Engineering from Purdue University in 2010, and a Bachelor of Science in Computer Engineering from University of Peradeniya, Sri Lanka in 2007.

visit author page

biography

Nazmul H. Kazi Montana State University

visit author page

Nazmul Kazi is a master's student of Computer Science at Montana State University. His research interests include the application of Artificial Intelligence, Deep Learning, Natural Language Processing, and Parallel Computing.

visit author page

Download Paper |

Abstract

While the use of writing exercises in gateway STEM courses that focus on solving numeric problems is not widespread, there is evidence that students could benefit from the addition of such exercises [1]. Writing exercises may be effective in both uncovering student misconceptions that are not necessarily apparent with typical computation problems, and as tools to foster conceptual change and metacognitive skill.

In this paper, pilot studies of the use of two Natural Language Processing (NLP) techniques to identify common misconceptions in the writing of students in a course on electric circuit analysis are described. Performance on the writing exercise in question has been shown to correlate with a student’s performance in the course [2]. This is of particular interest as the writing exercise has been administered during the fifth class period, sufficiently early to direct additional resources to the success of students appearing to be at-risk for failing the course. Realizing an automated software solution to analyze the responses to this exercise would remove burden on instructor time and open the door to immediate and personalized feedback to the student.

The first pilot study was run to determine how successful a simplistic rule-based approach would be in identifying the most common misconceptions found in a writing exercise requiring a student to speculate on the change in the power in the elements of a resistive circuit with a change to a single resistor value. An open-source NLP rule-based matching engine within spaCy [3] was used. The corpus consisted of one hundred and eighty-five unique responses to the question. Precision, recall, and F1-score [4] were used to assess the effectiveness of the rule-based NLP pipeline in comparison to that of a subject matter expert in identifying responses exemplifying seven misconceptions. Should this NLP pipeline be used in a system in which feedback is to be given to the student, a Directed Line of Reasoning (DLR) approach [5] would be beneficial in cases in which identification of a given misconception is in doubt. Considering this pilot study employed an extremely simplistic purely lexical-level rule-based classifier, the results are very promising and suggest the planned approach of developing a highly accurate, advanced rule-based classifier encompassing lexical/syntax/semantic driven rules is viable. As a compliment to the rule-based approach, this paper also describes a pilot study of the use of BERT (Bidirectional Encoder Representations from Transformers) [6], a machine learning approach that has shown tremendous promise in short-answer grading [7].

Becker, J. P., & Kahanda, I., & Kazi, N. H. (2021, July), WIP: Detection of Student Misconceptions of Electrical Circuit Concepts in a Short Answer Question Using NLP Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. https://peer.asee.org/38076

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015