Asee peer logo

Keylogging in a Web-Based Code Editor for Fine-Grained Analysis and Early Prediction of Student Performance

Download Paper |

Conference

2024 ASEE Annual Conference & Exposition

Location

Portland, Oregon

Publication Date

June 23, 2024

Start Date

June 23, 2024

End Date

June 26, 2024

Conference Session

Educational Research and Methods Division (ERM) Technical Session 17

Tagged Division

Educational Research and Methods Division (ERM)

Page Count

8

DOI

10.18260/1-2--47709

Permanent URL

https://peer.asee.org/47709

Download Count

67

Paper Authors

author page

Xavier Rene Plourde University of California, Berkeley

biography

Garrett Ethan Katz Syracuse University

visit author page

Garrett Katz is an assistant professor at Syracuse University. He teaches a broad range of computer science courses, covering introductory programming, discrete math, introductory artificial intelligence, and graduate seminars. His research covers various topics in artificial intelligence and human-machine interaction, including in educational contexts. In particular, his recent work investigates reasoning and learning processes underlying program synthesis, both for automated program synthesis by machines as well as manual program synthesis by human computer science students.

visit author page

Download Paper |

Abstract

Paper type: Work in Progress

Abstract: We present an open-source and highly configurable web application for posing coding exercises to students, keylogging their attempted solutions, and administering surveys and tutorials between attempts. The application is aimed at assessment and analysis of the student problem-solving process. Its multi-language (Python and JavaScript) support and open and portable design remove barriers for both experimenters and participants, potentially enabling significant expansion of and collaboration across recent educational data mining efforts. We validate the application in a small pilot study involving three students and 16 coding exercises each, and demonstrate how the collected data can be used for analysis. Although small-scale, the preliminary pilot results suggest that coding performance is highly bimodal, imperfectly aligned with student perceptions of problem difficulty, and can be predicted in advance based on early cursor movements in the beginning of an attempt. We conclude with a discussion of future work to scale up our data collection efforts towards a more comprehensive and robust analysis.

Keywords: student assessment, undergraduate first-year curriculum, computer science education, problem solving, data collection

Plourde, X. R., & Katz, G. E. (2024, June), Keylogging in a Web-Based Code Editor for Fine-Grained Analysis and Early Prediction of Student Performance Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--47709

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2024 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015