Portland, Oregon
June 23, 2024
June 23, 2024
June 26, 2024
Educational Research and Methods Division (ERM) Technical Session 17
Educational Research and Methods Division (ERM)
8
10.18260/1-2--47709
https://peer.asee.org/47709
96
Garrett Katz is an assistant professor at Syracuse University. He teaches a broad range of computer science courses, covering introductory programming, discrete math, introductory artificial intelligence, and graduate seminars. His research covers various topics in artificial intelligence and human-machine interaction, including in educational contexts. In particular, his recent work investigates reasoning and learning processes underlying program synthesis, both for automated program synthesis by machines as well as manual program synthesis by human computer science students.
Paper type: Work in Progress
Abstract: We present an open-source and highly configurable web application for posing coding exercises to students, keylogging their attempted solutions, and administering surveys and tutorials between attempts. The application is aimed at assessment and analysis of the student problem-solving process. Its multi-language (Python and JavaScript) support and open and portable design remove barriers for both experimenters and participants, potentially enabling significant expansion of and collaboration across recent educational data mining efforts. We validate the application in a small pilot study involving three students and 16 coding exercises each, and demonstrate how the collected data can be used for analysis. Although small-scale, the preliminary pilot results suggest that coding performance is highly bimodal, imperfectly aligned with student perceptions of problem difficulty, and can be predicted in advance based on early cursor movements in the beginning of an attempt. We conclude with a discussion of future work to scale up our data collection efforts towards a more comprehensive and robust analysis.
Keywords: student assessment, undergraduate first-year curriculum, computer science education, problem solving, data collection
Plourde, X. R., & Katz, G. E. (2024, June), Keylogging in a Web-Based Code Editor for Fine-Grained Analysis and Early Prediction of Student Performance Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--47709
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2024 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015