Frias, A., & Krishnakumar, S., & Pandey, A. (2025, June), FlexiGrader: an LLM-based personalized autograder to enable flexible and open-ended creative exploration in CS1  Paper presented at 2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada . 10.18260/1-2--56580

\bibitem{asee_peer_56580}
  Frias, A., \& Krishnakumar, S., \& Pandey, A. (2025, June), \emph{FlexiGrader: an LLM-based personalized autograder to enable flexible and open-ended creative exploration in CS1}
  Paper presented at 2025 ASEE Annual Conference \& Exposition , Montreal, Quebec, Canada .
  10.18260/1-2--56580

Alexis Frias, Shrivaikunth Krishnakumar, and Ayush Pandey.  "FlexiGrader: an LLM-based personalized autograder to enable flexible and open-ended creative exploration in CS1".  2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada , 2025, June.  ASEE Conferences, 2025.  https://peer.asee.org/56580 Internet.  24 Aug, 2025

\bibitem{asee_peer_56580}
  Alexis Frias, Shrivaikunth Krishnakumar, and Ayush Pandey.
  "FlexiGrader: an LLM-based personalized autograder to enable flexible and open-ended creative exploration in CS1".
  \emph{2025 ASEE Annual Conference \& Exposition , Montreal, Quebec, Canada , 2025, June}.
  ASEE Conferences, 2025.
  https://peer.asee.org/56580 Internet.  24 Aug, 2025

@INPROCEEDINGS{asee_peer_56580
author = "Alexis Frias, Shrivaikunth Krishnakumar, and Ayush Pandey"
title = "FlexiGrader: an LLM-based personalized autograder to enable flexible and open-ended creative exploration in CS1"
booktitle = "2025 ASEE Annual Conference \& Exposition "
year = "2025"
month = "June"
address = "Montreal, Quebec, Canada "
publisher = "ASEE Conferences"
note = {https://peer.asee.org/56580}
number = {10.18260/1-2--56580}
}

TY  - CPAPER
AB  - Computer Science courses often rely on programming assignments for learning assessment. Automatic grading (autograding) is a common mechanism to provide quick feedback to students and reduce teacher workload, especially in large classes. However, traditional autograders offer limited personalized feedback and often require all students to solve the same predefined problem, restricting creativity. In this paper, we address these limitations by developing an AI-based autograder that (1) can grade diverse, open-ended assignments where students work on independent, creative projects, enabling a new set of assessments in CS1 (introductory programming) courses, and (2) provides personalized feedback using large language models (LLMs). We present the design of a new assessment strategy in introductory programming courses where each student works on an open-ended problem for their summative assessment. We design generalized scaffolds (project proposal, schematic development, pseudocode, integration of files, and graphs) for these open-ended assessments so that each student completes a project of desired complexity. Existing autograders require rigid structure of inputs and outputs, and therefore, cannot grade such assessments. Our tool, FlexiGrader, integrates code execution verification and unit testing tailored to the specifications of each student individually, followed by code analysis using our fine-tuned Llama model to generate feedback and grades. FlexiGrader is capable of handling submissions from large classes and ensures flexibility in grading free-form assignments, making it easier for instructors to design and assess varied projects. The input requirements of our tool is a cover sheet that describes the individualized project, provides paths to external files, and describes the inputs needed to run the program that the student submits. Beyond this student-driven cover sheet, FlexiGrader provides options for the instructor to describe the grading rubric and choose the criteria that will be graded by the AI model. We hypothesize that live implementation of FlexiGrader in CS1 classrooms can enhance student self-efficacy and creativity in CS education by fostering independent project development. We plan to study this hypothesis in future research. Additionally, we discuss the operational costs of our autograding system, its compatibility with existing autograding frameworks, and the current limitations of our approach. By enabling more creative and personalized assignments, FlexiGrader has the potential to transform assessment practices in introductory computer science courses.
AU  - Alexis Frias
AU  - Shrivaikunth Krishnakumar
AU  - Ayush Pandey
CY  - Montreal, Quebec, Canada 
DA  - 2025/06/22
PB  - ASEE Conferences
TI  - FlexiGrader: an LLM-based personalized autograder to enable flexible and open-ended creative exploration in CS1
UR  - https://peer.asee.org/56580
DO  - 10.18260/1-2--56580
ER  -