Chandarana, M., & Ramachandra, S., & Ball, J., & Lyons, M., & Snalune, P. (2025, June), Investigating the Impact of Codio Coach: A Specialized AI Learning Assistant on Computing Student Engagement and Performance  Paper presented at 2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada . 10.18260/1-2--56896

\bibitem{asee_peer_56896}
  Chandarana, M., \& Ramachandra, S., \& Ball, J., \& Lyons, M., \& Snalune, P. (2025, June), \emph{Investigating the Impact of Codio Coach: A Specialized AI Learning Assistant on Computing Student Engagement and Performance}
  Paper presented at 2025 ASEE Annual Conference \& Exposition , Montreal, Quebec, Canada .
  10.18260/1-2--56896

Mohit Chandarana, Sindhu Ramachandra, Joshua Ball, Maura Lyons, and Phillip Snalune.  "Investigating the Impact of Codio Coach: A Specialized AI Learning Assistant on Computing Student Engagement and Performance".  2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada , 2025, June.  ASEE Conferences, 2025.  https://peer.asee.org/56896 Internet.  01 Sep, 2025

\bibitem{asee_peer_56896}
  Mohit Chandarana, Sindhu Ramachandra, Joshua Ball, Maura Lyons, and Phillip Snalune.
  "Investigating the Impact of Codio Coach: A Specialized AI Learning Assistant on Computing Student Engagement and Performance".
  \emph{2025 ASEE Annual Conference \& Exposition , Montreal, Quebec, Canada , 2025, June}.
  ASEE Conferences, 2025.
  https://peer.asee.org/56896 Internet.  01 Sep, 2025

@INPROCEEDINGS{asee_peer_56896
author = "Mohit Chandarana, Sindhu Ramachandra, Joshua Ball, Maura Lyons, and Phillip Snalune"
title = "Investigating the Impact of Codio Coach: A Specialized AI Learning Assistant on Computing Student Engagement and Performance"
booktitle = "2025 ASEE Annual Conference \& Exposition "
year = "2025"
month = "June"
address = "Montreal, Quebec, Canada "
publisher = "ASEE Conferences"
note = {https://peer.asee.org/56896}
number = {10.18260/1-2--56896}
}

TY  - CPAPER
AB  - Recent research has demonstrated significant advancements in the applications of Large Language Models (LLMs) in educational environments, particularly in delivering immediate, personalized student feedback. This study examines the impact of Codio Coach, a specialized AI learning assistant integrated into the Codio platform, on student engagement and performance in asynchronous MOOC-style computer science courses.. It utilizes Large Language Models (LLMs) to provide support without supplying direct answers. It consists of three modules: Summarizer, which simplifies assignment instructions; Error Explanation, which clarifies programming error messages; and Hints, which provides Socratic-style hints by posing questions or suggestions to guide students toward solutions.

Analysis revealed an immediate and sustained uptake in assistant usage, with "Explain this error" being the most frequent interaction (56.3%), confirming engagement and highlighting student need for error comprehension support. Assignments where Coach was enabled showed improved student performance, with a 12% increase in Mean Grade and a 15% increase in Median Grade. Furthermore, an impressively low error event rate (0.12%) observed in these AI-assisted courses suggests early signs that such tools may contribute to more effective programming environments. 

These findings provide valuable evidence for the efficacy of tailored AI learning assistants in enhancing student engagement and performance in CS education. We recommend educators guide students in leveraging custom, context-specific assistants to improve learning and develop critical AI application skills.
AU  - Mohit Chandarana
AU  - Sindhu Ramachandra
AU  - Joshua Ball
AU  - Maura Lyons
AU  - Phillip Snalune
CY  - Montreal, Quebec, Canada 
DA  - 2025/06/22
PB  - ASEE Conferences
TI  - Investigating the Impact of Codio Coach: A Specialized AI Learning Assistant on Computing Student Engagement and Performance
UR  - https://peer.asee.org/56896
DO  - 10.18260/1-2--56896
ER  -