Smith, J. M., & McGill, M., & Koressel, J., & Twarek, B. (2025, June), Can AI Develop Curriculum? Integrated Computer Science As a Test Case (Research to Practice)  Paper presented at 2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada . https://peer.asee.org/56052

\bibitem{asee_peer_56052}
  Smith, J. M., \& McGill, M., \& Koressel, J., \& Twarek, B. (2025, June), \emph{Can AI Develop Curriculum? Integrated Computer Science As a Test Case (Research to Practice)}
  Paper presented at 2025 ASEE Annual Conference \& Exposition , Montreal, Quebec, Canada .
  https://peer.asee.org/56052

Julie M. Smith, Monica McGill, Jacob Koressel, and Bryan Twarek.  "Can AI Develop Curriculum? Integrated Computer Science As a Test Case (Research to Practice)".  2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada , 2025, June.  ASEE Conferences, 2025.  https://peer.asee.org/56052 Internet.  19 Aug, 2025

\bibitem{asee_peer_56052}
  Julie M. Smith, Monica McGill, Jacob Koressel, and Bryan Twarek.
  "Can AI Develop Curriculum? Integrated Computer Science As a Test Case (Research to Practice)".
  \emph{2025 ASEE Annual Conference \& Exposition , Montreal, Quebec, Canada , 2025, June}.
  ASEE Conferences, 2025.
  https://peer.asee.org/56052 Internet.  19 Aug, 2025

@INPROCEEDINGS{asee_peer_56052
author = "Julie M. Smith, Monica McGill, Jacob Koressel, and Bryan Twarek"
title = "Can AI Develop Curriculum? Integrated Computer Science As a Test Case (Research to Practice)"
booktitle = "2025 ASEE Annual Conference \& Exposition "
year = "2025"
month = "June"
address = "Montreal, Quebec, Canada "
publisher = "ASEE Conferences"
note = {https://peer.asee.org/56052}
}

TY  - CPAPER
AB  - Introduction: Because developing integrated computer science (CS) curriculum is a resource-intensive process, there is interest in leveraging the capabilities of AI tools, including large language models (LLMs), to streamline this task. However, given the novelty of LLMs, little is known about their ability to generate appropriate curriculum content. 

Research Question: How do current LLMs perform on the task of creating appropriate learning activities for integrated computer science education? 

Methods: We tested two LLMs (Claude 3.5 Sonnet and ChatGPT 4-o) by providing them with a subset of national learning standards for both CS and language arts and asking them to generate a high-level description of learning activities that met standards for both disciplines. Four humans rated the LLM output – using an aggregate rating approach – in terms of (1) whether it met the CS learning standard, (2) whether it met the language arts learning standard, (3) whether it was equitable, and (4) its overall quality. 

Results: For Claude AI, 52% of the activities met language arts standards, 64% met CS standards, and the average quality rating was middling. For ChatGPT, 75% of the activities met language arts standards, 63% met CS standards, and the average quality rating was low. Virtually all activities from both LLMs were rated as neither actively promoting nor inhibiting equitable instruction. 

Discussion: Our results suggest that LLMs are not (yet) able to create appropriate learning activities from learning standards. The activities were generally not usable by classroom teachers without further elaboration and/or modification. There were also grammatical errors in the output, something not common with LLM-produced text. Further, standards in one or both disciplines were often not addressed, and the quality of the activities was often low. We conclude with recommendations for the use of LLMs in curriculum development in light of these findings.
AU  - Julie M. Smith
AU  - Monica McGill
AU  - Jacob Koressel
AU  - Bryan Twarek
CY  - Montreal, Quebec, Canada 
DA  - 2025/06/22
PB  - ASEE Conferences
TI  - Can AI Develop Curriculum? Integrated Computer Science As a Test Case (Research to Practice)
UR  - https://peer.asee.org/56052
ER  -