Asee peer logo

A Department’s Syllabi Review for LLM Considerations Prior to University-standard Guidance

Download Paper |

Conference

2024 ASEE Annual Conference & Exposition

Location

Portland, Oregon

Publication Date

June 23, 2024

Start Date

June 23, 2024

End Date

June 26, 2024

Conference Session

ML and Generative AI Tools and Policies

Tagged Division

Computers in Education Division (COED)

Page Count

10

DOI

10.18260/1-2--46436

Permanent URL

https://peer.asee.org/46436

Download Count

83

Paper Authors

biography

Lucas J. Wiese Purdue University at West Lafayette Orcid 16x16 orcid.org/0009-0008-3620-0035

visit author page

Lucas Wiese is a PhD student in Computer and Information Technology at Purdue University. He studies AI ethics education and workforce development and works in the Research on Computing in Engineering and Technology Education lab (ROCkETEd) and the Governance and Responsible AI Lab (GRAIL).

visit author page

biography

Alejandra J. Magana Purdue University at West Lafayette Orcid 16x16 orcid.org/0000-0001-6117-7502

visit author page

Alejandra J. Magana, Ph.D., is the W.C. Furnas Professor in Enterprise Excellence in the Department of Computer and Information Technology and Professor of Engineering Education at Purdue University.

visit author page

Download Paper |

Abstract

The release and widespread use of generative artificial intelligence causes concern for the future of teaching and learning. Since the release of ChatGPT, some institutions released guidance on its use in education, while other institutions waited for the technology to mature. This study is contextually situated during the Fall 2023 semester at a single university; Unique because the university had not published LLM guidance yet, but the technology had been out long enough for students to become familiar with its use. Through the conceptual lens of Teacher Noticing This study examined (a) whether faculty saw the potential use of LLMs for teaching and learning, and (b) how they responded to the rapid impact of LLMs in the classroom before university-standard guidance. Via document analysis, we found that despite LLM chatbots being widespread for roughly 9 months before the Fall semester, only a third of faculty acknowledged its use in the classroom. Faculty took three positions toward it: encouraged, discouraged, and prohibited. As found in qualitative analysis, most of the language was precautionary and discouraging. Through the lens of Teacher Noticing, we suggest that this is worrisome since faculty beliefs seemed to be mismatched with the enthusiasm and excitement of AI from students. Only a few months later, the university encouraged the use of creatively incorporating LLMs in the classroom to foster learning and increase students’ awareness of the limitations of the tools. In a technology department especially, instructors falling behind the curve of digital literacy may impact students’ satisfaction with their education. Future work should be done to understand how university guidance impacts faculty beliefs and how that translates to pedagogical techniques and learning outcomes.

Wiese, L. J., & Magana, A. J. (2024, June), A Department’s Syllabi Review for LLM Considerations Prior to University-standard Guidance Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--46436

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2024 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015