-dictive power on performance outcomes. Finally, we call for continued empirical research on theefficacy of LLM-based technologies in STEM education and propose future research directions inexploring their impact on teaching and learning.1 IntroductionThe introduction of OpenAI’s ChatGPT in November 2022 [1] triggered an unprecedented surgeof interest in applications of artificial intelligence (AI) based on Large Language Models (LLMs)and their underlying transformer architecture.In particular, LLMs appear to be exceptional in applications that involve human interaction, infor-mation retrieval, and summation, making them an attractive prospect for improving the effective-ness and accessibility of education in the digital age [2, 3, 4]. However
standards management [2].The advent of advanced machine learning mechanisms—evolving from early neural networks tomodern transformer architectures—has ushered in a new renaissance in artificial intelligence andits practical applications. The rapid development of large language models (LLMs), capable ofprocessing substantial volumes of unstructured text and generating structured outputs, nowempowers framework mapping projects at a quality level that was inconceivable less than adecade ago. SMEs now have access to AI tools that facilitate comprehensive reviews of localguidance documents and alignment exercises with strategic frameworks. In practice, instructionaldesign teams have used tools like ChatGPT and Copilot to accelerate the development of
a growth in academic integrityfilings since the advent of ChatGPT. In fact, [2] points to a Stanford University survey where1/6th of students said they had used ChatGPT on assignments or exams. This article [2] alsopoints towards the issues of hallucinations, where AI focuses on generating text that sounds goodbut may not be scientifically accurate. However, [1] also points to potential efficiencies andutility of AI in higher education, such as teaching ethical use of AI, growth of tutoring/teachingassistants and for operational efficiencies. Auon [3] discussed the impact of AI on the humanexperience in physical (personalized medicine/drug delivery and disease identification),cognitive (increased workplace productivity, focused effort on
expert for students in theirlearning process.This research aims to redefine the creation of engineering problems by utilizing generative AI,particularly ChatGPT. The study assesses student performance by conducting a mixed-methodapproach that combines both quantitative and qualitative analyses. By adopting a novel approachfor creating engineering problems beyond traditional textbook problems, we explore a way toimprove student learning outcomes and enhance the essence of engineering education.This research specifically addresses a major research question illustrated in Figure 1: Figure 1: Assessment of Research Question 2. Background 2.1. Pedagogy of Engineering ProblemSeveral recent studies focused on reshaping
was used only in a few instances, but given the focus of the course, in mostcases, it did not meet the expectations of the professor.Background and Related LiteratureThe impact of ChatGPT has led to a significant increase in awareness and experimentation withgenerative AI tools among educators since its release in November 2022 [1]. As generativeartificial intelligence technologies have emerged onto the landscape of higher education, therehas been a healthy research interest in how students are using AI to promote their success inclasses, how faculty might integrate AI into their teaching, and how staff employees, in general,might use AI to work more efficiently [2]. The use of generative AI in all these areas isconsiderably nascent and needs
, interrogative, imperative, exclamative, andinvalid.Figure 1 provides the illustrated design of the model. Fig. 1. Sentence type detection model.DatasetThe dataset used to test this model contains 500 sentences, all generated by ChatGPT, an NLP-powered chatbot [21]. Every sentence within the dataset has been verified to be correctlyclassified as its respective type of sentence. All sentences were designed to be simple, with nocompound or complex sentences. Out of the 500 sentences, 100 sentences were simpledeclarative sentences, 100 sentences were interrogative, 100 were imperative, and 100 wereexclamative. The last 100 sentences were invalid, incomplete sentences that were none of thefour types of sentences. The created
papers may be subject tohigher standards of review and scrutiny, however, due to the propensity for false or misleadinginformation to appear in LLMs. Given that higher bar, some may be tempted to not provideattribution to AI-assisted technical writing. LLM watermarking, a process whereby resultingsyntactic patterns in AI-generated text mathematically ‘signal’ an AI source (as opposed to ahuman source) have been embedded in GPT-4 and other LLMs. These so-called watermarksallow for ‘detectors’ to provide the statistical likelihood of AI use. Some examples sourced fromindustry, academia, and students follow: 1) GPT-2 Output Detector [23]: (From Open AI, the makers of ChatGPT) Claims a detection rate of 95% for machine-generated text using
be better. Everyone is used to that now and this ChatBox technology seems a bit outdated (e.g. people are using Google and this is Yahoo). Incorporating AI into it might make it a lot better.” “Ask follow-up questions after asking ChatGPT, and somewhat integrate the chatbot answers with the response acquired from ChatGPT (record it in database or something so that when the same or a similar question is asked again the bot is able to provide answers relevant to the course)” 9Overall, the preliminary finding based on the initial prototype of the chatbot was promising. Thechatbot made accessing information much easier for students and other users
, images, and music [7],using deep learning models like GANs and transformers to generate original data by learningpatterns from training datasets. Unlike traditional machine learning, which primarily analyzes orclassifies existing data [8], GenAI models, such as OpenAI’s GPT, leverage large languagemodels (LLMs) trained on vast text datasets [9, 10]. Beyond LLMs, models like GANs, VAEs,and diffusion models further expand GenAI’s capabilities, with applications spanning NLP [11],art creation [12], and game design [13]. The release of ChatGPT quickly raised concerns aboutacademic integrity and student overreliance, potentially hindering learning due to its accessibility[14, 15]. However, these concerns have also driven interest in leveraging GenAI
deconstructed the question using the Population, Concept, and Context (PCC)framework, a widely used approach in systematic literature reviews [7]. The PCC guidelines for this review arePopulation – engineering educators and students; Concept – utilization of generative models (e.g., GenerativeAI, ChatGPT, GPT); Context – formal and informal engineering education settings.3.2. Identifying Relevant StudiesThe search strategy is structured into concept lines, following the approach outlined in [8] for scopingreviews, which is designed to identify and include articles, conference papers, and gray literature relevant tothe research question. For the scope of our project, we define this as an "Aspect": An aspect is an element ordimension of the research
limitations. First, AI tools, such as ChatGPT andCopilot are relatively new, and constantly evolving in their abilities. Secondly, the sampledstudent work represented a small portion of available data and thus is not representative of thewhole set. Finally, while every effort was made to ensure that the evaluators were consistent withthe application of the rubric, it is simply not realistic to expect people with varying expertise tobe completely consistent. While these limitations were important to acknowledge, they do notlimit the importance of this work. We see potential not only for optimizing writing support butalso for fostering student-involved negotiations on how AI can aid the writing process.MethodsContextThis study occurred at a small
adding section labels that indicate to the graderswhich prompt the report addresses in each section of text. It is worth noting that this approachmay be overly complicated with the recent deployment of ChatGPT 4.0, where PDFs can beuploaded and modified directly by the LLM. Nevertheless, the course under investigation hasupward of 40 submissions which could quickly reach any ChatGPT data limits. Additionally, thisapproach is mostly automated so dozens to even hundreds of group reports could be analyzed andhighlighted with minimal user interaction.Results - Sentiment AnalysisWe present findings from the application of our sentiment analysis technique on two lab activitieseach lasting about seven weeks. Despite the limited scope, the participation
, personalized online learning experiences. We evaluate the effectiveness of this methodthrough a series of case studies and provide guidelines for instructors to leverage these technologiesin their courses.1 IntroductionLarge Language Models (LLMs) and their emerging skills provide educators with new capabilitiesto improve our teaching and save time. LLMs like ChatGPT have emerged as powerful tools thatcan assist in creating educational content and interactive learning experiences [1].For digital system design and computer architecture, traditional education often relies on expen-sive hardware, specialized software, and physical laboratory spaces. These requirements can limitaccess to hands-on learning experiences, particularly for students in
ChatGPT [1], Google’s Gemini [2],and Microsoft’s Copilot [3] have gained widespread student adoption due to their free access andease of use. This expansion has occurred amid varying acceptance [4–6] and trust [7] in digitallearning technologies across student populations through the COVID-19 pandemic and into thepresent day. Approximately one-third (35.4%) of students reported regular usage of ChatGPT,while 47% expressed concern about AI’s impact in education [8]. Additionally, 60% reported thattheir instructors or schools had not yet provided guidelines for ethical or responsible AI tool use [8].As students increasingly use available online AI assistants, researchers have concurrently devel-oped specialized educational AI tools designed
publications have been recognized by leading engineering education research journalsat both national and international levels. Dr. McCall has led several workshops promoting the inclusionof people with disabilities and other minoritized groups in STEM. She holds B.S. and M.S. degrees incivil engineering with a structural engineering emphasis. ©American Society for Engineering Education, 2025 WIP: Understanding Patterns of Generative AI Use: A Study of Student Learning Across University CollegesIntroductionDue to the relatively recent introduction of AI to academia, facilitated by the development andrelease of popular generative AI systems such as ChatGPT, few studies have examined theeffects of AI use on
Perceptions: The Impact of AI Tools on Engineering Education Sofia M. Vidalis, Associate Professor at Pennsylvania State University - Harrisburg, Rajarajan Subramanian, Associate Teaching Professor at Pennsylvania State University – Harrisburg, and Fazil T. Najafi, Professor at University of Florida Abstract The rapid advancement of artificial intelligence (AI) has led to the integration of chatbots like ChatGPT or Chat AI into various sectors, including education. This study investigates the impact of many AI tools in engineering education, focusing on their potential to enhance learning
, Use of AI tools and Peer Collaboration on AI Assisted Learning: Perceptions of the University students.,” Digit. Educ. Rev., no. 45, pp. 43–49, Jun. 2024, doi: 10.1344/der.2024.45.43-49.[5] M. Edali, A. Milad, H. Saad, Z. Sahem, T. Alajaili, and A. Elkamel, “ChatGPT and Artificial Intelligence (AI) Massive Transformation of Trainers’ Education Sector Revolutionizing How Students Learn,” in Proceedings of the International Conference on Industrial Engineering and Operations Management, Dubai, UAE: IEOM Society International, Feb. 2024. doi: 10.46254/AN14.20240340.[6] C. Spreitzer, O. Straser, S. Zehetmeier, and K. Maaß, Mathematical Modelling Abilities of Artificial Intelligence Tools: The Case of ChatGPT., vol. 14
personalizationdevelop custom graphical user interfaces (GUI)—such as that developed in the follow-up studyby Vaccaro et al. [22]—rather than rely on public-facing interfaces like ChatGPT as it minimizesthe potential for user error. Such a controlled GUI is also beneficial from an experimental contextwhere consistency in implementation is of critical importance. Finally, it should be noted thatsuch an environment allows for strict control over the types of information students can sharewith an LLM, thus maintaining student privacy.Integration of Personalized Learning in Engineering Education through LLMsThe integration of PL into engineering education through advanced AI and LLMs represents atransformative yet nascent field. The use of cutting-edge LLMs, such
from AI – and discovered a bimodal distribution. Thus, weshow that the student body at Mines is polarized with respect to future impacts of GenAI on theengineering workforce and society, despite being increasingly willing to explore GenAI overtime. We discuss implications of these findings for future research and for integrating GenAI inengineering education.IntroductionRecent advancements in Generative Artificial Intelligence (GenAI), esp. large language models(LLMs) like ChatGPT, have significantly impacted both industry and educational sectors [1, 2].These models, equipped with sophisticated algorithms and trained on vast datasets, canunderstand and generate human-like text [3], expanding their use from simple text prediction tocomposing
through the platform of Google Earth.Throughout the activity, they were actively encouraged to leverage a wide array of online tools,encompassing resources such as usage of large language models such as ChatGPT and variousothers, to collaboratively solve the questions. During the exercise, students encountered encryptedmessages at various stages and to progress in the activity had to apply cryptographic principles todecipher these messages. The proposed practical application of cryptography involved tasks likedecrypting codes, solving puzzles, or using ciphers to reveal clues led them closer to the final chal-lenge. By introducing scavenger hunt at the intersection of computer system security education,we open a gateway to experiential learning
faculty, who are oftenconcurrently engaged in research, service duties, and mentoring activities [2], [3].To support instructional designers and faculty in this endeavor, we have leveraged the APIs ofOpenAI tools to create Transcriptto, a Python program that contains clever algorithms that aid inthe crucial steps in lecture preparation, allowing instructional designers and faculty to have abetter starting point when starting the development of an online course. Transcriptto utilizes astraightforward yet robust workflow, incorporating openly available technologies such asPymovie, FFmpeg, OpenAI’s Whisper, and ChatGPT. It transforms video lectures into polishedtext, supporting various input types, including audio files, and pre-existing scripts
most people in the first world to access (at reasonable cost) fabrication technology and computation at the scale of an individual — this trend, pushed by artists [5] and engineers, makes the public and our students not only aware but experienced in building things using these tools. However, the assumption that a student, because they lived before or during a technology emergence, is strongly skilled with that technology is false [6].The biggest of these trends that we address in this work is trend 4, commonly referred to in thegeneral population as chatbots such as ChatGPT — the continued emergence of AI capabilities —specifically, the emergence of Large Language Models (LLMs) [7] means our curriculums need tobe
of the dual-submission strategies, there is more variety to what is submitted for thesecond deadline. Sometimes students are asked to self-grade their homework [2–4]; usually theyare asked to make corrections, and some strategies ask them to undertake other activities, such asa quiz [5], group discussion [6–7], filling in missing steps in a derivation [8], or filling out a“homework wrapper” [9–10] that asks about the strategies that students used in doing thehomework and how successful they proved to be.However, the rise of Large Language Models (LLMs) like ChatGPT presents a challenge. Thesemodels can solve simple homework problems, but can they also produce credible reflections? IfLLMs can generate authentic-looking reflections, the dual
language understanding, code summarization, andnatural language-based programming, where students learn to express programming concepts innatural language 48,49 . 29 outlined the use of dialogue-centric methods in AI-enhanced tutoringsystems, which aid students in formulating pseudocode answers in a natural language formattailored to particular challenges. Furthermore, NLP offers an additional advantage by enablingconversational student support, leveraging knowledge representation to depict a cohort of studentsand their communicative dynamics during collaborative learning in CS. More recently, largelanguage models like ChatGPT are used to assists users by clarifying intricate ideas andtechnologies, offering examples, and directing them to
were recorded and uploaded on CLAS, they couldsee the difference between their original and improved lessons. It was an empowering learningexperience that gave the preservice teachers the much-needed confidence that they can figurethings out and if a lesson doesn’t go as well as they wanted the first time around, they alwayshave a second chance.Exploration of Novel Pedagogical ApproachesLearning to remove yourself from your own lessons and to reflect on them in order to teachbetter in the future is a core quality of a STEM educator in the 21st century. To be successful inthe era of fast-changing student population, rapidly evolving technologies, that haveunprecedented pedagogical potential, such as ChatGPT [42, 43], continuously
Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al. Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLoS digital health, 2(2):e0000198, 2023.[13] Enkelejda Kasneci, Kathrin Seßler, Stefan K¨uchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G¨unnemann, Eyke H¨ullermeier, et al. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences, 103:102274, 2023.[14] Ramteja Sajja, Yusuf Sermet, Muhammed Cikmaz, David Cwiertny, and Ibrahim Demir. Artificial intelligence- enabled intelligent assistant for personalized and adaptive learning in higher education
eNotebook to include a tutoring AI feature that students could talk to along with their favoritestudy methods. eNotebook provides a general platform for nearly all of today’s study methods andmaterials students use to create and customize for efficient access and assessment. For example,we have implemented a two-way talking conversation feature called Jarvis, which is an audio-to-text / text-to-audio feature with a ChatGPT engine with AI-specific aids to improve the quality ofAI responses. We have embedded weblinks to over 50 of the most popular study apps easilyaccessible through a pull-down menu, where favorites appear at the top of the list. We haveimplemented a feature that converts handwritten notes into typed text. Images, audio, videos
context of SQL query feedback?Our research questions aim to evaluate the feasibility and effectiveness of utilizing a GenerativeAI model to provide semantic error feedback without revealing the correct solution. We targetprecise error detection and insightful feedback to enhance the educational experience by makingit more tailored to each student. The effectiveness of our fine-tuned model was assessed throughcomparative analyses with the outputs from the standard ChatGPT model. This validation processwas crucial in establishing the refined model’s advancement and distinction in providing preciseand contextually appropriate feedback for SQL queries.2 Related WorkResearch on SQL learning has explored various types of errors and student challenges
, challenges in assessment persist, including the ethical considerations of dataprivacy and the potential biases in interpreting user feedback. Addressing these issues requirestransparent methodologies and a commitment to refining the design of AI-driven educationaltools based on evidence-based practices [14]. Through rigorous assessment, AI chatbots can beoptimized as transformative tools in engineering education.AI Chatbot As mentioned, a chatbot is a chat-based algorithm that uses natural language processing(NLP) algorithms to converse with the user. OpenAI’s ChatGPT is an example of a chatbotbecause it uses both natural language processing and proprietary algorithms to communicate withusers in a conversation-like manner. The algorithm
of GenAI but the most sorted types based on the input and outputformats are eleven types as shown in Table[3], which are Text-to-Text, Text-to-Image, Text-to-3D, Text-to-Audio, Text-to-Video, Text-to-Code, Text-to-Scientific text, Text-to-Chemical Formula, Text-to-Synthetic data, Text-to-Algorithm and Image-to-Text [38]. Thereare also some subtypes such as Image-to-3D, Image or Video-to-3D, Text-to-Video, Image-to-Science, Text-to-Speech, Speech-to-Text, and Speech-to-Speech. We will talk about eachtype correspondingly [39]. Text-to-Text is the most well-known type of all that generates texts based on textinputs. An example of this type is ChatGPT. To generate a text response, we need to usemachine learning, and existing data in