BOARD # 322: An NSF FMRG Supported Exploratory Study of Prompting Large Language Models for a Conversational Manufacturing Education Platform

Fatemeh Karimi Kenari; yasaswi bhumireddy; Xiaoliang Yan; Mahmoud Dinar; Shreyes N Melkote

Download Paper | Permalink

Conference: 2025 ASEE Annual Conference & Exposition
Location: Montreal, Quebec, Canada
Publication Date: June 22, 2025
Start Date: June 22, 2025
End Date: August 15, 2025
Conference Session: NSF Grantees Poster Session II
Tagged Topic: NSF Grantees Poster Session
Page Count: 6
Permanent URL: https://peer.asee.org/55689

Mahmoud Dinar University of North Carolina at Charlotte orcid.org/0000-0002-0670-3861

visit author page

Mahmoud Dinar is an assistant professor of mechanical engineering. His main research interest involves integrating AI in multimodal computational frameworks to understand and aid design for manufacturing. His ongoing projects are creating a conversational platform to evaluate and teach manufacturing skills to the future workforce, and solving geometric puzzles to design sustainable and resilient products as a 3D composition of pieces.

visit author page

author page

Shreyes N Melkote Georgia Institute of Technology

Download Paper | Permalink

Abstract

With the emergence of Artificial Intelligence (AI) and Large Language Models (LLMs), new approaches to conversation-based teaching have risen. Manufacturing education can benefit from AI in democratizing access to know-how rapidly, upskilling the future workforce with a high economic impact due to the multiplier effect of manufacturing. However, platforms like ChatGPT should be assessed for their capability to provide accurate and relevant manufacturing knowledge. This NSF FMRG supported paper aims to test modern AI tools by prompting three types of questions selected from manufacturing textbooks categorized as General Process, Sub-process, and Process Parameters. The prompts explicitly request answers for four different user levels: Children, Teenagers, Undergraduates, and experts. The responses generated by ChatGPT are evaluated qualitatively for correctness, relevance, and suitability for the user’s level of understanding of manufacturing, and quantitively using semantic similarity between keywords of responses and questions. Results show lower similarity for children responses indicating simpler and more abstract terms used which are more suitable for children. Yet, some responses have complex details that are not likely understandable for children. Additionally, some responses for higher levels are inconsistent and incomprehensive. Several future steps can mitigate the limitations and improve reliability of adopting current AI and LLM tools. Retrieval Augmented Generation, i.e., creating specialized Generative Pre-trained Transformer models trained on acquired manufacturing corpora results in more accurate responses with less computational cost. Integrating Visual Language Models to answer more complex queries that involve CAD models and images facilitates introducing manufacturability of a design as a fourth category of questions. Explicitly searching for analogies can lead to more effective explanation of complex responses suitable for not only children but advanced learners. User studies are needed to finetune and validate an optimized personalized conversational educational platform to train and encourage a broader population to adopt manufacturing skills.

Citation
Format

Karimi Kenari, F., & bhumireddy, Y., & Yan, X., & Dinar, M., & Melkote, S. N. (2025, June), BOARD # 322: An NSF FMRG Supported Exploratory Study of Prompting Large Language Models for a Conversational Manufacturing Education Platform Paper presented at 2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada . https://peer.asee.org/55689

TY - CPAPER
AB - With the emergence of Artificial Intelligence (AI) and Large Language Models (LLMs), new approaches to conversation-based teaching have risen. Manufacturing education can benefit from AI in democratizing access to know-how rapidly, upskilling the future workforce with a high economic impact due to the multiplier effect of manufacturing. However, platforms like ChatGPT should be assessed for their capability to provide accurate and relevant manufacturing knowledge. This NSF FMRG supported paper aims to test modern AI tools by prompting three types of questions selected from manufacturing textbooks categorized as General Process, Sub-process, and Process Parameters. The prompts explicitly request answers for four different user levels: Children, Teenagers, Undergraduates, and experts. The responses generated by ChatGPT are evaluated qualitatively for correctness, relevance, and suitability for the user’s level of understanding of manufacturing, and quantitively using semantic similarity between keywords of responses and questions. Results show lower similarity for children responses indicating simpler and more abstract terms used which are more suitable for children. Yet, some responses have complex details that are not likely understandable for children. Additionally, some responses for higher levels are inconsistent and incomprehensive. Several future steps can mitigate the limitations and improve reliability of adopting current AI and LLM tools. Retrieval Augmented Generation, i.e., creating specialized Generative Pre-trained Transformer models trained on acquired manufacturing corpora results in more accurate responses with less computational cost. Integrating Visual Language Models to answer more complex queries that involve CAD models and images facilitates introducing manufacturability of a design as a fourth category of questions. Explicitly searching for analogies can lead to more effective explanation of complex responses suitable for not only children but advanced learners. User studies are needed to finetune and validate an optimized personalized conversational educational platform to train and encourage a broader population to adopt manufacturing skills.
AU - Fatemeh Karimi Kenari
AU - yasaswi bhumireddy
AU - Xiaoliang Yan
AU - Mahmoud Dinar
AU - Shreyes N Melkote
CY - Montreal, Quebec, Canada
DA - 2025/06/22
PB - ASEE Conferences
TI - BOARD # 322: An NSF FMRG Supported Exploratory Study of Prompting Large Language Models for a Conversational Manufacturing Education Platform
UR - https://peer.asee.org/55689
ER -