Equipping First-Year Engineering Students with Artificial Intelligence Literacy (AI-L): Implementation, Assessment, and Impact

Uri Feldman; Callie Cherry

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Engineering Libraries Division (ELD) Technical Session 4
Tagged Division: Engineering Libraries Division (ELD)
Permanent URL: https://peer.asee.org/47327

Request a correction

Paper Authors

biography

Uri Feldman Wentworth Institute of Technology orcid.org/0000-0002-2986-748X

visit author page

Uri Feldman is an Associate Professor of Biomedical Engineering in the School of Engineering at Wentworth Institute of Technology in Boston. He received a Ph.D. from the Massachusetts Institute of Technology’s Media Lab, a B.S. in Electrical Engineering from Case Western Reserve University in Cleveland, and an M.S. in Electrical Engineering from University of Illinois at Urbana Champaign. As a Postdoctoral Fellow at Harvard Medical School at Brigham and Women’s Hospital in Boston, Dr. Feldman developed informatics metrics to quantify performance of clinicians when using digital diagnostic tools. He has published in Radiology, Academic Radiology, IS&T, SPIE, and RESNA. As a Latino and native Spanish speaker, born in Peru, Dr. Feldman has created markets and commercialized innovative telemedicine products in Latin America for medical device companies, including Orex Computed Radiography, Kodak Health Group, and ICRco. Dr. Feldman also served as Chief Information Officer (CIO) of Boston Healthcare for the Homeless Program where he led the strategic planning and migration to EPIC Electronic Health Records system and novel meaningful use implementations through the Massachusetts Health Information Exchange. At Wentworth, Dr. Feldman is focused on project-based instruction, hands-on simulations, experiential learning approaches, and first year curriculum. Dr. Feldman is one of the lead instructors for Introduction to Engineering courses, with enrollments in the hundreds each fall. His research and teaching interests, in addition to first year engineering, include telemedicine, health informatics, rehabilitation engineering, and medical robotics. Dr. Feldman has collaborated with researchers and engineers from organizations including Tufts School of Veterinary Medicine, Boston Children’s Hospital, Vecnacares, and Restoreskills.

visit author page

biography

Callie Cherry Wentworth Institute of Technology

visit author page

Callie Cherry (she/her) is a Reference & Instruction Librarian, the exhibits coordinator, and the liaison to the School of Architecture & Design at Wentworth Institute of Technology. She is also the Moderator of the Art Libraries Society of North America’s Architecture & Planning Section. Her research interests include critical pedagogy; diversity, equity, and inclusion in collection development; and information literacy instruction related to artificial intelligence.

visit author page

Download Paper | Permalink

Abstract

The discourse surrounding the use of artificial intelligence (AI) in higher education has been dominated by a sense of intrigue, uncertainty, and apprehension. Higher education is faced with the challenge of addressing this emerging technology within their academic settings. In order to tackle the use of AI in the classroom head-on, a teaching team consisting of an instructor and a librarian sought to showcase how AI can be conscientiously and responsibly integrated into the existing curriculum. The teaching team designed an instruction module with two goals in mind: first, to train students on how to use a leading AI large language model generative chatbot, ChatGPT, and second, on how to analyze and interpret the synthetically generated outputs. This paper presents a preliminary analysis of the efficacy and impact of this instructional module.

The approach adopted by the teaching team is grounded in the American Association of Colleges and Universities (AAC&U) Valid Assessment of Learning in Undergraduate Education (VALUE) Rubrics for Information Literacy. In particular, students were exposed to the following three dimensions: Accessing the Needed Information, Evaluating Information and its Sources Critically, and Accessing and Using Information Ethically and Legally. In this paper, the term Artificial Intelligence Literacy (AI-L), refers to equipping students with skills necessary to evaluate AI critically and apply these new skills to their work responsibly.

Introduction to Engineering Experience is a required course offered every Fall semester to all first-year engineering students at an undergraduate engineering institution. The course is grounded on the approach of Raymond Landis, who coined the term World Class Engineering Student (WCES). The approach focuses on development of soft skills including collaboration, reflection, peer review, and time management; skills which are increasingly recognized as an important part of student development and success in engineering education, and essential in the development of WCES’s. In the current Fall 2023 semester, an AI-L module was added, with the belief that AI is an emerging, and essential component of being a WCES. The AI-L module was delivered in two sessions to 128 students. In the first session, students were asked to create their own ChatGPT account, and were instructed on how generative large language model AI systems work. In small groups and as a whole class, students discussed how to create effective prompts, and how ChatGPT responds to prompts. In the second session, students learned how ChatGPT processes information and constructs its outputs. Students also discussed the limitations of AI bots, and learned the importance of thinking critically about AI-generated content. Subsequently, the teaching team equipped students with a rubric based on the VALUES dimensions to evaluate content generated by AI tools.

In this paper, a preliminary assessment of the effectiveness of this AI-L instruction module is presented. Initial observations indicate that the module seems to have been effective in training students to recognize and critically evaluate AI-generated content. Students were surprised and intrigued by how much AI is capable of doing, and eager to learn how to be responsible users of such technologies. As a result of this instruction module, this cohort of first year engineering students seemed to possess a more comprehensive understanding of how ChatGPT can both facilitate and potentially hinder their educational and professional development. This empowered them to make informed decisions about the extent and manner in which they wish to engage with AI technology in their current studies and in the future as engineers and as lifelong learners.

Citation
Format

Feldman, U., & Cherry, C. (2024, June), Equipping First-Year Engineering Students with Artificial Intelligence Literacy (AI-L): Implementation, Assessment, and Impact Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/47327

TY - CPAPER
AB - The discourse surrounding the use of artificial intelligence (AI) in higher education has been dominated by a sense of intrigue, uncertainty, and apprehension. Higher education is faced with the challenge of addressing this emerging technology within their academic settings. In order to tackle the use of AI in the classroom head-on, a teaching team consisting of an instructor and a librarian sought to showcase how AI can be conscientiously and responsibly integrated into the existing curriculum. The teaching team designed an instruction module with two goals in mind: first, to train students on how to use a leading AI large language model generative chatbot, ChatGPT, and second, on how to analyze and interpret the synthetically generated outputs. This paper presents a preliminary analysis of the efficacy and impact of this instructional module.

The approach adopted by the teaching team is grounded in the American Association of Colleges and Universities (AAC&amp;amp;U) Valid Assessment of Learning in Undergraduate Education (VALUE) Rubrics for Information Literacy. In particular, students were exposed to the following three dimensions: Accessing the Needed Information, Evaluating Information and its Sources Critically, and Accessing and Using Information Ethically and Legally. In this paper, the term Artificial Intelligence Literacy (AI-L), refers to equipping students with skills necessary to evaluate AI critically and apply these new skills to their work responsibly.

Introduction to Engineering Experience is a required course offered every Fall semester to all first-year engineering students at an undergraduate engineering institution. The course is grounded on the approach of Raymond Landis, who coined the term World Class Engineering Student (WCES). The approach focuses on development of soft skills including collaboration, reflection, peer review, and time management; skills which are increasingly recognized as an important part of student development and success in engineering education, and essential in the development of WCES’s. In the current Fall 2023 semester, an AI-L module was added, with the belief that AI is an emerging, and essential component of being a WCES.

The AI-L module was delivered in two sessions to 128 students. In the first session, students were asked to create their own ChatGPT account, and were instructed on how generative large language model AI systems work. In small groups and as a whole class, students discussed how to create effective prompts, and how ChatGPT responds to prompts. In the second session, students learned how ChatGPT processes information and constructs its outputs. Students also discussed the limitations of AI bots, and learned the importance of thinking critically about AI-generated content. Subsequently, the teaching team equipped students with a rubric based on the VALUES dimensions to evaluate content generated by AI tools.

In this paper, a preliminary assessment of the effectiveness of this AI-L instruction module is presented. Initial observations indicate that the module seems to have been effective in training students to recognize and critically evaluate AI-generated content. Students were surprised and intrigued by how much AI is capable of doing, and eager to learn how to be responsible users of such technologies. As a result of this instruction module, this cohort of first year engineering students seemed to possess a more comprehensive understanding of how ChatGPT can both facilitate and potentially hinder their educational and professional development. This empowered them to make informed decisions about the extent and manner in which they wish to engage with AI technology in their current studies and in the future as engineers and as lifelong learners.

AU - Uri Feldman
AU - Callie Cherry
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Equipping First-Year Engineering Students with Artificial Intelligence Literacy (AI-L): Implementation, Assessment, and Impact
UR - https://peer.asee.org/47327
ER -