Working Towards GenAI Literacy: Assessing First-Year Engineering Students’ Attitudes towards, Trust in, and Ethical Opinions of ChatGPT

Campbell R. Bego; Tessa Withorn; Judith Danovitch; Angela Thompson; Elisabeth Thomas; Gabriel Ethan Gatsos; Alvin Tran

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Educational Research and Methods Division (ERM) Technical Session 1
Tagged Division: Educational Research and Methods Division (ERM)
Permanent URL: https://peer.asee.org/48555

Request a correction

Paper Authors

biography

Campbell R. Bego University of Louisville orcid.org/0000-0002-8125-3178

visit author page

Campbell Rightmyer Bego, PhD, PE, studies learning and retention in undergraduate engineering programs in the Department of Engineering Fundamentals at the University of Louisville's Speed School of Engineering. She obtained a BS from Columbia University in Mechanical Engineering, a PE license in Mechanical Engineering from the state of New York, and an MS and PhD in Cognitive Science from the University of Louisville. Her current interests are generative AI and information literacy for engineering, individualized, first-year persistence interventions, and the effectiveness of evidence-based practices in the engineering classroom.

visit author page

author page

Tessa Withorn University of Louisville

biography

Judith Danovitch University of Louisville

visit author page

Dr. Judith Danovitch is a Professor in the Department of Psychological and Brain Sciences at the
University of Louisville. Dr. Danovitch earned her PhD in Psychology from Yale University. She researches cognition and learning, and is particularly interested in children's and adults' beliefs about and learning from technology.

visit author page

biography

Angela Thompson P.E. University of Louisville

visit author page

Dr. Angela Thompson is an Associate Professor in the Department of Engineering Fundamentals at the University of Louisville. Dr. Thompson received her PhD in Mechanical Engineering from the University of Louisville. Her research interests are in biomechanics and engineering education, particularly related to first-year students.

visit author page

author page

Abstract

This Complete Research paper will present the findings of a survey conducted at the beginning of the Fall 2023 semester assessing first-year engineering student attitudes about ChatGPT. Engineering students need to be able to find, evaluate, integrate, and ethically use information sources to solve problems during the design process [1], skills otherwise known as information literacy [2]. While there have been many effective interventions for integrating information literacy instruction into engineering curricula [3], this framework has not yet been revised to include the use of generative artificial intelligence (GenAI). In recent years, GenAI technologies have made significant advances and are increasingly integrated into various products and services, either explicitly or behind the scenes [4]. Consequently, there is an urgent need to introduce GenAI literacy within first-year engineering programs so students can engage with these technologies intelligently, both during their academic journey and in their professional careers [5]–[8]. This inclusion is important since students may not fully understand where information presented in GenAI is sourced, the authority or accuracy of information from GenAI, nor how they can appropriately attribute information gained by GenAI. Therefore, the first-semester engineering course at a large Midwestern university was updated in 2023 to include lessons and projects around ChatGPT, a prominent and accessible large-language chatbot. Prior to any instruction, a survey was given to examine students’ experience and understanding. Questions asked about students’: • Prior experience with ChatGPT, • General attitudes toward ChatGPT, using an adapted version of the General Internet Attitudes Scale (GIAS; [9]), including subscales on affect, exhilaration, social benefit, and detriment, • Trust in correctness of ChatGPT responses to sample prompts, gauged by likelihood ratings, and • Responses to ethical questions related to ChatGPT usage in academic and professional engineering scenarios. A total of 453 students out of 487 (93% response rate) participated in the survey. The findings showed that only 31% of the first-year cohort had no prior experience with ChatGPT. Among the remaining students, 32% stated that they had moderate experience or higher, which was described as having “used it several times with a purpose in mind.” The survey also revealed a wide variety of attitudes towards ChatGPT. For instance, when asked about the ethics of “using ChatGPT for writing programming code in an engineering workplace,” the histogram of responses was completely platykurtic (flat), demonstrating extremely high variance. The paper will describe the means and variance for each scale in detail and relations between student experience with ChatGPT and their attitudes. These results establish a valuable baseline for understanding the attitudes and ethical perspectives of first-year engineering students toward AI, underscoring the necessity of incorporating GenAI literacy into engineering curricula. Keywords: first-year, generative AI, survey, ethics, information literacy

References [1] M. Fosmire, “Making Informed Decisions: The Role of Information Literacy in Ethical and Effective Engineering Design,” Theory Into Practice, vol. 56, no. 4, pp. 308–317, Oct. 2017, doi: 10.1080/00405841.2017.1350495. [2] Association of College and Research Libraries, “Framework for Information Literacy for Higher Education,” Association of College & Research Libraries (ACRL). Accessed: Oct. 30, 2023. [Online]. Available: https://www.ala.org/acrl/standards/ilframework [3] M. Phillips, A. Van Epps, N. Johnson, and D. Zwicky, “Effective Engineering Information Literacy Instruction: A Systematic Literature Review,” The Journal of Academic Librarianship, vol. 44, no. 6, pp. 705–711, Nov. 2018, doi: 10.1016/j.acalib.2018.10.006. [4] S. Lee, M. Lee, and S. Lee, “What If Artificial Intelligence Become Completely Ambient in Our Daily Lives? Exploring Future Human-AI Interaction through High Fidelity Illustrations,” International Journal of Human–Computer Interaction, vol. 39, no. 7, pp. 1371–1389, Apr. 2023, doi: 10.1080/10447318.2022.2080155. [5] I. Celik, “Exploring the Determinants of Artificial Intelligence (AI) Literacy: Digital Divide, Computational Thinking, Cognitive Absorption,” Telematics and Informatics, vol. 83, p. 102026, Sep. 2023, doi: 10.1016/j.tele.2023.102026. [6] D. T. K. Ng, J. K. L. Leung, S. K. W. Chu, and M. S. Qiao, “Conceptualizing AI literacy: An exploratory review,” Computers and Education: Artificial Intelligence, vol. 2, p. 100041, Jan. 2021, doi: 10.1016/j.caeai.2021.100041. [7] D. T. K. Ng, J. K. L. Leung, S. K. W. Chu, and M. S. Qiao, “Conceptualizing AI literacy: An exploratory review,” Computers and Education: Artificial Intelligence, vol. 2, p. 100041, Jan. 2021, doi: 10.1016/j.caeai.2021.100041. [8] Y. Yi, “Establishing the concept of AI literacy,” Jahr – European Journal of Bioethics, vol. 12, no. 2, Art. no. 2, 2021. [9] M. Joyce and J. Kirakowski, “Measuring Attitudes Towards the Internet: The General Internet Attitude Scale,” International Journal of Human–Computer Interaction, vol. 31, no. 8, pp. 506–517, Aug. 2015, doi: 10.1080/10447318.2015.1064657.

Citation
Format

Bego, C. R., & Withorn, T., & Danovitch, J., & Thompson, A., & Thomas, E., & Gatsos, G. E., & Tran, A. (2024, June), Working Towards GenAI Literacy: Assessing First-Year Engineering Students’ Attitudes towards, Trust in, and Ethical Opinions of ChatGPT Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/48555

TY - CPAPER
AB - This Complete Research paper will present the findings of a survey conducted at the beginning of the Fall 2023 semester assessing first-year engineering student attitudes about ChatGPT.
Engineering students need to be able to find, evaluate, integrate, and ethically use information sources to solve problems during the design process [1], skills otherwise known as information literacy [2]. While there have been many effective interventions for integrating information literacy instruction into engineering curricula [3], this framework has not yet been revised to include the use of generative artificial intelligence (GenAI). In recent years, GenAI technologies have made significant advances and are increasingly integrated into various products and services, either explicitly or behind the scenes [4]. Consequently, there is an urgent need to introduce GenAI literacy within first-year engineering programs so students can engage with these technologies intelligently, both during their academic journey and in their professional careers [5]–[8]. This inclusion is important since students may not fully understand where information presented in GenAI is sourced, the authority or accuracy of information from GenAI, nor how they can appropriately attribute information gained by GenAI.
Therefore, the first-semester engineering course at a large Midwestern university was updated in 2023 to include lessons and projects around ChatGPT, a prominent and accessible large-language chatbot. Prior to any instruction, a survey was given to examine students’ experience and understanding. Questions asked about students’:
• Prior experience with ChatGPT,
• General attitudes toward ChatGPT, using an adapted version of the General Internet Attitudes Scale (GIAS; [9]), including subscales on affect, exhilaration, social benefit, and detriment,
• Trust in correctness of ChatGPT responses to sample prompts, gauged by likelihood ratings, and
• Responses to ethical questions related to ChatGPT usage in academic and professional engineering scenarios.
A total of 453 students out of 487 (93% response rate) participated in the survey. The findings showed that only 31% of the first-year cohort had no prior experience with ChatGPT. Among the remaining students, 32% stated that they had moderate experience or higher, which was described as having “used it several times with a purpose in mind.” The survey also revealed a wide variety of attitudes towards ChatGPT. For instance, when asked about the ethics of “using ChatGPT for writing programming code in an engineering workplace,” the histogram of responses was completely platykurtic (flat), demonstrating extremely high variance. The paper will describe the means and variance for each scale in detail and relations between student experience with ChatGPT and their attitudes.
These results establish a valuable baseline for understanding the attitudes and ethical perspectives of first-year engineering students toward AI, underscoring the necessity of incorporating GenAI literacy into engineering curricula.
Keywords: first-year, generative AI, survey, ethics, information literacy

References
[1] M. Fosmire, “Making Informed Decisions: The Role of Information Literacy in Ethical and Effective Engineering Design,” Theory Into Practice, vol. 56, no. 4, pp. 308–317, Oct. 2017, doi: 10.1080/00405841.2017.1350495.
[2] Association of College and Research Libraries, “Framework for Information Literacy for Higher Education,” Association of College &amp;amp; Research Libraries (ACRL). Accessed: Oct. 30, 2023. [Online]. Available: https://www.ala.org/acrl/standards/ilframework
[3] M. Phillips, A. Van Epps, N. Johnson, and D. Zwicky, “Effective Engineering Information Literacy Instruction: A Systematic Literature Review,” The Journal of Academic Librarianship, vol. 44, no. 6, pp. 705–711, Nov. 2018, doi: 10.1016/j.acalib.2018.10.006.
[4] S. Lee, M. Lee, and S. Lee, “What If Artificial Intelligence Become Completely Ambient in Our Daily Lives? Exploring Future Human-AI Interaction through High Fidelity Illustrations,” International Journal of Human–Computer Interaction, vol. 39, no. 7, pp. 1371–1389, Apr. 2023, doi: 10.1080/10447318.2022.2080155.
[5] I. Celik, “Exploring the Determinants of Artificial Intelligence (AI) Literacy: Digital Divide, Computational Thinking, Cognitive Absorption,” Telematics and Informatics, vol. 83, p. 102026, Sep. 2023, doi: 10.1016/j.tele.2023.102026.
[6] D. T. K. Ng, J. K. L. Leung, S. K. W. Chu, and M. S. Qiao, “Conceptualizing AI literacy: An exploratory review,” Computers and Education: Artificial Intelligence, vol. 2, p. 100041, Jan. 2021, doi: 10.1016/j.caeai.2021.100041.
[7] D. T. K. Ng, J. K. L. Leung, S. K. W. Chu, and M. S. Qiao, “Conceptualizing AI literacy: An exploratory review,” Computers and Education: Artificial Intelligence, vol. 2, p. 100041, Jan. 2021, doi: 10.1016/j.caeai.2021.100041.
[8] Y. Yi, “Establishing the concept of AI literacy,” Jahr – European Journal of Bioethics, vol. 12, no. 2, Art. no. 2, 2021.
[9] M. Joyce and J. Kirakowski, “Measuring Attitudes Towards the Internet: The General Internet Attitude Scale,” International Journal of Human–Computer Interaction, vol. 31, no. 8, pp. 506–517, Aug. 2015, doi: 10.1080/10447318.2015.1064657.
AU - Campbell R. Bego
AU - Tessa Withorn
AU - Judith Danovitch
AU - Angela Thompson P.E.
AU - Elisabeth Thomas
AU - Gabriel Ethan Gatsos
AU - Alvin Tran
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Working Towards GenAI Literacy: Assessing First-Year Engineering Students’ Attitudes towards, Trust in, and Ethical Opinions of ChatGPT
UR - https://peer.asee.org/48555
ER -