Board 20: Work in Progress: Understanding Student Perceptions and Use of Generative Artificial Intelligence for Technical Writing

Amy N Adkins; Naji S Husseini; Lianne Cartee

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Biomedical Engineering Division (BED) Poster Session
Tagged Division: Biomedical Engineering Division (BED)
Tagged Topic: Diversity
Permanent URL: https://peer.asee.org/46766

Request a correction

Paper Authors

biography

Amy N Adkins North Carolina State University orcid.org/0000-0001-5525-740X

visit author page

Amy N. Adkins is a n Assistant Teaching Professor in the Joint Department of Biomedical Engineering at University of North Carolina (UNC) and North Carolina State University (NCSU). She received her Ph.D. and M.S. in Biomedical Engineering from Northwestern University and her B.S. in Engineering Science from St. Mary's University in San Antonio. Her technical research which relates to her PhD Dissertation is focused on utilizing novel imaging techniques to quantifying adaptation of muscle architecture in humans. She also desires to implement innovative teaching, mentoring, and hands-on problem solving to develop students’ deep understanding of engineering principles and to inspire them to tackle real-world problems which can aid human health.

visit author page

biography

Naji S Husseini North Carolina State University

visit author page

Naji Husseini is an associate teaching professor and associate director of undergraduate studies in the Joint Department of Biomedical Engineering at the University of North Carolina at Chapel Hill and North Carolina State University.

visit author page

biography

Lianne Cartee North Carolina State University

visit author page

Lianne Cartee is Director of Undergraduate Studies in the Joint Department of Biomedical Engineering at the Univ. of North Carolina at Chapel Hill and North Carolina State University.

visit author page

Download Paper | Permalink

Abstract

With the improvement in open generative artificial intelligence’s (AI’s) ability to craft human-like text, there is a valid concern by educators that this technology will be used by students to complete assignments without learning the subject matter. Also problematic is the limited ability to verify or prove a student’s potentially unauthorized or unethical use of AI. While these fears are valid, we believe the best way forward is to focus on educating students on how to use this powerful technology ethically and effectively. Though there is some work establishing best practices for using AI in writing scientific manuscripts [1], how best to utilize AI as an instructional aid for teaching scientific writing is less understood. For biomedical engineers, technical writing is particularly important: they need to master both engineering and scientific approaches to written communication across multiple formats. We have previously developed evidence-based technical writing modules, tailored to biomedical students, and vertically integrated them throughout our core curriculum [2]. These modules were developed prior to the recent advent of publicly available AI. To develop guidelines on instructional AI use, we first need to understand 1) student perception on the utility and ethics of AI, 2) student prior and current use of AI, and 3) how proficient AI is in providing students with adequate feedback.

To test student perceptions and use of AI, pre- and post- course surveys were administered to second- and third-year students in our department enrolled in writing-intensive lab courses (Biomedical Mechanics, Biomaterials, Human Physiology). The anonymous pre-course survey asked students to describe their experience level with using AI, how they have previously used AI in college, and their opinion on how ethical and useful AI is for an array of common technical writing assignments on a 4-point Likert scale (strongly agree, agree, disagree, strongly disagree). Students were informed by their instructor of record that use of AI on their writing assignments was permitted without penalty if they cited the name of the AI tool used and a description of how the tool was used. Data on student use of AI was collected both through the number of writing submissions that cited AI and the post-course survey. The post-course survey asked students about their use of AI during the semester and whether they felt AI was an effective tool. Total number of students who cited the use of AI and numbers who anonymously attested to using AI were compared. A two-sample nonparametric Wilcoxon Mann-Whitney Test will be used to make group comparisons between 1) student perceptions and use pre- vs post- course survey and 2) student in-semester reporting of AI use vs anonymously reported AI use in post survey. To determine AI’s efficacy in providing appropriate feedback on student writing we aim to use student submissions and instructor feedback to verify and test how closely AI feedback matches the instructor provided feedback. Ultimately, we believe these insights will enable us to provide guidelines for using AI as an instructional tool that we can incorporate into existing technical writing modules.

XXX University’s Institutional Review Board has reviewed and approved the procedures of this study. To date 100 students, 32 second year students, and 68 third year students have enrolled in the study.

Citation
Format

Adkins, A. N., & Husseini, N. S., & Cartee, L. (2024, June), Board 20: Work in Progress: Understanding Student Perceptions and Use of Generative Artificial Intelligence for Technical Writing Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/46766

TY - CPAPER
AB - With the improvement in open generative artificial intelligence’s (AI’s) ability to craft human-like text, there is a valid concern by educators that this technology will be used by students to complete assignments without learning the subject matter. Also problematic is the limited ability to verify or prove a student’s potentially unauthorized or unethical use of AI. While these fears are valid, we believe the best way forward is to focus on educating students on how to use this powerful technology ethically and effectively. Though there is some work establishing best practices for using AI in writing scientific manuscripts [1], how best to utilize AI as an instructional aid for teaching scientific writing is less understood. For biomedical engineers, technical writing is particularly important: they need to master both engineering and scientific approaches to written communication across multiple formats. We have previously developed evidence-based technical writing modules, tailored to biomedical students, and vertically integrated them throughout our core curriculum [2]. These modules were developed prior to the recent advent of publicly available AI. To develop guidelines on instructional AI use, we first need to understand 1) student perception on the utility and ethics of AI, 2) student prior and current use of AI, and 3) how proficient AI is in providing students with adequate feedback.

To test student perceptions and use of AI, pre- and post- course surveys were administered to second- and third-year students in our department enrolled in writing-intensive lab courses (Biomedical Mechanics, Biomaterials, Human Physiology). The anonymous pre-course survey asked students to describe their experience level with using AI, how they have previously used AI in college, and their opinion on how ethical and useful AI is for an array of common technical writing assignments on a 4-point Likert scale (strongly agree, agree, disagree, strongly disagree). Students were informed by their instructor of record that use of AI on their writing assignments was permitted without penalty if they cited the name of the AI tool used and a description of how the tool was used. Data on student use of AI was collected both through the number of writing submissions that cited AI and the post-course survey. The post-course survey asked students about their use of AI during the semester and whether they felt AI was an effective tool. Total number of students who cited the use of AI and numbers who anonymously attested to using AI were compared. A two-sample nonparametric Wilcoxon Mann-Whitney Test will be used to make group comparisons between 1) student perceptions and use pre- vs post- course survey and 2) student in-semester reporting of AI use vs anonymously reported AI use in post survey. To determine AI’s efficacy in providing appropriate feedback on student writing we aim to use student submissions and instructor feedback to verify and test how closely AI feedback matches the instructor provided feedback. Ultimately, we believe these insights will enable us to provide guidelines for using AI as an instructional tool that we can incorporate into existing technical writing modules.

XXX University’s Institutional Review Board has reviewed and approved the procedures of this study. To date 100 students, 32 second year students, and 68 third year students have enrolled in the study.
AU - Amy N Adkins
AU - Naji S Husseini
AU - Lianne Cartee
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Board 20: Work in Progress: Understanding Student Perceptions and Use of Generative Artificial Intelligence for Technical Writing
UR - https://peer.asee.org/46766
ER -