Board 19: Work in Progress: Towards Self-reported Student Usage of AI to Direct Curriculum in Technical Communication Courses

Kavon Karrobi; Angela Lai

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Biomedical Engineering Division (BED) Poster Session
Tagged Division: Biomedical Engineering Division (BED)
Permanent URL: https://peer.asee.org/46754

Request a correction

Paper Authors

biography

Kavon Karrobi Boston University orcid.org/0000-0001-7034-6937

visit author page

Kavon Karrobi is a Lecturer in the Department of Biomedical Engineering, as well as the Manager of the Bioengineering Technology & Entrepreneurship Center (BTEC) at Boston University. As a Lecturer in BME, Kavon teaches and mentors students in courses on biomedical measurements, analysis, and instrumentation. As Manager of BTEC, Kavon provides guidance, training, and mentorship of student projects that use BTEC ranging from student-initiated projects to senior design projects in the areas of biosensors and instrumentation, molecular/cellular/tissue engineering, and digital and predicative medicine. Kavon received his PhD in Biomedical Engineering with focus in biomedical optics from Boston University under the mentorship of Darren Roblyer. In addition to his research activities in biomedical optics, Kavon is working on collaborative research efforts at the intersection of Artificial Intelligence (AI) and BME education to understand how and why BME students use AI, as well as the potential opportunities and challenges AI may present in BME education.

visit author page

biography

Angela Lai Tufts University

visit author page

I am a current Assistant Teaching Professor at Tufts University in the Department of Biomedical Engineering. I am involved in mentoring students in both the laboratory and in the classroom and have research interests in peer feedback, team dynamics, and incorporating more translatable skills to my classes. Currently, I teach senior capstone, research and experimental design, and medical device design.

visit author page

Download Paper | Permalink

Abstract

The rate at which generative AI is entering the education space is vastly outpacing the rate at which institutional policies are being crafted to address how best to handle these disruptive tools in courses. Consequently, both faculty and students alike are left to their own devices and judgements on how to best navigate the role (if any) that generative AI tools will play in the experience and learning outcomes in their courses. It is yet to be revealed what impact such tools will have and how course dependent the impact may be. Accordingly, the proposed work described herein aims to evaluate the possible roles and impacts generative AI will have in Biomedical Engineering (BME) courses that require extensive technical and scientific writing. In such courses, a policy on student use of generative A has been adopted in which students fill out and submit a “Generative AI Assistance Disclosure” (GAIA) form with every assignment. On the form, the students indicate whether they used generative AI tool(s) as part of working on a given assignment. If the answer is “No”, then students are not required to fill out the rest of the form. If they select “Yes”, then students must fill out the remaining sections of the form, which include the following: (i) naming the generative AI tool(s) used, (ii) explaining how and why the tool(s) were used for the assignment, and (iii) providing the entire exchange with the generative AI tool(s). Assessments generally involve writing technical reports and conducting literature searches. When students report the usage of AI tools, the GAIA form offers options such as writing and/or coding support, aiding in understanding and explaining concepts, and saving time or reducing stress. The purpose of the proposed work is to establish a baseline understanding of student use of generative AI tools and their impact specifically in the context of BME courses with technical and scientific writing components, which will involve both quantitative and qualitative analyses and insights.

The authors are interested in how generative AI tools can enhance students’ experiences and learning outcomes related to course objectives. The long-term goal of the study is to collect and analyze cross-institutional data to investigate and compare AI usage by BME students from different institutions, and the investigators are currently working towards a multi-institution Institutional Review Board approval for this purpose. Preliminary results from this proposed work will prove useful in starting to gather data and extract insights around the role and impact of generative AI in BME courses with a technical writing component. We aim to conduct qualitative and quantitative analyses on the GAIA disclosure form responses. Some quantitative explorations include but are not limited to: (i) analyzing what percentage of students used versus did not use generative AI tools and how those percentages might change between assignments and topics; (ii) comparing individual assignment performance between students who used to students who did not use generative AI tools; and (iii) assessing overall student performances in the courses as a function of percent AI usage across all assignments. Some qualitative explorations include but are not limited to: (i) investigating the main ways in which students were using generative AI tools (the how and why on the GAIA forms), and thus the main ways in which the students potentially found the tools most useful with respect to course material and assessments; (ii) identifying potential areas where students need the most help with respect to core skills and competencies in the courses based on how students used generative AI tools; (iii) understanding the degree of complexity with respect to the exchanges between students and generative AI tools.

Outcomes from this study will also contribute to an ever-evolving understanding of the opportunities and challenges generative AI will present in BME education and education more broadly, and what frameworks and policies should be considered as the community navigates these pervasive and powerful tools.

Citation
Format

Karrobi, K., & Lai, A. (2024, June), Board 19: Work in Progress: Towards Self-reported Student Usage of AI to Direct Curriculum in Technical Communication Courses Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/46754

TY  - CPAPER
AB  - The rate at which generative AI is entering the education space is vastly outpacing the rate at which institutional policies are being crafted to address how best to handle these disruptive tools in courses. Consequently, both faculty and students alike are left to their own devices and judgements on how to best navigate the role (if any) that generative AI tools will play in the experience and learning outcomes in their courses. It is yet to be revealed what impact such tools will have and how course dependent the impact may be. Accordingly, the proposed work described herein aims to evaluate the possible roles and impacts generative AI will have in Biomedical Engineering (BME) courses that require extensive technical and scientific writing. In such courses, a policy on student use of generative A has been adopted in which students fill out and submit a “Generative AI Assistance Disclosure” (GAIA) form with every assignment. On the form, the students indicate whether they used generative AI tool(s) as part of working on a given assignment. If the answer is “No”, then students are not required to fill out the rest of the form. If they select “Yes”, then students must fill out the remaining sections of the form, which include the following: (i) naming the generative AI tool(s) used, (ii) explaining how and why the tool(s) were used for the assignment, and (iii) providing the entire exchange with the generative AI tool(s). Assessments generally involve writing technical reports and conducting literature searches. When students report the usage of AI tools, the GAIA form offers options such as writing and/or coding support, aiding in understanding and explaining concepts, and saving time or reducing stress. The purpose of the proposed work is to establish a baseline understanding of student use of generative AI tools and their impact specifically in the context of BME courses with technical and scientific writing components, which will involve both quantitative and qualitative analyses and insights. 

The authors are interested in how generative AI tools can enhance students’ experiences and learning outcomes related to course objectives. The long-term goal of the study is to collect and analyze cross-institutional data to investigate and compare AI usage by BME students from different institutions, and the investigators are currently working towards a multi-institution Institutional Review Board approval for this purpose. Preliminary results from this proposed work will prove useful in starting to gather data and extract insights around the role and impact of generative AI in BME courses with a technical writing component. We aim to conduct qualitative and quantitative analyses on the GAIA disclosure form responses. Some quantitative explorations include but are not limited to: (i) analyzing what percentage of students used versus did not use generative AI tools and how those percentages might change between assignments and topics; (ii) comparing individual assignment performance between students who used to students who did not use generative AI tools; and (iii) assessing overall student performances in the courses as a function of percent AI usage across all assignments. Some qualitative explorations include but are not limited to: (i) investigating the main ways in which students were using generative AI tools (the how and why on the GAIA forms), and thus the main ways in which the students potentially found the tools most useful with respect to course material and assessments; (ii) identifying potential areas where students need the most help with respect to core skills and competencies in the courses based on how students used generative AI tools; (iii) understanding the degree of complexity with respect to the exchanges between students and generative AI tools.

Outcomes from this study will also contribute to an ever-evolving understanding of the opportunities and challenges generative AI will present in BME education and education more broadly, and what frameworks and policies should be considered as the community navigates these pervasive and powerful tools.

AU  - Kavon Karrobi
AU  - Angela Lai
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - Board 19: Work in Progress: Towards Self-reported Student Usage of AI to Direct Curriculum in Technical Communication Courses
UR  - https://peer.asee.org/46754
ER  -