Asee peer logo

Paradigm Shift? Preliminary Findings of Engineering Faculty Members’ Mental Models of Assessment in the Era of Generative AI

Download Paper |

Conference

2024 ASEE Annual Conference & Exposition

Location

Portland, Oregon

Publication Date

June 23, 2024

Start Date

June 23, 2024

End Date

July 12, 2024

Conference Session

DSA Technical Session 7

Tagged Topic

Data Science & Analytics Constituent Committee (DSA)

Permanent URL

https://peer.asee.org/47829

Request a correction

Paper Authors

biography

Isil Anakok Virginia Polytechnic Institute and State University

visit author page

Ms.Anakok is Ph.D. candidate in the Department of Engineering Education at Virginia Tech. She has a Ms. degree in Mechanical Engineering at Virginia Tech, and Bs. in Mechatronics Engineering from Kocaeli University, Turkey.

visit author page

biography

Kai Jun Chew Embry-Riddle Aeronautical University, Daytona Beach

visit author page

Kai Jun "KJ" Chew is an assistant professor in the Engineering Fundamentals department at Embry-Riddle Aeronautical University. He is passionate about teaching and research, and he strives to produce knowledge that informs better teaching. His research intersects assessment and evaluation, motivation, and equity. His research goal is to promote engineering as a way to advance social justice causes.

visit author page

biography

Holly M Matusovich Virginia Polytechnic Institute and State University

visit author page

Dr. Holly Matusovich is the Associate Dean for Graduate and Professional Studies in the College of Engineering at Virginia Tech and a Professor in the Department of Engineering Education where she has also served in key leadership positions. Dr. Matusovich is recognized for her research and leadership related to graduate student mentoring and faculty development. She won the Hokie Supervisor Spotlight Award in 2014, received the College of Engineering Graduate Student Mentor Award in 2018, and was inducted into the Virginia Tech Academy of Faculty Leadership in 2020. Dr. Matusovich has been a PI/Co-PI on 19 funded research projects including the NSF CAREER Award, with her share of funding being nearly $3 million. She has co-authored 2 book chapters, 34 journal publications, and more than 80 conference papers. She is recognized for her research and teaching, including Dean’s Awards for Outstanding New Faculty, Outstanding Teacher Award, and a Faculty Fellow. Dr. Matusovich has served the Educational Research and Methods (ERM) division of ASEE in many capacities over the past 10+ years including serving as Chair from 2017-2019. Dr. Matusovich is currently the Editor-in-Chief of the journal, Advances in Engineering Education and she serves on the ASEE committee for Scholarly Publications.

visit author page

biography

Andrew Katz Virginia Polytechnic Institute and State University

visit author page

Andrew Katz is an assistant professor in the Department of Engineering Education at Virginia Tech. He leads the Improving Decisions in Engineering Education Agents and Systems (IDEEAS) Lab.

visit author page

Download Paper |

Abstract

The emergence of generative artificial intelligence (GAI) has started to introduce a fundamental reexamination of established teaching methods. These GAI systems offer a chance for both educators and students to reevaluate their academic endeavors. Reevaluation of current practices is particularly pertinent in assessment within engineering instruction, where advanced generative text algorithms are proficient in addressing intricate challenges like those found in engineering courses. While this juncture presents a moment to revisit general assessment methods, the actual response of faculty to the incorporation of GAI in their evaluative techniques remains unclear. To investigate this, we have initiated a study delving into the mental constructs that engineering faculty hold about evaluation, focusing on their evolving attitudes and responses to GAI, as reported in the Fall of 2023. Adopting a long-term data-gathering strategy, we conducted a series of surveys, interviews, and recordings targeting the evaluative decision-making processes of a varied group of engineering educators across the United States. This paper presents the data collection process, our participants’ demographics, our data analysis plan, and initial findings based on the participants’ backgrounds, followed by our future work and potential implications. The analysis of the collected data will utilize qualitative thematic analysis. Once we complete our study, we believe our findings will sketch the early stages of this emerging paradigm shift in the assessment of undergraduate engineering education, offering a novel perspective on the discourse surrounding evaluation strategies in the field. These insights are vital for stakeholders such as policymakers, educational leaders, and instructors, as they have significant ramifications for policy development, curriculum planning, and the broader dialogue on integrating GAI into educational evaluation.

Anakok, I., & Chew, K. J., & Matusovich, H. M., & Katz, A. (2024, June), Paradigm Shift? Preliminary Findings of Engineering Faculty Members’ Mental Models of Assessment in the Era of Generative AI Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/47829

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2024 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015