Asee peer logo

Developing a Program to Assist in Qualitative Data Analysis: How Engineering Students’ Discuss Model Types

Download Paper |

Conference

2022 ASEE Annual Conference & Exposition

Location

Minneapolis, MN

Publication Date

August 23, 2022

Start Date

June 26, 2022

End Date

June 29, 2022

Conference Session

ERM: Exploring Educational Technology in Engineering

Page Count

17

Permanent URL

https://peer.asee.org/41032

Download Count

19

Request a correction

Paper Authors

biography

Kelsey Rodgers Embry-Riddle Aeronautical University - Daytona Beach

visit author page

Dr. Kelsey Joy Rodgers is an engineer, engineering educator, and a researcher. She previously worked as an Assistant Professor and Visiting Research Scholar for Embry-Riddle Aeronautical University. She received her PhD in Engineering Education from Purdue University and her Bachelor's in Mechanical Engineering from Arizona State University - Polytechnic campus.

Follow her on YouTube: https://www.youtube.com/c/EngineeringwithDrKelseyJoy

visit author page

biography

Matthew Verleger Embry-Riddle Aeronautical University - Daytona Beach

visit author page

Matthew Verleger is a Professor of Engineering Fundamentals at Embry-Riddle Aeronautical University. He received his PhD in Engineering Education from Purdue University in 2010. His research interests include student use of models and modeling, flipped-classroom environments, development of educational software, and gamification of engineering courses.

visit author page

biography

Farshid Marbouti San Jose State University

visit author page

Farshid Marbouti is an Assistant Professor of General (interdisciplinary) Engineering at San Jose State University (SJSU). He is currently the chair of SJSU Senate Student Success Committee. Farshid completed his Ph.D. in Engineering Education at Purdue University. His research interests center on First-Year Engineering student success and engineering design.

visit author page

biography

Angela Thompson University of Louisville

visit author page

Associate Professor of Engineering Fundamentals and Director of Undergraduate Affairs, J. B. Speed School of Engineering, University of Louisville

visit author page

author page

Nicholas Hawkins University of Louisville

Download Paper |

Abstract

This Research paper discusses the opportunities that utilizing a computer program can present in analyzing large amounts of qualitative data collected through a survey tool. When working with longitudinal qualitative data, there are many challenges that researchers face. The coding scheme may evolve over time requiring re-coding of early data. There may be long periods of time between data analysis. Typically, multiple researchers will participate in the coding, but this may introduce bias or inconsistencies. Ideally the same researchers would be analyzing the data, but often there is some turnover in the team, particularly when students assist with the coding. Computer programs can enable automated or semi-automated coding helping to reduce errors and inconsistencies in the coded data.

In this study, a modeling survey was developed to assess student awareness of model types and administered in four first-year engineering courses across the three universities over the span of three years. The data collected from this survey consists of over 4,000 students’ open-ended responses to three questions about types of models in science, technology, engineering, and mathematics (STEM) fields. A coding scheme was developed to identify and categorize model types in student responses. Over two years, two undergraduate researchers analyzed a total of 1,829 students’ survey responses after ensuring intercoder reliability was greater than 80% for each model category. However, with much data remaining to be coded, the research team developed a MATLAB program to automatically implement the coding scheme and identify the types of models students discussed in their responses.

MATLAB coded results were compared to human-coded results (n = 1,829) to assess reliability; results matched between 81%-99% for the different model categories. Furthermore, the reliability of the MATLAB coded results are within the range of the interrater reliability measured between the 2 undergraduate researchers (86-100% for the five model categories).

With good reliability of the program, all 4,358 survey responses were coded; results showing the number and types of models identified by students are presented in the paper.

Keywords: first-year engineering, modeling, qualitative survey data, data analysis, MATLAB tool

Rodgers, K., & Verleger, M., & Marbouti, F., & Thompson, A., & Hawkins, N. (2022, August), Developing a Program to Assist in Qualitative Data Analysis: How Engineering Students’ Discuss Model Types Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. https://peer.asee.org/41032

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2022 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015