Baltimore , Maryland
June 25, 2023
June 25, 2023
June 28, 2023
Work-in-Progress Session: Emergent Methods for Engineering Education Research
Educational Research and Methods Division (ERM)
Diversity
9
10.18260/1-2--44164
https://peer.asee.org/44164
203
Jiafu Niu is a M.S. Student in Engineering Education at the University of Cincinnati. He holds a B.S. in Applied Statistics and Actuarial Science from Purdue University.
Dr. David Reeping is an Assistant Professor in the Department of Engineering and Computing Education at the University of Cincinnati. He earned his Ph.D. in Engineering Education from Virginia Tech and was a National Science Foundation Graduate Research Fellow. He received his B.S. in Engineering Education with a Mathematics minor from Ohio Northern University. His main research interests include transfer student information asymmetries, threshold concepts, curricular complexity, and advancing quantitative and fully integrated mixed methods.
In this Work in Progress (WIP) paper, we describe preliminary results from a systematic literature review (SLR) examining how engineering education researchers humanize data-driven quantitative methods by using “person-centered” approaches. Engineering education researchers commonly use quantitative methods to seek causal or correlational relationships between observed phenomena of interest. However, under the assumptions of many traditional statistical methods, there is a tendency to minimize the voice of underrepresented engineering student groups because of low sample sizes. This minimization is accomplished by grouping underrepresented students into heterogenous categories or removing them altogether as outliers. As a result, these methods can incorrectly generalize the findings based on the dominant group to the entire engineering student body. Moreover, data-driven research is often associated with objectivity, encapsulated by the phrase “letting the data speak for itself.” Drawing conclusions from incomplete demographic samples and generalizing biased results can further damage the benefits of minoritized groups in the engineering student body and exacerbate the existing inequality.
Despite these challenges, thoughtfully applied data-driven methods have the potential to value the experience of minoritized individuals. In particular, Godwin et al. (2021) introduce the concept of person-centered approaches to the community. A person-centered approach recognizes heterogeneity and attempts to find relationships among variables to form latent groupings among individuals in the sample. More traditional statistical analyses, called variable-centered approaches, are concerned with prediction and work best with homogeneity in the sample. To explore how engineering education researchers employed person-centered approaches, we followed Borrego et al.’s (2014) process for conducting a systematic literature review (I.e., PRISMA) to analyze publications from four journals from the last decade (2011-2021): Journal of Engineering Education, European Journal of Engineering Education, International Journal of Engineering Education, and IEEE Transactions on Education. We formed our search string based on common data-driven techniques, such as cluster analysis and random forest, including more general keywords like machine learning. The initial data set yielded by the search was 236 articles. After screening abstracts to remove articles that did not apply the techniques to human subjects and were not empirical articles employing data-driven methods, we retained 98 articles. A full-text review using the same exclusion criteria removed 19 articles, resulting in our final dataset (n = 79).
Our analyses are still in progress. The planned analysis is to apply the principles of QuantCrit (Garcia et al., 2018; Gillborn et al., 2018) combined with a categorization scheme of person and variable-centered analyses to elicit themes of how engineering education researchers make analytical decisions when applying data-driven methods. We anticipate that this SLR could help the field to understand how researchers use data-driven quantitative methods and offset the issues with these methods using person-centered approaches. This SLR also will reveal what kinds of engineering education problems are more likely to be studied with data-driven methods.
References:
Borrego, M., Foster, M. J., & Froyd, J. E. (2014). Systematic Literature Reviews in Engineering Education and Other Developing Interdisciplinary Fields: Systematic Literature Reviews in Engineering Education. Journal of Engineering Education, 103(1), 45–76. https://doi.org/10.1002/jee.20038
Garcia, N. M., López, N., & Vélez, V. N. (2018). QuantCrit: Rectifying quantitative methods through critical race theory. Race Ethnicity and Education, 21(2), 149–157. https://doi.org/10.1080/13613324.2017.1377675
Gillborn, D., Warmington, P., & Demack, S. (2018). QuantCrit: Education, policy, ‘Big Data’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179. https://doi.org/10.1080/13613324.2017.1377417
Godwin, A., Benedict, B., Rohde, J., Thielmeyer, A., Perkins, H., Major, J., Clements, H., & Chen, Z. (2021). New Epistemological Perspectives on Quantitative Methods: An Example Using Topological Data Analysis. Studies in Engineering Education, 2(1), 16-34. https://doi.org/10.21061/see.18
Niu, J., & Reeping, D. (2023, June), Work in Progress: A Systematic Literature Review of Person-Centered Approaches and Data-Driven Methods in Engineering Education Research Paper presented at 2023 ASEE Annual Conference & Exposition, Baltimore , Maryland. 10.18260/1-2--44164
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2023 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015