Work in Progress: A Systematic Literature Review of Person-Centered Approaches and Data-Driven Methods in Engineering Education Research

Jiafu Niu; David Reeping

Download Paper | Permalink

Conference: 2023 ASEE Annual Conference & Exposition
Location: Baltimore , Maryland
Publication Date: June 25, 2023
Start Date: June 25, 2023
End Date: June 28, 2023
Conference Session: Work-in-Progress Session: Emergent Methods for Engineering Education Research
Tagged Division: Educational Research and Methods Division (ERM)
Tagged Topic: Diversity
Page Count: 9
DOI: 10.18260/1-2--44164
Permanent URL: https://peer.asee.org/44164
Download Count: 204

Paper Authors

biography

Jiafu Niu University of Cincinnati

visit author page

Jiafu Niu is a M.S. Student in Engineering Education at the University of Cincinnati. He holds a B.S. in Applied Statistics and Actuarial Science from Purdue University.

visit author page

biography

David Reeping University of Cincinnati orcid.org/0000-0002-0803-7532

visit author page

Dr. David Reeping is an Assistant Professor in the Department of Engineering and Computing Education at the University of Cincinnati. He earned his Ph.D. in Engineering Education from Virginia Tech and was a National Science Foundation Graduate Research Fellow. He received his B.S. in Engineering Education with a Mathematics minor from Ohio Northern University. His main research interests include transfer student information asymmetries, threshold concepts, curricular complexity, and advancing quantitative and fully integrated mixed methods.

visit author page

Download Paper | Permalink

Abstract

In this Work in Progress (WIP) paper, we describe preliminary results from a systematic literature review (SLR) examining how engineering education researchers humanize data-driven quantitative methods by using “person-centered” approaches. Engineering education researchers commonly use quantitative methods to seek causal or correlational relationships between observed phenomena of interest. However, under the assumptions of many traditional statistical methods, there is a tendency to minimize the voice of underrepresented engineering student groups because of low sample sizes. This minimization is accomplished by grouping underrepresented students into heterogenous categories or removing them altogether as outliers. As a result, these methods can incorrectly generalize the findings based on the dominant group to the entire engineering student body. Moreover, data-driven research is often associated with objectivity, encapsulated by the phrase “letting the data speak for itself.” Drawing conclusions from incomplete demographic samples and generalizing biased results can further damage the benefits of minoritized groups in the engineering student body and exacerbate the existing inequality.

Despite these challenges, thoughtfully applied data-driven methods have the potential to value the experience of minoritized individuals. In particular, Godwin et al. (2021) introduce the concept of person-centered approaches to the community. A person-centered approach recognizes heterogeneity and attempts to find relationships among variables to form latent groupings among individuals in the sample. More traditional statistical analyses, called variable-centered approaches, are concerned with prediction and work best with homogeneity in the sample. To explore how engineering education researchers employed person-centered approaches, we followed Borrego et al.’s (2014) process for conducting a systematic literature review (I.e., PRISMA) to analyze publications from four journals from the last decade (2011-2021): Journal of Engineering Education, European Journal of Engineering Education, International Journal of Engineering Education, and IEEE Transactions on Education. We formed our search string based on common data-driven techniques, such as cluster analysis and random forest, including more general keywords like machine learning. The initial data set yielded by the search was 236 articles. After screening abstracts to remove articles that did not apply the techniques to human subjects and were not empirical articles employing data-driven methods, we retained 98 articles. A full-text review using the same exclusion criteria removed 19 articles, resulting in our final dataset (n = 79).

Our analyses are still in progress. The planned analysis is to apply the principles of QuantCrit (Garcia et al., 2018; Gillborn et al., 2018) combined with a categorization scheme of person and variable-centered analyses to elicit themes of how engineering education researchers make analytical decisions when applying data-driven methods. We anticipate that this SLR could help the field to understand how researchers use data-driven quantitative methods and offset the issues with these methods using person-centered approaches. This SLR also will reveal what kinds of engineering education problems are more likely to be studied with data-driven methods.

References:

Borrego, M., Foster, M. J., & Froyd, J. E. (2014). Systematic Literature Reviews in Engineering Education and Other Developing Interdisciplinary Fields: Systematic Literature Reviews in Engineering Education. Journal of Engineering Education, 103(1), 45–76. https://doi.org/10.1002/jee.20038

Garcia, N. M., López, N., & Vélez, V. N. (2018). QuantCrit: Rectifying quantitative methods through critical race theory. Race Ethnicity and Education, 21(2), 149–157. https://doi.org/10.1080/13613324.2017.1377675

Gillborn, D., Warmington, P., & Demack, S. (2018). QuantCrit: Education, policy, ‘Big Data’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179. https://doi.org/10.1080/13613324.2017.1377417

Godwin, A., Benedict, B., Rohde, J., Thielmeyer, A., Perkins, H., Major, J., Clements, H., & Chen, Z. (2021). New Epistemological Perspectives on Quantitative Methods: An Example Using Topological Data Analysis. Studies in Engineering Education, 2(1), 16-34. https://doi.org/10.21061/see.18

Citation
Format

Niu, J., & Reeping, D. (2023, June), Work in Progress: A Systematic Literature Review of Person-Centered Approaches and Data-Driven Methods in Engineering Education Research Paper presented at 2023 ASEE Annual Conference & Exposition, Baltimore , Maryland. 10.18260/1-2--44164

TY - CPAPER
AB - In this Work in Progress (WIP) paper, we describe preliminary results from a systematic literature review (SLR) examining how engineering education researchers humanize data-driven quantitative methods by using “person-centered” approaches. Engineering education researchers commonly use quantitative methods to seek causal or correlational relationships between observed phenomena of interest. However, under the assumptions of many traditional statistical methods, there is a tendency to minimize the voice of underrepresented engineering student groups because of low sample sizes. This minimization is accomplished by grouping underrepresented students into heterogenous categories or removing them altogether as outliers. As a result, these methods can incorrectly generalize the findings based on the dominant group to the entire engineering student body. Moreover, data-driven research is often associated with objectivity, encapsulated by the phrase “letting the data speak for itself.” Drawing conclusions from incomplete demographic samples and generalizing biased results can further damage the benefits of minoritized groups in the engineering student body and exacerbate the existing inequality.

Despite these challenges, thoughtfully applied data-driven methods have the potential to value the experience of minoritized individuals. In particular, Godwin et al. (2021) introduce the concept of person-centered approaches to the community. A person-centered approach recognizes heterogeneity and attempts to find relationships among variables to form latent groupings among individuals in the sample. More traditional statistical analyses, called variable-centered approaches, are concerned with prediction and work best with homogeneity in the sample. To explore how engineering education researchers employed person-centered approaches, we followed Borrego et al.’s (2014) process for conducting a systematic literature review (I.e., PRISMA) to analyze publications from four journals from the last decade (2011-2021): Journal of Engineering Education, European Journal of Engineering Education, International Journal of Engineering Education, and IEEE Transactions on Education. We formed our search string based on common data-driven techniques, such as cluster analysis and random forest, including more general keywords like machine learning. The initial data set yielded by the search was 236 articles. After screening abstracts to remove articles that did not apply the techniques to human subjects and were not empirical articles employing data-driven methods, we retained 98 articles. A full-text review using the same exclusion criteria removed 19 articles, resulting in our final dataset (n = 79).

Our analyses are still in progress. The planned analysis is to apply the principles of QuantCrit (Garcia et al., 2018; Gillborn et al., 2018) combined with a categorization scheme of person and variable-centered analyses to elicit themes of how engineering education researchers make analytical decisions when applying data-driven methods. We anticipate that this SLR could help the field to understand how researchers use data-driven quantitative methods and offset the issues with these methods using person-centered approaches. This SLR also will reveal what kinds of engineering education problems are more likely to be studied with data-driven methods.

References:

Borrego, M., Foster, M. J., &amp;amp; Froyd, J. E. (2014). Systematic Literature Reviews in Engineering Education and Other Developing Interdisciplinary Fields: Systematic Literature Reviews in Engineering Education. Journal of Engineering Education, 103(1), 45–76. https://doi.org/10.1002/jee.20038

Garcia, N. M., López, N., &amp;amp; Vélez, V. N. (2018). QuantCrit: Rectifying quantitative methods through critical race theory. Race Ethnicity and Education, 21(2), 149–157. https://doi.org/10.1080/13613324.2017.1377675

Gillborn, D., Warmington, P., &amp;amp; Demack, S. (2018). QuantCrit: Education, policy, ‘Big Data’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179. https://doi.org/10.1080/13613324.2017.1377417

Godwin, A., Benedict, B., Rohde, J., Thielmeyer, A., Perkins, H., Major, J., Clements, H., &amp;amp; Chen, Z. (2021). New Epistemological Perspectives on Quantitative Methods: An Example Using Topological Data Analysis. Studies in Engineering Education, 2(1), 16-34. https://doi.org/10.21061/see.18

AU - Jiafu Niu
AU - David Reeping
CY - Baltimore , Maryland
DA - 2023/06/25
PB - ASEE Conferences
TI - Work in Progress: A Systematic Literature Review of Person-Centered Approaches and Data-Driven Methods in Engineering Education Research
UR - https://peer.asee.org/44164
DO - 10.18260/1-2--44164
ER -