Board 245: Does Integrating Innovative Technologies into STEM Education Help Advance K-12 Students’ STEM Career Outcomes? A Synthesis Study

Yue Li; Maressa L. Dixon; Sarah Woodruff

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: June 26, 2024
Conference Session: NSF Grantees Poster Session
Tagged Topic: NSF Grantees Poster Session
Page Count: 7
DOI: 10.18260/1-2--46816
Permanent URL: https://peer.asee.org/46816
Download Count: 180

Paper Authors

biography

Yue Li Miami University orcid.org/0000-0002-0508-6451

visit author page

Dr. Yue Li, Associate Director of the Discovery Center for Evaluation, Research, and Professional Learning at Miami University (Oxford, OH), has extensive quantitative research skills and experiences of leading research and evaluation projects related to systemic reform efforts, with specific proficiency in equitable and inclusive STEM environments. Over the last 17 years at the Discovery Center, Dr. Li has conducted research or evaluation for more than 60 STEM education related projects funded by NSF (e.g., projects funded through the ADVANCE, AISL, DRK-12, ITEST, IUSE, Noyce, STC, and STEM+C programs), state, and local agencies. She has experiences working as a member of the evaluation team on several projects focusing on engineering education at K-12 or post-secondary levels. She is currently serving as the PI of an NSF funded ITEST Synthesis project (Award #1949437), co-PI of an NSF-funded DRK-12 project (Award #2010351), Senior Personnel (mentor) of an NSF-funded RIEF project (Award #2306176), and the lead evaluator for numerous evaluation projects.

visit author page

biography

Maressa L. Dixon Miami University

visit author page

Ms. Maressa L. Dixon (M.A.), is a Senior Research Associate with the Discovery Center for Evaluation, Research, and Professional Learning at Miami University (Oxford, OH). Ms. Dixon is a mixed-methods researcher with expertise in qualitative research and significant experience with multiple methods for the collection, analysis, and synthesis of quantitative and qualitative data. She has led qualitative analyses for more than twenty externally-funded research and evaluation projects, primarily focused on K-16+ education. Her research interests include education and workforce outcomes for students underrepresented in STEM, racial equity, critical and transformative inquiry approaches, innovative qualitative methods, and applied mixed methods.

visit author page

author page

Sarah Woodruff orcid.org/0000-0001-5634-9517

Download Paper | Permalink

Abstract

Funded through the NSF ITEST program, the primary objective of this mixed-methods meta-analysis and qualitative synthesis study is to review and synthesize research and evaluation findings demonstrating the effects of integrating innovative technologies and technology-based learning experiences in STEM education on K-12 students’ STEM career-related outcomes. This study synthesizes the rigorous intervention research on Grades K-12 students’ STEM career-related outcomes from 1995 to the present and across characteristics of innovative technology-based STEM education interventions, learning contexts, student demographics, and study designs. This study advances understanding of effects of integrating innovative educational technologies and technology-based learning experiences in PreK-12 classrooms on students' STEM career outcomes, enables generalization of the magnitudes and variations of effects on students, what settings specifies what technologies and interventions have been effective for which groups of students and in, and provides insight to how and why such interventions produced positive outcomes. The method for conducting this study follows steps common to meta-analysis and qualitative synthesis studies, that is, establishing inclusion/exclusion criteria and search terms following the PICOS framework (i.e., participant, intervention, comparison condition/study design, outcome, and setting), conducting database searches, screening for study inclusion, coding, and conducting analyses.

The preliminary meta-analysis involved 94 effect sizes from 42 primary studies, most published within the past decade. These studies encompassed 13,069 student participants, primarily in middle or high school. STEM career-related outcomes were measured as dispositions, including interest, aspiration, motivation, confidence, and self-efficacy. A small number of studies also assessed knowledge in specific STEM careers. Overall, a small positive effect was observed (effect size mean = 0.189, SE=0.046, 95%CI = 0.099 – 0.280, p < .001), with significant heterogeneity (Q = 932.51, p < .001, I2 = 0.94), suggesting the need to explore potential moderator variables. Intervention characteristics revealed that 48% targeted underrepresented and/or underserved populations, 40% included explicit career development, and interdisciplinary content was common. Additionally, 60% of studies took place in informal settings. The study also considered intervention format, duration, pedagogical practices, study design, and publication type as potential moderators in the final analysis. The preliminary qualitative synthesis revealed that 21 of 27 studies reported student outcomes along at least one of two trajectories: prior STEM career interest sustained or strengthened after participation in the intervention (“STEM-supported”) or STEM career interest piqued by participation in the intervention (“STEM-Influenced”). Because these trajectories were not clearly associated with potential moderators, more analysis is required to further explore the extent to which combinations of moderators are more or less associated with different STEM career development trajectories, including trajectories that did not result in greater interest in STEM career fields.

Citation
Format

Li, Y., & Dixon, M. L., & Woodruff, S. (2024, June), Board 245: Does Integrating Innovative Technologies into STEM Education Help Advance K-12 Students’ STEM Career Outcomes? A Synthesis Study Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--46816

TY  - CPAPER
AB  - Funded through the NSF ITEST program, the primary objective of this mixed-methods meta-analysis and qualitative synthesis study is to review and synthesize research and evaluation findings demonstrating the effects of integrating innovative technologies and technology-based learning experiences in STEM education on K-12 students’ STEM career-related outcomes. This study synthesizes the rigorous intervention research on Grades K-12 students’ STEM career-related outcomes from 1995 to the present and across characteristics of innovative technology-based STEM education interventions, learning contexts, student demographics, and study designs. This study advances understanding of effects of integrating innovative educational technologies and technology-based learning experiences in PreK-12 classrooms on students&#39; STEM career outcomes, enables generalization of the magnitudes and variations of effects on students, what settings specifies what technologies and interventions have been effective for which groups of students and in, and provides insight to how and why such interventions produced positive outcomes. The method for conducting this study follows steps common to meta-analysis and qualitative synthesis studies, that is, establishing inclusion/exclusion criteria and search terms following the PICOS framework (i.e., participant, intervention, comparison condition/study design, outcome, and setting), conducting database searches, screening for study inclusion, coding, and conducting analyses. 

The preliminary meta-analysis involved 94 effect sizes from 42 primary studies, most published within the past decade. These studies encompassed 13,069 student participants, primarily in middle or high school. STEM career-related outcomes were measured as dispositions, including interest, aspiration, motivation, confidence, and self-efficacy. A small number of studies also assessed knowledge in specific STEM careers. Overall, a small positive effect was observed (effect size mean = 0.189, SE=0.046, 95%CI = 0.099 – 0.280, p &amp;lt; .001), with significant heterogeneity (Q = 932.51, p &amp;lt; .001, I2 = 0.94), suggesting the need to explore potential moderator variables. Intervention characteristics revealed that 48% targeted underrepresented and/or underserved populations, 40% included explicit career development, and interdisciplinary content was common. Additionally, 60% of studies took place in informal settings. The study also considered intervention format, duration, pedagogical practices, study design, and publication type as potential moderators in the final analysis. 
 
The preliminary qualitative synthesis revealed that 21 of 27 studies reported student outcomes along at least one of two trajectories: prior STEM career interest sustained or strengthened after participation in the intervention (“STEM-supported”) or STEM career interest piqued by participation in the intervention (“STEM-Influenced”). Because these trajectories were not clearly associated with potential moderators, more analysis is required to further explore the extent to which combinations of moderators are more or less associated with different STEM career development trajectories, including trajectories that did not result in greater interest in STEM career fields.

AU  - Yue Li
AU  - Maressa L. Dixon
AU  - Sarah Woodruff
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - Board 245: Does Integrating Innovative Technologies into STEM Education Help Advance K-12 Students’ STEM Career Outcomes? A Synthesis Study
UR  - https://peer.asee.org/46816
DO  - 10.18260/1-2--46816
ER  -