Baltimore , Maryland
June 25, 2023
June 25, 2023
June 28, 2023
Educational Research and Methods Division (ERM)
Diversity
14
10.18260/1-2--42737
https://peer.asee.org/42737
178
Dr. Vanessa Svihla is a learning scientist and associate professor at the University of New Mexico in the Organization, Information and Learning Sciences program and in the Chemical and Biological Engineering Department.
Andrew Olewnik is an Assistant Professor in the Department of Engineering Education at the University at Buffalo. His research includes undergraduate engineering education with focus on engineering design, problem-based learning, co-curricular involvement and its impact on professional formation, and the role of reflection practices in supporting engineering undergraduates as they transition from student to professional.
In this research paper, we investigate the structure and validity of survey data related to students’ framing agency. In order to promote increased opportunities for students to engage in and learn to frame design problems that are innovative and empathetic, there is a need for instruments that can provide information about student progress and the quality of learning experiences. This is a complex problem because, compared to problem solving, design problem framing is less studied and harder to predict due to the higher levels of student agency involved. To address this issue, we developed a survey to measure framing agency, which is defined as opportunities to frame and reframe design problems and learn in the process. This study extends past research which focused on the construct of framing agency and developing an instrument to measure it following best practices in survey design, including using exploratory factor analysis of pilot data, which recovered six factors related to shared and individual consequentiality, problem structure and constrainedness, and learning. However, as a pilot, the sample limited generalizability; the current study addresses this limitation. We used a national cohort that included multiple engineering disciplines (biomedical, mechanical, chemical, electrical, computer, aerospace), types of formal design projects (e.g., first-year, design-spine, senior capstone) and institution types, including private religious; Hispanic-serving; public land-grant; and research flagship institutions (N=449). We report sample characteristics and used confirmatory factor analysis (CFA) to provide validity evidence, reporting the chi-square and standardized root mean square residual as estimates of fit. We report Cronbach’s alpha as a measure of internal consistency. We found that overall, the CFA aligned with the prior exploratory results, in this case, recovering four factors, measured on a seven-point scale: shared consequentiality (the extent to which the student identifies that their understanding of the problem changed as result of a teammate’s decision, M = 6.15; SD = 1.13); learning as consequentiality (the extent to which the student identifies learning as the result of decisions, M = 5.88; SD = 0.98); constrainedness (the extent to which the student reports the ability to make decisions despite design constraints, M = 4.95; SD = 1.49); and shared tentativeness (the extent to which the student identifies uncertainty about the problem and solution, M = 4.02; SD = 1.76). This suggests the survey can provide valid data for instructional decisions and further research into how students learn to frame engineering design problems and what role framing plays in their professional formation.
Wilson-Fetrow, M., & Svihla, V., & Olewnik, A. (2023, June), Confirmatory factor analysis of the framing agency survey Paper presented at 2023 ASEE Annual Conference & Exposition, Baltimore , Maryland. 10.18260/1-2--42737
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2023 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015