Building and Revising an Assessment to Measure Students’ Self-Efficacy in Systems Thinking
- Conference
- Location
-
Virtual Conference
- Publication Date
-
July 26, 2021
- Start Date
-
July 26, 2021
- End Date
-
July 19, 2022
- Conference Session
- Tagged Division
-
Systems Engineering
- Page Count
-
12
- DOI
-
10.18260/1-2--36766
- Permanent URL
-
https://peer.asee.org/36766
- Download Count
-
350
Paper Authors
Marsha Lovett Carnegie Mellon University
Dr. Marsha Lovett is Associate Vice Provost of Teaching Innovation, Director of the Eberly Center for Teaching Excellence and Educational Innovation, and Teaching Professor of Psychology -- all at Carnegie Mellon University. She applies theoretical and empirical principles from learning science research to improve teaching and learning. She has published more than fifty articles in this area, co-authored the book How Learning Works: 7 Research-Based Principles for Smart Teaching, and developed several innovative, educational technologies, including StatTutor and the Learning Dashboard.
Mark David Bedillion
Carnegie Mellon University
orcid.org/0000-0001-5065-4131
Dr. Bedillion received the BS degree in 1998, the MS degree in 2001, and the PhD degree in 2005, all from the mechanical engineering department of Carnegie Mellon University. After a seven year career in the hard disk drive industry, Dr. Bedillion was on the faculty of the South Dakota School of Mines and Technology for over 5 years before joining Carnegie Mellon as a Teaching Faculty in 2016. Dr. Bedillion's research interests include distributed manipulation, control applications in data storage, control applications in manufacturing, and STEM education.
Cassandra M. Birrenkott
South Dakota School of Mines and Technology
orcid.org/0000-0001-6137-0820
Dr. Cassandra (Degen) Birrenkott received her B.S. degree in Metallurgical Engineering from the South Dakota School of Mines and Technology in 2007. She received her Ph.D. in Materials Science and Engineering in 2012 from the University of Illinois at Urbana-Champaign, studying mechanochemical reactions of a spiropyran mechanophore in polymeric materials under shear loading. She is currently an Assistant Professor in the Mechanical Engineering department at the South Dakota School of Mines and Technology where her research interests include novel manufacturing and characterization techniques of polymer and composite structures and the incorporation of multifunctionality by inducing desired responses to mechanical loading.
Karim Heinz Muci-Kuchler South Dakota School of Mines and Technology
Dr. Karim Muci-Küchler is a Professor of Mechanical Engineering and Director of the Experimental and Computational Mechanics Laboratory at the South Dakota School of Mines and Technology (SDSMT). Before joining SDSMT, he was an Associate Professor of Mechanical Engineering at the University of Detroit Mercy. He received his Ph.D. in Engineering Mechanics from Iowa State University in 1992. His main interest areas include Computational Mechanics, Solid Mechanics, and Product Design and Development. He has taught several different courses at the undergraduate and graduate level, has over 50 publications, is co-author of one book, and has done consulting for industry in Mexico and the US. He can be reached at Karim.Muci@sdsmt.edu.
Laura Ochs Pottmeyer Carnegie Mellon University
Laura Pottmeyer is a Data Science Research Associate at Carnegie Mellon University's Eberly Center for Teaching Excellence and Educational Innovation. She consults with faculty members and graduate students on implementing educational research projects. She assists with study design, data collection, and data analysis. Laura’s training includes a Ph.D. in Science Education and M.Ed. in Educational Psychology from the University of Virginia, where she studied the impact of engineering design integrated science on student learning.
Abstract
Given the growing interest in teaching systems thinking and systems engineering in traditional engineering majors like mechanical engineering, a critical goal involves creating valid and reliable methods for assessing students’ development in systems thinking and systems engineering. To measure students’ development, it is important to distinguish changes in performance from changes in attitudes and beliefs. This paper discusses ongoing refinements to the first half of an existing Systems Thinking Skills Survey (STSS; Muci-Kuchler, 2017), which is designed to measure students’ self-efficacy (i.e., belief in one’s capacity, cf. Bandura, 1997; 2006) to apply their knowledge and skills to systems engineering problems. Across multiple semesters, pre- and post STSS datasets from two universities were analyzed to gauge the validity and internal consistency of 44 self-efficacy items. Results guided changes to the items, while also enabling a reduction in the instrument’s length. With a noticeable reduction to only 30 items, the new version showed higher internal consistency (as measured by Cronbach’s alpha), while maintaining validity. In addition, specific subscales were identified to measure students’ self-efficacy for particular systems engineering topic/skill areas, e.g., concept generation, setting target specifications, identifying customer needs, and systems architecture. Finally, results from administering the new version as a pre/post-test showed that students had significant gains in self-efficacy after taking a mechanical engineering course that incorporated systems thinking/systems engineering material. These significant gains were present in students’ overall self-efficacy ratings, as well as their ratings for each self-efficacy subscale. Overall, these results highlight the value of iterating on assessment practices as part of curriculum development.
Lovett, M., & Bedillion, M. D., & Birrenkott, C. M., & Muci-Kuchler, K. H., & Pottmeyer, L. O. (2021, July), Building and Revising an Assessment to Measure Students’ Self-Efficacy in Systems Thinking Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36766
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015