July 26, 2021
July 26, 2021
July 19, 2022
Given the growing interest in teaching systems thinking and systems engineering in traditional engineering majors like mechanical engineering, a critical goal involves creating valid and reliable methods for assessing students’ development in systems thinking and systems engineering. To measure students’ development, it is important to distinguish changes in performance from changes in attitudes and beliefs. This paper discusses ongoing refinements to the first half of an existing Systems Thinking Skills Survey (STSS; Muci-Kuchler, 2017), which is designed to measure students’ self-efficacy (i.e., belief in one’s capacity, cf. Bandura, 1997; 2006) to apply their knowledge and skills to systems engineering problems. Across multiple semesters, pre- and post STSS datasets from two universities were analyzed to gauge the validity and internal consistency of 44 self-efficacy items. Results guided changes to the items, while also enabling a reduction in the instrument’s length. With a noticeable reduction to only 30 items, the new version showed higher internal consistency (as measured by Cronbach’s alpha), while maintaining validity. In addition, specific subscales were identified to measure students’ self-efficacy for particular systems engineering topic/skill areas, e.g., concept generation, setting target specifications, identifying customer needs, and systems architecture. Finally, results from administering the new version as a pre/post-test showed that students had significant gains in self-efficacy after taking a mechanical engineering course that incorporated systems thinking/systems engineering material. These significant gains were present in students’ overall self-efficacy ratings, as well as their ratings for each self-efficacy subscale. Overall, these results highlight the value of iterating on assessment practices as part of curriculum development.
Lovett, M., & Bedillion, M. D., & Birrenkott, C. M., & Muci-Kuchler, K. H., & Pottmeyer, L. O. (2021, July), Building and Revising an Assessment to Measure Students’ Self-Efficacy in Systems Thinking Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36766
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015