science on student learning. American c Society for Engineering Education, 2021 Building and Revising an Assessment to Measure Students’ Self-Efficacy in Systems Thinking Mark D. Bedillion1*, Cassandra M. Birrenkott2, Marsha C. Lovett3, Karim H. Muci-Kuchler2, and Laura O. Pottmeyer3 1 Mechanical Engineering Department, Carnegie Mellon University 2 Mechanical Engineering Department, South Dakota School of Mines and Technology3 Eberly Center for Teaching Excellence and Educational Technology, Carnegie Mellon University
feedback they received on their cases, and their generalexperiences with the course SYS 2001. Three major categories of surveys were used to assessstudents’ perceptions (timeline of the use of the technologies included in Figure 1): Page 24.547.7 Self-efficacy surveys were modified based on an instrument measuring engineeringdesign self-efficacy by Carberry and Lee24. The surveys were believed to identify students’self-concepts to engineering design tasks24. Students were asked to rate on a scale of 0-100their confidence, motivation, success, and anxiety in completing each of the 10 tasks whichrepresent a systems approach. Grading surveys
for engineers who understand the fundamentals ofsystems engineering. This paper has presented an effort to improve mechanical engineeringstudents’ systems engineering skills through the redesign of a sophomore design course. Studentswere exposed to primers and case studies that covered essential steps in the systems engineeringprocess and completed a semester-long project that required integration of various subsystems.The effectiveness of the intervention was assessed through a newly designed systems thinkingskills survey and through a course satisfaction survey. Students showed a statistically significantimprovement in self-efficacy for all measured skills, but showed a statistically significant gainover the control group only for the skill
, with technical contentembedded in an online learning module and class time used to perform a group designactivity.An effective means of measuring students’ systems thinking / systems engineering skills isneeded to assess the effectiveness of the intervention. There have been several approaches in theliterature, ranging from comprehensive written / practical exams 9 to computerized tests thatmeasure specific systems engineering skills 10 . This paper uses a survey instrument called theSystems Thinking Skills Survey (STSS) 6 which includes both self-efficacy questions andtechnical questions to assess students systems engineering skills.This paper describes results of a flipped-classroom learning experience on systems engineeringgeared toward
interested in the freshman engi- neering experience and student self-efficacy related to capstone courses. Bauer’s educational background centers around human factors and ergonomics, and she is particularly interested in issues that concern the safety and comfort of middle school students. Her research has also included topics such as design for the seeing impaired, backpack safety of college students, safety of pedestrians, and ergonomics of industrial tools.Dr. Jessica L. Heier Stamm, Kansas State UniversityDr. Lesley Strawderman, Mississippi State University Page 25.98.1 c American
(.58) -5.28 33 <0.001* 2.25 (.62) 2.96 (.67) -3.34 15 0.001*Generation Systems 1.55 (.91) 2.90 (.51) -8.16 33 <0.001* 2.07 (.64) 3.09 (.68) -3.08 15 0.002* Architecturea repeated measures paired t-testb non-parametric Wilcoxon-sign rank test*significant at the 0.05 levelResults of paired t-test indicated that the overall increase in self-efficacy scores over the course ofthe Fall 2019 semester (pre M=1.61, SD=0.82; post M=2.80, SD=0.50) was statistically significantt(33)=-8.02, p<0.001. Additionally, the increase in self-efficacy scores in Fall 2020 (pre M=2.13,SD=0.64; post M=3.03, SD=0.60) was also statistically significant Z=-3.41, p<0.001.Overall, these self-efficacy results demonstrate
Understand System verification and validation UnderstandTo gauge the benefits of an intervention aimed at teaching ST and SE concepts in a mechanicalengineering undergraduate course, it is desirable to have an assessment instrument that is not tiedto the course and that can supplement data collected from evaluations based on course activitiessuch as homework assignments. One instrument that is available for that purpose is the SystemsThinking Skills Survey (STSS) [26]. The STSS has two main sections. In the first sectionstudents report their perceived self-efficacy in a number of ST/SE knowledge, skills, and abilities(KSAs). In the second, students demonstrate their proficiency in selected ST/SE concepts
measurable constructs of affect.Exploring affect in the STEM education literature11, we identified possible constructs. For thepurpose of this study, the constructs of affect are: interest, motivation, attitude, and self-efficacy.Interest reflects the level of student engagement or reengagement with the STEM contentpresented in the Structures course. Motivation triggers or maintains a students’ goal-orientatedbehavior to further engage with STEM content. Students’ attitude, positive or negative, providesan evaluation about the content presented in Structures. Students’ self-efficacy for STEM contentreflects their expectancy about his or her ability to do well on a specific STEM-related task.Utilizing the framework defining STEM-literacy and student
report theirperceived level of self-efficacy in different topics related to ST and SE. This section representsan indirect measure of students’ abilities because students are reporting their perceptions of theirabilities. By contrast, in the second section, students need to apply knowledge in ST and SE toanswer several questions (i.e., direct measure of students’ ability to apply ST and SE conceptsand skills). Each question provides a product or system familiar to most engineering students forcontext.The first section of the STSS includes 44 items asking students “How well do you think that youcan apply the topics mentioned below to an engineering project?” Student responses arecollected via a 5-point Likert scale, ranging from 1=Not at all to 5
. M. Camacho, R. A. Layton, R. A. Long, S. M. Lord, and M. H. Wasburn, “Race, Gender, and Measures of Success in Engineering Education,” Journal of Engineering Education, vol. 100, no. 2, pp. 225–252, 2011.[15] D. A. Weiser and H. R. Riggio, “Family background and academic achievement: does self-efficacy mediate outcomes?” Social Psychology of Education, vol. 13, no. 3, pp. 367– 383, 2010.[16] R. M. Jaradat and C. B. Keating, “Systems thinking capacity: implications and challenges for complex system governance development,” International Journal of System of Systems Engineering, vol. 7, no. 1/2/3, p. 75, 2016.[17] K. M. Castelle and R. M. Jaradat, “Development of an Instrument to Assess Capacity for Systems Thinking,” Procedia
, and demographic characteristics, in conjunction with ST skills and PP, might influence the academic performance of engineering students. All the mentioned measures and scales, including the Big five personality instrument, motivation, self-efficacy, time management, and demographic characteristics, as well as the current study variables, which are part of the comprehensive theoretical model of a bigger study, will be used for future data collection and analysis. Moreover, it would be beneficial to compare the predictive validity of ST skills and PP to other well-known predictors of academic performance. Future studies would also include the type of training needed to enhance students' level of systems skills based on
promote effective knowledge construction (i.e., mentalprocess in which a learner takes many separate pieces of information and usesthem to build an overall understanding or interpretation), skill mastery (i.e.,learning a skill thoroughly), learning transfer, and motivational aspects associatedwith effective learning, such as self-efficacy (i.e., belief that one is capable ofexecuting certain tasks and achieving certain goals) and mastery orientation (i.e.,belief that one is capable of accomplishing challenging tasks) [10].With a digital twin, students can get immediate feedback on system behavior (inresponse to what-if injects), identify issues, and develop a mental model for howthe physical twin is likely to perform in the real-world under
achievement measures, the authors found weak relations between the amount ofhomework assigned and student achievement, and positive relations between the amount ofhomework students completed and achievement, especially at upper grades (6-12). Hoover-Dempsey et al.[13] reviewed research on parental involvement in student homework, and foundthat parents involve themselves in student homework because they believe that they should beinvolved, believe that their involvement will make a positive difference, and perceive that theirchildren or children's teachers want their involvement. Unfortunately, the first-generation collegestudents fail to get that at TAMIU. Leone and Richards[17] found that in instructive practice,homework plays a critical, long-term
senior design or capstone projects,” ASEE Annual. Conf. Expo. Conf. Proc., vol. 2018-June, no. July, 2018.[20] S. Gillespie and A. Maccalman, “A Case Study in Developing an Integrated Data and Model Management System for the Development of a Complex Engineered System,” in 2018 IEEE Technology and Engineering Management Conference, 2018.[21] K. Laitinen and M. Valo, “Meanings of communication technology in virtual team meetings: Framing technology-related interaction,” Int. J. Hum. Comput. Stud., vol. 111, pp. 12–22, 2018.[22] R. Khan, C. Whitcomb, and C. White, “Self-efficacy analysis of student learning in systems engineering,” ASME Int. Mech. Eng. Congr. Expo. Proc., vol. 5, 2016.[23] E
procedure, Design groups’ unique visions andinterests are important topics which are addressed and covered during the aircraft design teachingprocess. These experiences will significantly impact on student development, particularly onlearning, self-efficacy, diversity, and the ability to innovate. The implementation of systems engineering requires a flawless interface between teammembers working toward a common system thinking to correctly execute systems engineeringprocess. Although there is a general agreement regarding the principles and objectives of systemsengineering, its actual implementation will vary from on discipline to the next. The processapproach and steps used will depend on the backgrounds and experiences of the
, and 82 teachers completed a questionnaire concerning amount of homeworkassigned by teachers, portion of assignments completed by students, and attitudes abouthomework. Upon collection of student achievement measures, the authors found weak relationsbetween the amount of homework assigned and student achievement, and positive relationsbetween the amount of homework students completed and achievement, especially at uppergrades (6-12). Hoover-Dempsey et al.[16] reviewed research on parental involvement in studenthomework, and found that parents involve themselves in student homework because they believethat they should be involved, believe that their involvement will make a positive difference, andperceive that their children or children's
questionnaire concerning amount of homework assigned by teachers, portion ofassignments completed by students, and attitudes about homework. Upon collection of studentachievement measures, the authors found weak relations between the amount of homeworkassigned and student achievement, and positive relations between the amount of homeworkstudents completed and achievement, especially at upper grades (6-12). Bempechat3 obtainedresearch on parental involvement in student homework, and found that parents involvethemselves in student homework because they believe that they should be involved, believe thattheir involvement will make a positive difference, and perceive that their children or children'steachers want their involvement. Unfortunately, the first