Paper ID #43539Validating Assessment Instruments for Use in Engineering Education: A Primerfor Conducting and Interpreting Factor AnalysisDr. Susan L. Amato-Henderson, Michigan Technological University Susan Amato-Henderson is an Associate Professor Emeritus of Psychology in the Department of Cognitive and Learning Sciences at Michigan Technological University. She received her Ph.D. in Experimental Psychology from the University of North Dakota. Her research interests broadly include STEM education, and focus on individual differences in terms of motivation, self-regulated learning, self-efficacy, grit, resilience, and
between team dynamics. The findings in this study also have limitations at the team forming stage. While UDO scoreswere used a criteria in different ways, it wasn’t the only criteria for team forming. Traditionalcriteria used in the course were given priority and UDO was used as a last criteria in formingteams. This could have significant implications to the interpretation of findings. A trulyexperimental setup was not feasible for a course offered at such a large scale. Furthermore, teameffectiveness can also vary with different factors in the course such as different instructors, priorexperience of students with teamwork, self-efficacy in course content, personality difference, andteam player disposition. These confounding factors need to be
, vol. 37, no. 4, pp. 391–400, Jun. 2014, doi: 10.1016/j.adolescence.2014.03.003.[26] Trochim, Donnelly, and A. Kanika, Research Methods: The Essential Knowledge Base, 2nd edition. Boston, MA: Cengage Learning, 2015.[27] S. Zappe, S. Cutler, S. Spiegel, J. Blacklock, and D. Jordan, “Development of Self- Efficacy and Mindset Scales for Advanced Manufacturing and Data Sciences,” In ASEE annual conference exposition, 2022.[28] E. Muraki, “INFORMATION FUNCTIONS OF THE GENERALIZED PARTIAL CREDIT MODEL,” ETS Research Report Series, vol. 1993, no. 1, Jun. 1993, doi: 10.1002/j.2333-8504.1993.tb01538.x.[29] G. N. Masters, “A rasch model for partial credit scoring,” Psychometrika, vol. 47, no. 2, pp. 149–174, Jun. 1982, doi: 10.1007
allowsstudents to get involved [2] in meaningful ways in their campus community (and beyond).Understanding the navigation and impact of undergraduate research experiences for STEMstudents is limited and primarily derived from Research Experiences for Undergraduate (REU)programs [3]. There is a recognized need for research that more broadly explores undergraduateresearch experiences [3] considering their potential impact on individual students, institutions,and the engineering disciplines.For individual students, there are potential benefits in the form of professional competencydevelopment, persistence, self-efficacy, and GPA [3], [4], [5], [6], [7], [8], [9], [10], [11]. Forinstitutions (especially R1 institutions like the one where this study was
retention, as well as methods to improve NTS outcomes.Several studies focus on the pedagogy of NTS [26], [27].Factors affecting NTS persistence, completion, attrition, or retentionExisting studies have identified factors that affect NTS persistence, completion, attrition, orretention, that may be categorized as social factors, academic performance factors, demographicfactors, inter-role conflicts, and academic and social integration.Social factors refer to emotional and behavioral influences from another person including but notlimited to family members, classmates, and friends. Social factors affect NTS retention byinfluencing NTS self-efficacy and motivation in completing college [11]. As support orencouragement increases, student’s perception of
]. In this work, weemploy an identity framework consisting of four sub pillars: competence, performance, interest,and recognition taken from the work of Carlone and Johnson [6], Hazari [5], and Godwin [4].The first sub pillar, competence, is the ability [6] or belief in one’s ability [5] to understand math,science, or engineering content. In some instances, competence could be combined withperformance. In these cases it could be measured by looking at self-efficacy beliefs and taskattainment [4]. Performance is the ability or belief in the ability to do things related to math,science, and engineering. Students could show performance through having the skills to performscientific practices and task attainment, such as, getting good grades [6
previous EFA,indicating that the Framing Agency Survey provides data that are valid for uses like instructionalrefinement and further studies into the role that framing agency plays in the professionalformation of engineers. However, such studies will require a larger dataset, as well as analysisexamining the structure of the survey that includes measures of relevant constructs, such asengineering identity, engineering self-efficacy, and persistence intentions. Our ongoing researchaims to develop full structural models that include demographic covariates to permitinvestigation of varied impacts on privileged and minoritized students.AcknowledgmentsThis material is based upon work supported by the National Science Foundation under Grant No.1751369
, “The development and validation of a new multidimensional test anxiety scale (mtas),” European Journal of Psychological Assessment, 2020.[23] R. Driscoll, “Westside test anxiety scale validation.” Online submission, 2007.[24] J. Taylor and F. P. Deane, “Development of a short form of the test anxiety inventory (tai),” The Journal of General Psychology, vol. 129, no. 2, pp. 127–136, 2002.[25] C. D. Spielberger, “Manual for the state-trait anxietry, inventory,” Consulting Psychologist, 1970.[26] E. S. Cerino, “Relationships between academic motivation, self-efficacy, and academic procrastination.” Psi Chi Journal of Psychological Research, vol. 19, no. 4, 2014.[27] P. Steel, “The nature of procrastination: a meta-analytic and
disciplines, CBE—Life Sciences Education, 19(3), ar46, 2020.[18] C. S. Dweck, “Mindset: The new psychology of success,” Random House, 2006.[19] R. W. Hass, J. Katz-Buonincontro, and R. Reiter-Palmon, “Disentangling creative mindsets from creative self- efficacy and creative identity: Do people hold fixed and growth theories of creativity?” Psychology of Aesthetics, Creativity, and the Arts, 10(4), 436, 2016.[20] K. Schumann, J. Zaki, and C.S. Dweck, “Addressing the empathy deficit: beliefs about the malleability of empathy predict effortful responses when empathy is challenging,” Journal of Personality and social psychology, 107(3), 475, 2014.[21] Q. Cutts, E. Cutts, S. Draper, P. O'Donnell, and P. Saffrey
. Ahmad, "Learning styles and critical thinking skills of engineering students," in 2017 IEEE 9th International Conference on Engineering Education (ICEED), 2017.[5] E. E. Miskioğlu and D. W. Wood, "That's not my style: Understanding the correlation of learning style preferences, self-efficacy, and student performance in an introductory chemical engineering course," in 2014 IEEE Frontiers in Education Conference (FIE) Proceedings, 2014.[6] R. J. Kapadia, "Teaching and learning styles in engineering education," in 2008 38th Annual Frontiers in Education Conference, 2008.[7] D. Rohrer and H. Pashler, "Learning Styles: Where's the Evidence?.," Online Submission, vol. 46, p. 634–635, 2012.[8] J
resources must be invested while doing the task)[12].Viewing doctoral attrition through the lens of EVT allows us to map graduate students’ concernswith each one of the STVs. For instance, the connection between a sense of belonging and self-efficacy can be observed in the attainment STV. Additionally, doctoral students need to bewilling to persist to engage in tasks and spend time and effort, two traits that map with theintrinsic and the cost STV [6]. Finally, the role of peers and faculty relationships as part of thestudent’s daily life (i.e. socialization) during their doctoral degree process, can be linked withintrinsic, attainment, and utility [1].Expectancy Value Theory and Engineering EducationIn a review of the application of EVT within
part of wellbeing andexplored two dimensions of mental health: psychological illness and emotional wellness. Studiesfocusing on psychological illness focused on mental health conditions such as depression, anxietyand suicidal ideation[26], [27], [28]. Whereas those that emphasized emotional wellness usedconstructs such as sense of belonging, social self-efficacy, social support, and flourishing. Manyshortlisted studies identified under the mental health theme considered both dimensions of mentalhealth to get a complete picture of psychological well-being. For instance, Bork and Mondisa [26], and Bork et al [29] considered both dimensions of mental health encapsulating elements of bothpsychological illness and emotional wellness. Table IV
professional(i.e. progressing careers, enhancing skills, completing work tasks). Much of the engineeringstudent goal literature is at the undergraduate level. This body of work tends to focus onundergraduate engineering students’ career goals. Researchers have studied the relationshipsbetween students’ future career goals and motivation to learn and persist through [24]–[27],continued interest in their engineering major [28], career goal commitment after graduation [29],and engineering self-efficacy [30], [31]. In their exploration of what factors motivateundergraduate students to enroll in engineering graduate programs, Borrego et al., [32] andKyoung Ro et al., [33] found that career goals can predict enrollment. In one of the onlylongitudinal
in higher education: Psychological barriers to success and interventions to reduce social-class inequality,” Journal of Social Issues, vol. 73, no. 1, pp. 23–41, 2017.[8] A. Kezar, L. Hypolite, and J. A. Kitchen, “Career Self-Efficacy: A Mixed-Methods Study of an Underexplored Research Area for First-Generation, Low-Income, and Underrepresented College Students in a Comprehensive College Transition Program,” American Behavioral Scientist, vol. 64, no. 3, pp. 298–324, Mar. 2020, doi: 10.1177/0002764219869409.[9] J. Roksa and P. Kinsley, “The Role of Family Support in Facilitating Academic Success of Low-Income Students,” Res High Educ, vol. 60, no. 4, pp. 415–436, Jun. 2019, doi: 10.1007/s11162-018-9517-z.[10] C. E
skills and motivation might limit students' outcomes in capstone design projects. The lack of the supervisors' practical communication skills ultimately affected the students' motivation to develop the projects and apply the required project management techniques. In other words, if students sense that their opinions are respected, they will feel more empowered and motivated. Similar results are observed by the National Institute of Construction Management and Research and Walden University [13,14]. Along with motivation, social-cognitive theory suggests that effective communication affects students' deep thinking and motivation [15]. Hence, learners with strong self-efficacy are more likely to engage in activities that
; Zyngier, D. (2012). How Motivation Influences Student Engagement: A Qualitative Case Study. Journal of Education and Learning, 1(2), 252–267.Schuman, H., Walsh, E., Olson, C., & Etheridge, B. (1985). Effort and reward: The assumption that college grades are affected by quantity of study. Social Forces, 63(4), 945–966. https://doi.org/10.2307/2578600Schunk, B. J. Z., Dale H. (2007). Motivation: An Essential Dimension of Self-Regulated Learning. In Motivation and Self-Regulated Learning. Routledge.Shu, K. (2022). Teachers’ Commitment and Self-Efficacy as Predictors of Work Engagement and Well-Being. Frontiers in Psychology, 13. https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg
-school outreachprogram in engineering design for middle school students (ages 11-14), and how instructorsviewed the successes, challenges, and tensions of their students’ laboratory experiences. A challenge associated with NGSS and ASEE implementation is the meaningful integrationof science and engineering knowledge and skills in precollege teaching and learning. Researchhas identified issues that science teachers encounter with integrated STEM instruction, includinglack of relevant content knowledge, lack of administrative support, and weak self-efficacy inengineering pedagogy [4,10,11]. Research in STEM integration education has suggested thatinnovative instructional models and curricular resources are needed to demonstrate how scienceand
might be better retained if their social-cognitive disposition with respect to attritionwas supported by social-cognitive skills and strategies adapted from SOB and SRL models.However, of the many interventions currently being employed, most focus on improving thematch of the student to the institution and major, and specific core competencies. While this doesrecognize a cognitive element of student success (i.e., how a student thinks about their majorimpacts their decision to remain in it), it does not fully support SOB. Although they are aminority, there interventions at the college level for engineering majors that specifically addressSOB [18] or self-efficacy [14]. Self-efficacy is an important but small element of these socio-cognitive
. Self-efficacy Q14 At this point, I am confident that I can complete my program of study (e.g., MS, PhD). Advisor relationship Q15 At this point in my program, I consider my advisor a mentor. Support network Q16 In the last four months, I felt well-supported by people in my network outside the university. Passion Q17 At this point, I consider myself passionate about my research. Quarterly Motivation Q18 In the last four months, I was motivated to do my research (December, because of external factors, such as external
critique of flexible online learning and its “anytime anyplace” claims. British Journal of Educational Technology, 50(3), 1005-1018.[10] Kuo, Y. C., Walker, A. E., Belland, B. R., Schroder, K. E., & Kuo, Y. T. (2014). A case study of integrating Interwise: Interaction, internet self-efficacy, and satisfaction in synchronous online learning environments. International Review of Research in Open and Distributed Learning, 15(1), 161-181.[11] Humiston, J. P., Marshall, S. M., Hacker, N. L., & Cantu, L. M. (2020). Intentionally creating an inclusive and welcoming climate in the online learning classroom. In Handbook of Research on Creating Meaningful Experiences in Online Courses (pp. 173-186). IGI Global.[12] Martin, F
single goal. Engagement in unstructured-specific activities is characterized by high self-motivation and high self-efficacy. For theseactivities, sometimes people engage to learn a task, while in others, engagement is driven by afixed-mindset: their mind was set on independent task completion. For example, one studentreflected on how he approaches working on his car in high school in this manner, saying, “I'd belike, all right, I'm going to try and identify the problem. I'm going to try and look at a video, seehow to do it.” Here we see that the activity was focused, goal-oriented, and driven by one’s self.A noted difference between unstructured-specific and structured-specific, is in the type ofactivities; with unstructured activities, the
Machine (SVM) and Random Forest (RF)approaches, and the unsupervised Latent Dirichlet Allocation (LDA) method. Free responses toopen-ended questions from student surveys in multiple courses at University of IllinoisUrbana-Champaign were previously collected by engineering education accessibility researchers.The data (N=129 with seven free response questions per student) were previously analyzed toassess the effectiveness, satisfaction, and quality of adding accessible digital notes to multipleengineering courses and the students’ perceived belongingness, and self-efficacy. Manual codingsfor the seven open-ended questions were generated for qualitative tasks of sentiment analysis,topic modeling, and summarization and were used in this study as a
classattendance) [26]. When looking to combine a strengths-based language with individualreflection, a conversation-based strengths-based intervention showed students had high regardsfor their strengths and had higher levels of optimism and self-efficacy throughout the course[27]. Collectively, these programs emphasize the role of strengths-based languages in fostering athriving atmosphere for undergraduate students.Research into CliftonStrengths is also increasing in popularity. Specifically, researchers approveof using a cohesive assessment tool to provide students with a consistent language to discusstheir strengths and other traits. CliftonStrengths has been proven reliable through its evidence ofconsistency over time, where individuals tend to
self-efficacy orreduce opportunities to make the course more challenging if they had higher personal learninggoals. Researchers speculated that some students may not make the connection between failureand iteration as an effective learning strategy.CollaborationStudents are required to work in groups in both courses and at times considered a “divide andconquer” approach to teamwork, in which each team member undertook a separate task, as themost efficient strategy. Investigators identified a need for a cooperative, integrated teamapproach to learning that would help students make sense of course material together with theirpeers by benefitting from one another’s knowledge and diverse experience.Creative ConfidenceAccording to David and Tom
-Pohl, “Self-efficacy and subjective task values in relation to choice, effort, persistence, and continuation in engineering: an Expectancy-value theory perspective,” European Journal of Engineering Education, vol. 45, no. 1, pp. 151–163, Jan. 2020, doi: 10.1080/03043797.2019.1659231. [6] J. P. Monat, T. F. Gannon, and M. Amissah, “The Case for Systems Thinking in Undergraduate Engineering Education,” International Journal of Engineering Pedagogy (iJEP), vol. 12, no. 3, pp. 50–88, May 2022, doi: 10.3991/ijep.v12i3.25035. [7] C. L. Dym, A. M. Agogino, O. Eris, D. D. Frey, and L. J. Leifer, “Engineering Design Thinking, Teaching, and Learning,” Journal of Engineering Education, vol. 94, no. 1, pp. 103–120, Jan
students (Part II). ASEE Annual Conference and Exposition, Conference Proceedings.Shell, D. F., & Husman, J. (2008). Control, motivation, affect, and strategic self-regulation in the college classroom: A multidimensional phenomenon. Journal of Educational Psychology, 100(2), 443–459. https://doi.org/10.1037/0022-0663.100.2.443Schell, W. J., & Hughes, B. E., & Tallman, B., & Annand, E., & Beigel, R. M., & Kwapisz, M. B. (2019, June), Exploring the Relationship Between Students’ Engineering Identity and Leadership Self-Efficacy Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--32817Schnitker, S. A. (2012). An examination of patience and
of the Research Self-Efficacy Scale,” Journal of Career Assessment, vol. 4, no. 1, pp. 59-75, July 1996, doi: 10.1177/106907279600400104.Appendix A: Survey QuestionsTable 1. Example questions for each scale included in the survey. Number Response Scale of Example Question Type Questions Dyadic 17 Rate to what extent the following statements are true Likert Psychological for you: I feel like my research advisor makes 1-9 Safety
attitudes and higher self-efficacy for innovation. Theydefined a mindful attitude as the willingness to engage in situations that may seem uncertain andnovel in one’s daily life. Respondents to the survey with mindful attitudes were more likely tohave been involved with leadership and entrepreneurial clubs, courses, and initiatives. Theresults suggest that students who exercise mindfulness in their routine are set up for successfulentrepreneurial and design-related endeavors.Through a meta-analysis of literature in psychology, Lebuda et al. (2016) demonstrated that thereis a statistically significant link between mindfulness and creativity. They found certain abilitiesthat promote creativity to be fostered through mindfulness training or trait
also enrollstudents who are different from those beginning their studies at four-year institutions in manyways [40]. This does not entirely discount the value of single-institution studies, however. Astudy [41] conducted at Louisiana Tech University explored the impact of backgroundknowledge on the success of freshman engineering students, as measured by grades andgraduation rates. The author acknowledges the limitation of analyzing data from a singleuniversity, and also notes that the focused examination of a specific context provided a nuancedunderstanding that might be obscured by institutional variation.Collectively, these studies underscore the potential benefit of a nuanced approach that considersinstitutional differences in the context
, participants seem to have strikingly different ideas of what it means to persist in ML/AI, and future work should continue to capture this. In quantitative studies, ML/AI should be clearly defined, with examples of roles.2. Questions about social belonging should assess the ability for students to develop meaningful close social connection, such as mentorship, rather than general social connection.3. Persistence research in tech-heavy engineering fields should assess programming self-efficacy, which women report lower levels of [52]. The increasing demand for these skills in engineering may exacerbate the technical confidence gap, further perpetuating the gender gap in these fields.We report on some initial ways persistence in ML/AI