Tampa, Florida
June 15, 2019
June 15, 2019
June 19, 2019
Design in Engineering Education Division: Design Mental Frameworks
Design in Engineering Education
Diversity
10
10.18260/1-2--31957
https://strategy.asee.org/31957
511
Dr. Buckley is an Associate Professor of Mechanical Engineering at University of Delaware. She received her Bachelor’s of Engineering (2001) in Mechanical Engineering from the University of Delaware, and her MS (2004) and PhD (2006) in Mechanical Engineering from the University of California, Berkeley, where she worked on computational and experimental methods in spinal biomechanics. Since 2006, her research efforts have focused on the development and mechanical evaluation of medical and rehabilitation devices, particularly orthopaedic, neurosurgical, and pediatric devices. She teaches courses in design, biomechanics, and mechanics at University of Delaware and is heavily involved in K12 engineering education efforts at the local, state, and national levels.
Dr. Grajeda's research interests lie in applied measurement work and policy analyses in education and public health areas. Her measurement work has involved developing and analyzing observational rubrics and surveys in both K12 and higher education settings in various content areas.
Amy Trauth, Ph.D., is the Senior Associate Director of Science Education at the University of Delaware's Professional Development Center for Educators. In her role, Amy works collaboratively with K-12 science and engineering teachers to develop and implement standards-based curricula and assessments. She also provides mentoring and coaching and co-teaching support to K-12 teachers across the entire trajectory of the profession. Her research focuses on teacher education, classroom assessment, and P-16 environmental and engineering education.
Dustyn Roberts received her B.S. in Mechanical and Biomedical Engineering from Carnegie Mellon University (2003), her M.S. in Biomechanics & Movement Science (2004) from the University of Delaware, and her Ph.D. in Mechanical Engineering (2014) from New York University. She is passionate about translational research and engineering education.
Team-based projects are widely used in engineering courses [1], particularly product or process design courses in disciplines like mechanical, electrical, and civil engineering. While the intention of team-based design projects is to provide all students with a range of technical and non-technical mastery experiences [1,2], students enter into these experiences with differences - whether real or perceived – in relevant technical skills that affect division of labor amongst team members. For example, prior work by Chachra and Kilgore [3] has demonstrated a ‘confidence gap’ in open-ended problem solving between men and women specific to work on engineering design-based projects. This disparity in self-confidence leads to a lower likelihood that women will take an active role in technical tasks and instead relegate themselves to administrative or people-oriented tasks on design projects [4,5].
We hypothesize that task choice on team-based projects leads directly to learning experiences for individual students that can either reinforce or undermine self-confidence in particular engineering tasks. These effects may be particularly pronounced for underrepresented students, specifically women in traditionally male-dominated engineering fields, students of color, and first-generation students. To address this hypothesis, we developed a novel framework for synchronously quantifying students’ task-choice and self-confidence for team-based engineering design projects. This framework, which will be the focus of this study, leverages multiple validated instruments from the literature and combines them into two validated, multi-factorial outcomes – one for self-confidence and one for task-choice – that are inherently aligned with each other as well as the necessary learning elements of a typical engineering project. To facilitate visualization of individual students’ outcomes across two multi-factorial measures, we also developed a novel graphical representation of our validated task-choice and self-confidence measures.
The development of this novel framework began with the creation of holistic, validated instruments to quantify self-confidence and task-choice. For self-confidence, we developed an initial instrument by combining the well-established APPLES instrument [3,4,6], which focuses on self-confidence in interpersonal skills, problem solving, and math and science theory, with an established but unvalidated instrument [7] that measures self-confidence in tinkering and engineering applications. The combination of the two surveys allowed us to capture the entire range of typical learning outcomes of design-based projects. Exploratory and confirmatory factor analyses were conducted on data from a large pilot (N=602) of first-year engineering students. The instrument was shortened by eliminating items that did not correlate to a factor (low factor loadings) or correlated with multiple factors (high crossloadings). The refined instrument was administered to another cohort of first semester engineering students (N=632), which yielded better fit (RMSEA=.058, CFI=.857) and high reliability on each factor (Cronbach’s alpha ranging from .77 to .84). Five principal factors that encompass self-confidence or self-efficacy on design-related tasks were measured in the final version of the survey: (1) open-ended problem solving; (2) interpersonal; (3) math and science skills; (4) engineering applications; and (5) tinkering.
A similar multi-factorial instrument was developed to quantify students’ task-choice. A complete set of project tasks was constructed by identifying overlapping categories across two prior instruments [4,6] and cross-referencing these tasks with common elements found in peer evaluations, student deliverables, and grading rubrics for engineering design courses at our institution and others [8]. This yielded eight task categories: (1) problem definition; (2) concept generation; (3) prototype fabrication; (4) design schematics; (5) engineering analyses; (6) design validation; (7) project management; and (8) technical communication. Our initial task-choice survey (analgous in implementation to a prior study [6]) was administered to the same cohort of students as our self-confidence CFA. It asked students post-hoc to estimate how many hours they spent on a particular task for the entirety of the project as well as the total time that their team spent on this task. This item structure led to numerous internal inconsistencies, most notably individual time contributions exceeding total team contribution; and the item was redesigned to ask students for their self-assessed contribution relative to the team contribution for a particular task.
Our framework also includes a novel approach to graphically representing individual students’ multi-factorial outcome for self-confidence and task-choice. Influenced by the Basadaur Profile [9], a graphical representation of personal creative problem-solving strategies, we represented each students’ 5-factor self-confidence score as a radar plot, normalized to maximum achievable values. A similar approach was taken to generate a visualization of students’ 8-factor task-choice outcomes. The axes for self-confidence and task-choice were aligned by theme to facilitate direct comparison between the measures, e.g., task-choice in “project management” aligned graphically with self-confidence in “professional and interpersonal skills”. This graphical representation facilitates comparison between student mindset (self-confidence) and behavior on team-based projects.
The framework introduced in this study includes two novel elements that will be valuable in future investigations of student learning on team-based design projects. First, our measures of self-confidence and task-choice, while built on prior work [3,4,6], are the first to reflect learning outcomes and tasks that encompass the entirety of the engineering design process. In future studies, we will use these measures to gain a deeper understanding of how student behavior (task-choice) is affected by a prior self-confidence as well as demographic considerations such as race and gender. The second novel element of this work is the development of a graphical representation that aligns multi-factorial outcomes for self-confidence and self-efficacy on a per-student basis. We anticipate that this tool will be useful for both instructors and students in identifying individual students’ strengths and opportunities for growth as they engage with their peers on team-based design projects.
Buckley, J., & Grajeda, S. B., & Trauth, A., & Roberts, D. (2019, June), A Framework for Quantifying Student Self-Confidence and Task Choice in Engineering Design-related Activities Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--31957
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2019 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015