A Framework for Quantifying Student Self-Confidence and Task Choice in Engineering Design-related Activities

Jenni Buckley; Sara Bernice Grajeda; Amy Trauth; Dustyn Roberts

Download Paper | Permalink

Conference: 2019 ASEE Annual Conference & Exposition
Location: Tampa, Florida
Publication Date: June 15, 2019
Start Date: June 15, 2019
End Date: June 19, 2019
Conference Session: Design in Engineering Education Division: Design Mental Frameworks
Tagged Division: Design in Engineering Education
Tagged Topic: Diversity
Page Count: 10
DOI: 10.18260/1-2--31957
Permanent URL: https://peer.asee.org/31957
Download Count: 445

Request a correction

Paper Authors

biography

Jenni Buckley University of Delaware

visit author page

Dr. Buckley is an Associate Professor of Mechanical Engineering at University of Delaware. She received her Bachelor’s of Engineering (2001) in Mechanical Engineering from the University of Delaware, and her MS (2004) and PhD (2006) in Mechanical Engineering from the University of California, Berkeley, where she worked on computational and experimental methods in spinal biomechanics. Since 2006, her research efforts have focused on the development and mechanical evaluation of medical and rehabilitation devices, particularly orthopaedic, neurosurgical, and pediatric devices. She teaches courses in design, biomechanics, and mechanics at University of Delaware and is heavily involved in K12 engineering education efforts at the local, state, and national levels.

visit author page

biography

Sara Bernice Grajeda University of Delaware

visit author page

Dr. Grajeda's research interests lie in applied measurement work and policy analyses in education and public health areas. Her measurement work has involved developing and analyzing observational rubrics and surveys in both K12 and higher education settings in various content areas.

visit author page

biography

Amy Trauth University of Delaware orcid.org/0000-0002-5146-592X

visit author page

Amy Trauth, Ph.D., is the Senior Associate Director of Science Education at the University of Delaware's Professional Development Center for Educators. In her role, Amy works collaboratively with K-12 science and engineering teachers to develop and implement standards-based curricula and assessments. She also provides mentoring and coaching and co-teaching support to K-12 teachers across the entire trajectory of the profession. Her research focuses on teacher education, classroom assessment, and P-16 environmental and engineering education.

visit author page

biography

Dustyn Roberts P.E. Temple University

visit author page

Dustyn Roberts received her B.S. in Mechanical and Biomedical Engineering from Carnegie Mellon University (2003), her M.S. in Biomechanics & Movement Science (2004) from the University of Delaware, and her Ph.D. in Mechanical Engineering (2014) from New York University. She is passionate about translational research and engineering education.

visit author page

Download Paper | Permalink

Abstract

Team-based projects are widely used in engineering courses [1], particularly product or process design courses in disciplines like mechanical, electrical, and civil engineering. While the intention of team-based design projects is to provide all students with a range of technical and non-technical mastery experiences [1,2], students enter into these experiences with differences - whether real or perceived – in relevant technical skills that affect division of labor amongst team members. For example, prior work by Chachra and Kilgore [3] has demonstrated a ‘confidence gap’ in open-ended problem solving between men and women specific to work on engineering design-based projects. This disparity in self-confidence leads to a lower likelihood that women will take an active role in technical tasks and instead relegate themselves to administrative or people-oriented tasks on design projects [4,5].

We hypothesize that task choice on team-based projects leads directly to learning experiences for individual students that can either reinforce or undermine self-confidence in particular engineering tasks. These effects may be particularly pronounced for underrepresented students, specifically women in traditionally male-dominated engineering fields, students of color, and first-generation students. To address this hypothesis, we developed a novel framework for synchronously quantifying students’ task-choice and self-confidence for team-based engineering design projects. This framework, which will be the focus of this study, leverages multiple validated instruments from the literature and combines them into two validated, multi-factorial outcomes – one for self-confidence and one for task-choice – that are inherently aligned with each other as well as the necessary learning elements of a typical engineering project. To facilitate visualization of individual students’ outcomes across two multi-factorial measures, we also developed a novel graphical representation of our validated task-choice and self-confidence measures.

The development of this novel framework began with the creation of holistic, validated instruments to quantify self-confidence and task-choice. For self-confidence, we developed an initial instrument by combining the well-established APPLES instrument [3,4,6], which focuses on self-confidence in interpersonal skills, problem solving, and math and science theory, with an established but unvalidated instrument [7] that measures self-confidence in tinkering and engineering applications. The combination of the two surveys allowed us to capture the entire range of typical learning outcomes of design-based projects. Exploratory and confirmatory factor analyses were conducted on data from a large pilot (N=602) of first-year engineering students. The instrument was shortened by eliminating items that did not correlate to a factor (low factor loadings) or correlated with multiple factors (high crossloadings). The refined instrument was administered to another cohort of first semester engineering students (N=632), which yielded better fit (RMSEA=.058, CFI=.857) and high reliability on each factor (Cronbach’s alpha ranging from .77 to .84). Five principal factors that encompass self-confidence or self-efficacy on design-related tasks were measured in the final version of the survey: (1) open-ended problem solving; (2) interpersonal; (3) math and science skills; (4) engineering applications; and (5) tinkering.

A similar multi-factorial instrument was developed to quantify students’ task-choice. A complete set of project tasks was constructed by identifying overlapping categories across two prior instruments [4,6] and cross-referencing these tasks with common elements found in peer evaluations, student deliverables, and grading rubrics for engineering design courses at our institution and others [8]. This yielded eight task categories: (1) problem definition; (2) concept generation; (3) prototype fabrication; (4) design schematics; (5) engineering analyses; (6) design validation; (7) project management; and (8) technical communication. Our initial task-choice survey (analgous in implementation to a prior study [6]) was administered to the same cohort of students as our self-confidence CFA. It asked students post-hoc to estimate how many hours they spent on a particular task for the entirety of the project as well as the total time that their team spent on this task. This item structure led to numerous internal inconsistencies, most notably individual time contributions exceeding total team contribution; and the item was redesigned to ask students for their self-assessed contribution relative to the team contribution for a particular task.

Our framework also includes a novel approach to graphically representing individual students’ multi-factorial outcome for self-confidence and task-choice. Influenced by the Basadaur Profile [9], a graphical representation of personal creative problem-solving strategies, we represented each students’ 5-factor self-confidence score as a radar plot, normalized to maximum achievable values. A similar approach was taken to generate a visualization of students’ 8-factor task-choice outcomes. The axes for self-confidence and task-choice were aligned by theme to facilitate direct comparison between the measures, e.g., task-choice in “project management” aligned graphically with self-confidence in “professional and interpersonal skills”. This graphical representation facilitates comparison between student mindset (self-confidence) and behavior on team-based projects.

The framework introduced in this study includes two novel elements that will be valuable in future investigations of student learning on team-based design projects. First, our measures of self-confidence and task-choice, while built on prior work [3,4,6], are the first to reflect learning outcomes and tasks that encompass the entirety of the engineering design process. In future studies, we will use these measures to gain a deeper understanding of how student behavior (task-choice) is affected by a prior self-confidence as well as demographic considerations such as race and gender. The second novel element of this work is the development of a graphical representation that aligns multi-factorial outcomes for self-confidence and self-efficacy on a per-student basis. We anticipate that this tool will be useful for both instructors and students in identifying individual students’ strengths and opportunities for growth as they engage with their peers on team-based design projects.

Citation
Format

Buckley, J., & Grajeda, S. B., & Trauth, A., & Roberts, D. (2019, June), A Framework for Quantifying Student Self-Confidence and Task Choice in Engineering Design-related Activities Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--31957

TY  - CPAPER
AB  - Team-based projects are widely used in engineering courses [1], particularly product or process design courses in disciplines like mechanical, electrical, and civil engineering. While the intention of team-based design projects is to provide all students with a range of technical and non-technical mastery experiences [1,2], students enter into these experiences with differences - whether real or perceived – in relevant technical skills that affect division of labor amongst team members. For example, prior work by Chachra and Kilgore [3] has demonstrated a ‘confidence gap’ in open-ended problem solving between men and women specific to work on engineering design-based projects. This disparity in self-confidence leads to a lower likelihood that women will take an active role in technical tasks and instead relegate themselves to administrative or people-oriented tasks on design projects [4,5]. 

We hypothesize that task choice on team-based projects leads directly to learning experiences for individual students that can either reinforce or undermine self-confidence in particular engineering tasks. These effects may be particularly pronounced for underrepresented students, specifically women in traditionally male-dominated engineering fields, students of color, and first-generation students. To address this hypothesis, we developed a novel framework for synchronously quantifying students’ task-choice and self-confidence for team-based engineering design projects. This framework, which will be the focus of this study, leverages multiple validated instruments from the literature and combines them into two validated, multi-factorial outcomes – one for self-confidence and one for task-choice – that are inherently aligned with each other as well as the necessary learning elements of a typical engineering project. To facilitate visualization of individual students’ outcomes across two multi-factorial measures, we also developed a novel graphical representation of our validated task-choice and self-confidence measures.

The development of this novel framework began with the creation of holistic, validated instruments to quantify self-confidence and task-choice.  For self-confidence, we developed an initial instrument by combining the well-established APPLES instrument [3,4,6], which focuses on self-confidence in interpersonal skills, problem solving, and math and science theory, with an established but unvalidated instrument [7] that measures self-confidence in tinkering and engineering applications. The combination of the two surveys allowed us to capture the entire range of typical learning outcomes of design-based projects. Exploratory and confirmatory factor analyses were conducted on data from a large pilot (N=602) of first-year engineering students. The instrument was shortened by eliminating items that did not correlate to a factor (low factor loadings) or correlated with multiple factors (high crossloadings). The refined instrument was administered to another cohort of first semester engineering students (N=632), which yielded better fit (RMSEA=.058, CFI=.857) and high reliability on each factor (Cronbach’s alpha ranging from .77 to .84). Five principal factors that encompass self-confidence or self-efficacy on design-related tasks were measured in the final version of the survey: (1) open-ended problem solving; (2) interpersonal; (3) math and science skills; (4) engineering applications; and (5) tinkering.

A similar multi-factorial instrument was developed to quantify students’ task-choice. A complete set of project tasks was constructed by identifying overlapping categories across two prior instruments [4,6] and cross-referencing these tasks with common elements found in peer evaluations, student deliverables, and grading rubrics for engineering design courses at our institution and others [8]. This yielded eight task categories: (1) problem definition; (2) concept generation; (3) prototype fabrication; (4) design schematics; (5) engineering analyses; (6) design validation; (7) project management; and (8) technical communication. Our initial task-choice survey (analgous in implementation to a prior study [6]) was administered to the same cohort of students as our self-confidence CFA. It asked students post-hoc to estimate how many hours they spent on a particular task for the entirety of the project as well as the total time that their team spent on this task. This item structure led to numerous internal inconsistencies, most notably individual time contributions exceeding total team contribution; and the item was redesigned to ask students for their self-assessed contribution relative to the team contribution for a particular task.

Our framework also includes a novel approach to graphically representing individual students’ multi-factorial outcome for self-confidence and task-choice. Influenced by the Basadaur Profile [9], a graphical representation of personal creative problem-solving strategies, we represented each students’ 5-factor self-confidence score as a radar plot, normalized to maximum achievable values. A similar approach was taken to generate a visualization of students’ 8-factor task-choice outcomes. The axes for self-confidence and task-choice were aligned by theme to facilitate direct comparison between the measures, e.g., task-choice in “project management” aligned graphically with self-confidence in “professional and interpersonal skills”. This graphical representation facilitates comparison between student mindset (self-confidence) and behavior on team-based projects.

The framework introduced in this study includes two novel elements that will be valuable in future investigations of student learning on team-based design projects. First, our measures of self-confidence and task-choice, while built on prior work [3,4,6], are the first to reflect learning outcomes and tasks that encompass the entirety of the engineering design process. In future studies, we will use these measures to gain a deeper understanding of how student behavior (task-choice) is affected by a prior self-confidence as well as demographic considerations such as race and gender. The second novel element of this work is the development of a graphical representation that aligns multi-factorial outcomes for self-confidence and self-efficacy on a per-student basis. We anticipate that this tool will be useful for both instructors and students in identifying individual students’ strengths and opportunities for growth as they engage with their peers on team-based design projects.

AU  - Jenni Buckley
AU  - Sara Bernice Grajeda
AU  - Amy Trauth
AU  - Dustyn Roberts P.E.
CY  - Tampa, Florida
DA  - 2019/06/15
PB  - ASEE Conferences
TI  - A Framework for Quantifying Student Self-Confidence and Task Choice in Engineering Design-related Activities
UR  - https://peer.asee.org/31957
DO  - 10.18260/1-2--31957
ER  -