An In-depth Analysis of Open-ended Biomedical Engineering Design Problems and the Role of Metacognition in Their Solutions

Hannah Yssels; Marina Crowder; Ozcan Gulacar; Jennifer H. Choi

Download Paper | Permalink

Conference: 2018 ASEE Annual Conference & Exposition
Location: Salt Lake City, Utah
Publication Date: June 23, 2018
Start Date: June 23, 2018
End Date: July 27, 2018
Conference Session: Design in the BME curriculum
Tagged Division: Biomedical Engineering
Page Count: 12
DOI: 10.18260/1-2--29788
Permanent URL: https://peer.asee.org/29788
Download Count: 690

Request a correction

Paper Authors

biography

Hannah Yssels

visit author page

Hannah Yssels is a fourth year biomedical engineering student at UC Davis, specializing in medical devices. She is currently a research assistant to Jennifer Choi, PhD, investigating problem solving performance and the development of design thinking skills in biomedical engineering. She has also assisted in the Heinrich Lab, researching the characterization of monocyte membrane protein populations. Hannah is a three-time finalist in the UC Davis Biomedical Engineering Society’s Make-a-Thon medical device design and prototyping competition.

visit author page

biography

Marina Crowder

visit author page

Marina Crowder is currently Teaching Faculty in the Department of Molecular and Cellular Biology at UC Davis. In addition to teaching core undergraduate courses, Marina is aimed at understanding how to better support the development students' problem-solving skills. She has interests in graduate student teaching professional development, effective supplemental instruction models at the upper-division level, and improving the success of transfer students in STEM. Prior to joining UC Davis, Marina taught at Laney Community College and was a postdoctoral fellow in the laboratory of Dr. Rebecca Heald in the Molecular and Cellular Biology Department at UC Berkeley. She received her doctoral degree in Biochemistry, Molecular, Cellular and Developmental Biology and B.S. degree in Genetics, both from UC Davis.

visit author page

biography

Ozcan Gulacar University of California, Davis

visit author page

Dr. Gulacar has a Master’s degree in Physical Chemistry and a Ph.D. in Science Education. In the last 15 years, he has worked in settings including international high schools and doctorate granting institutions. He has designed and taught undergraduate/graduate chemistry and science education courses for a wide range of audiences. Due to his interest in investigating the effectiveness of different teaching methods and tools, he has received grants and established collaborations with colleagues from different fields and countries. Dr. Gulacar has developed and organized workshops about implementation of social constructivist methods and effective use of technological tools in science classrooms.

visit author page

biography

Jennifer H. Choi University of California, Davis

visit author page

Jennifer Choi is currently a Lecturer with potential for security of employment (LPSOE) in the Department of Biomedical Engineering (BME) at UC Davis. In addition to teaching core undergraduate courses, Jennifer is aimed at integrating engineering design principles and hands-on experiences throughout the curriculum, and playing an active role in the senior design course. She has interests in engineering education, curricular innovation, as well as impacting the community through increased K-12 STEM awareness and education. Prior to joining UC Davis, Jennifer taught in the BME Department at Rutgers University, and was a postdoctoral fellow at Advanced Technologies and Regenerative Medicine, LLC. She received her doctoral degree in Biomedical Engineering from Tufts University, M.S. degree from Syracuse University, and B.S. degree from Cornell University.

visit author page

Download Paper | Permalink

Abstract

An in-depth analysis of open-ended biomedical engineering design problems and the role of metacognition in their solutions

The need to build problem solving skills in STEM undergraduates has been widely reported1. This paper provides additional insight into the role of knowledge structure, knowledge retention, and misconceptions in solving open-ended biomedical engineering design problems. Correlations in problem solving performance to level of metacognitive awareness were also assessed.

Study participants were enrolled in a first-year introductory biomedical engineering (BME) course that introduces the field through BME specialization introductory lectures, prospective BME career guest lectures, and team-based hands-on design challenges. This two-unit course consists of one 50-minute lecture and a 3-hour discussion session focused on engineering design each week of a 10-week quarter. There were 142 students enrolled in this introductory course.

To gather a baseline of students’ design knowledge, the Comprehensive Assessment of Design Engineering Knowledge (CADEK) diagnostic test2 was administered to students in the first and last week of class. Students were also asked to complete an online Metacognitive Awareness Inventory (MAI)3 during week 2. In addition to the CADEK and MAI, students answered an open-ended design problem on their first quiz (in Week 5), from which ten high performing and ten low performing students were identified and asked to participate in one hour think-aloud interviews (TAInt). The TAInt were conducted during weeks 7 and 8 of the quarter and participants were encouraged to speak through their thought processes while being asked to solve three open ended BME design problems. To assess levels of knowledge retention, participants were asked to complete a follow up CADEK and participate in a second round of TAInt in the following quarter consisting of the same open-ended problems.

The COSINE (Coding System for Investigating Sub-problems and the Network) method was utilized to analyze the difficulties students have during the problem solving process on the open-ended design problems from the in-class quiz and TAInt4. In the COSINE method analysis, sub-problems that correlate with specific steps of the engineering design process were assigned a code based on student performance in a particular task. Quantitative metrics were developed based on resulting codes to gain insight into where and why students are unsuccessful. One developed metric is the complete success rate (CSR), which describes the percentage of successful attempts for an identified sub-problem relative to all codes that were assigned for that sub-problem over all participants. In the first TAInt, problem identification (sub-problem A), had a complete success rate of 70%. In contrast, the tasks of identifying user needs (sub-problem D), and engineering metric formulation (sub-problem F), had lower complete success rates of 16.67% and 1.67% respectively. 75% of all participants identified at least 50% of user needs for all three design problems yet not a single student could develop at least one metric for all three problems. Interestingly, 80% of the high-performing students were able to translate their identified user needs into their design solution (sub-problem E), whereas only 40% of the low-performing students incorporated their identified user needs. Behaviors such as re-reading the problem statement and drawing device designs were also evaluated. 100% of high performing students used drawing as a tool to succeed in problem three compared to 40% of low performing students. Collected data is currently being analyzed to determine how metacognition and prior knowledge correlate with problem-solving behavior. Assessing study participants over the course of the BME undergraduate curriculum will provide insight into strengths and areas for improvement of design instruction across the curriculum.

Literature Cited 1 Saavedra, A. R.; Saavedra, J. E., Do colleges cultivate critical thinking, problem solving, writing and interpersonal skills? Economics of Education Review 2011, 30 (6), 1516-1526. 2 Okudan, G.; Ogot, M.; Gupta, S. Assessment of Learning and its Retention in the Engineering Design Classroom Part A: Instrument Development. American Society for Engineering Education Conference Proceedings 2007, AC 2007-2261. 3 Mulford, D. R.; Robinson, W. R., An Inventory for Alternate Conceptions among First-Semester General Chemistry Students. Journal of Chemical Education 2002, 79 (6), 739-744. 4 Gulacar, O.; Overton, T. L.; Bowman, C. R.; Fynewever, H., A novel code system for revealing sources of students' difficulties with stoichiometry. Chemistry Education Research and Practice 2013, 14 (4), 507-515.

Citation
Format

Yssels, H., & Crowder, M., & Gulacar, O., & Choi, J. H. (2018, June), An In-depth Analysis of Open-ended Biomedical Engineering Design Problems and the Role of Metacognition in Their Solutions Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--29788

TY - CPAPER
AB - An in-depth analysis of open-ended biomedical engineering design problems and the role of metacognition in their solutions

The need to build problem solving skills in STEM undergraduates has been widely reported1. This paper provides additional insight into the role of knowledge structure, knowledge retention, and misconceptions in solving open-ended biomedical engineering design problems. Correlations in problem solving performance to level of metacognitive awareness were also assessed.

Study participants were enrolled in a first-year introductory biomedical engineering (BME) course that introduces the field through BME specialization introductory lectures, prospective BME career guest lectures, and team-based hands-on design challenges. This two-unit course consists of one 50-minute lecture and a 3-hour discussion session focused on engineering design each week of a 10-week quarter. There were 142 students enrolled in this introductory course.

To gather a baseline of students’ design knowledge, the Comprehensive Assessment of Design Engineering Knowledge (CADEK) diagnostic test2 was administered to students in the first and last week of class. Students were also asked to complete an online Metacognitive Awareness Inventory (MAI)3 during week 2. In addition to the CADEK and MAI, students answered an open-ended design problem on their first quiz (in Week 5), from which ten high performing and ten low performing students were identified and asked to participate in one hour think-aloud interviews (TAInt). The TAInt were conducted during weeks 7 and 8 of the quarter and participants were encouraged to speak through their thought processes while being asked to solve three open ended BME design problems. To assess levels of knowledge retention, participants were asked to complete a follow up CADEK and participate in a second round of TAInt in the following quarter consisting of the same open-ended problems.

The COSINE (Coding System for Investigating Sub-problems and the Network) method was utilized to analyze the difficulties students have during the problem solving process on the open-ended design problems from the in-class quiz and TAInt4. In the COSINE method analysis, sub-problems that correlate with specific steps of the engineering design process were assigned a code based on student performance in a particular task. Quantitative metrics were developed based on resulting codes to gain insight into where and why students are unsuccessful. One developed metric is the complete success rate (CSR), which describes the percentage of successful attempts for an identified sub-problem relative to all codes that were assigned for that sub-problem over all participants. In the first TAInt, problem identification (sub-problem A), had a complete success rate of 70%. In contrast, the tasks of identifying user needs (sub-problem D), and engineering metric formulation (sub-problem F), had lower complete success rates of 16.67% and 1.67% respectively. 75% of all participants identified at least 50% of user needs for all three design problems yet not a single student could develop at least one metric for all three problems. Interestingly, 80% of the high-performing students were able to translate their identified user needs into their design solution (sub-problem E), whereas only 40% of the low-performing students incorporated their identified user needs. Behaviors such as re-reading the problem statement and drawing device designs were also evaluated. 100% of high performing students used drawing as a tool to succeed in problem three compared to 40% of low performing students. Collected data is currently being analyzed to determine how metacognition and prior knowledge correlate with problem-solving behavior. Assessing study participants over the course of the BME undergraduate curriculum will provide insight into strengths and areas for improvement of design instruction across the curriculum.

Literature Cited
1 Saavedra, A. R.; Saavedra, J. E., Do colleges cultivate critical thinking, problem solving, writing and interpersonal skills? Economics of Education Review 2011, 30 (6), 1516-1526.
2 Okudan, G.; Ogot, M.; Gupta, S. Assessment of Learning and its Retention in the Engineering Design Classroom Part A: Instrument Development. American Society for Engineering Education Conference Proceedings 2007, AC 2007-2261.
3 Mulford, D. R.; Robinson, W. R., An Inventory for Alternate Conceptions among First-Semester General Chemistry Students. Journal of Chemical Education 2002, 79 (6), 739-744.
4 Gulacar, O.; Overton, T. L.; Bowman, C. R.; Fynewever, H., A novel code system for revealing sources of students&#39; difficulties with stoichiometry. Chemistry Education Research and Practice 2013, 14 (4), 507-515.

AU - Hannah Yssels
AU - Marina Crowder
AU - Ozcan Gulacar
AU - Jennifer H. Choi
CY - Salt Lake City, Utah
DA - 2018/06/23
PB - ASEE Conferences
TI - An In-depth Analysis of Open-ended Biomedical Engineering Design Problems and the Role of Metacognition in Their Solutions
UR - https://peer.asee.org/29788
DO - 10.18260/1-2--29788
ER -