Salt Lake City, Utah
June 23, 2018
June 23, 2018
July 27, 2018
An in-depth analysis of open-ended biomedical engineering design problems and the role of metacognition in their solutions
The need to build problem solving skills in STEM undergraduates has been widely reported1. This paper provides additional insight into the role of knowledge structure, knowledge retention, and misconceptions in solving open-ended biomedical engineering design problems. Correlations in problem solving performance to level of metacognitive awareness were also assessed.
Study participants were enrolled in a first-year introductory biomedical engineering (BME) course that introduces the field through BME specialization introductory lectures, prospective BME career guest lectures, and team-based hands-on design challenges. This two-unit course consists of one 50-minute lecture and a 3-hour discussion session focused on engineering design each week of a 10-week quarter. There were 142 students enrolled in this introductory course.
To gather a baseline of students’ design knowledge, the Comprehensive Assessment of Design Engineering Knowledge (CADEK) diagnostic test2 was administered to students in the first and last week of class. Students were also asked to complete an online Metacognitive Awareness Inventory (MAI)3 during week 2. In addition to the CADEK and MAI, students answered an open-ended design problem on their first quiz (in Week 5), from which ten high performing and ten low performing students were identified and asked to participate in one hour think-aloud interviews (TAInt). The TAInt were conducted during weeks 7 and 8 of the quarter and participants were encouraged to speak through their thought processes while being asked to solve three open ended BME design problems. To assess levels of knowledge retention, participants were asked to complete a follow up CADEK and participate in a second round of TAInt in the following quarter consisting of the same open-ended problems.
The COSINE (Coding System for Investigating Sub-problems and the Network) method was utilized to analyze the difficulties students have during the problem solving process on the open-ended design problems from the in-class quiz and TAInt4. In the COSINE method analysis, sub-problems that correlate with specific steps of the engineering design process were assigned a code based on student performance in a particular task. Quantitative metrics were developed based on resulting codes to gain insight into where and why students are unsuccessful. One developed metric is the complete success rate (CSR), which describes the percentage of successful attempts for an identified sub-problem relative to all codes that were assigned for that sub-problem over all participants. In the first TAInt, problem identification (sub-problem A), had a complete success rate of 70%. In contrast, the tasks of identifying user needs (sub-problem D), and engineering metric formulation (sub-problem F), had lower complete success rates of 16.67% and 1.67% respectively. 75% of all participants identified at least 50% of user needs for all three design problems yet not a single student could develop at least one metric for all three problems. Interestingly, 80% of the high-performing students were able to translate their identified user needs into their design solution (sub-problem E), whereas only 40% of the low-performing students incorporated their identified user needs. Behaviors such as re-reading the problem statement and drawing device designs were also evaluated. 100% of high performing students used drawing as a tool to succeed in problem three compared to 40% of low performing students. Collected data is currently being analyzed to determine how metacognition and prior knowledge correlate with problem-solving behavior. Assessing study participants over the course of the BME undergraduate curriculum will provide insight into strengths and areas for improvement of design instruction across the curriculum.
Literature Cited 1 Saavedra, A. R.; Saavedra, J. E., Do colleges cultivate critical thinking, problem solving, writing and interpersonal skills? Economics of Education Review 2011, 30 (6), 1516-1526. 2 Okudan, G.; Ogot, M.; Gupta, S. Assessment of Learning and its Retention in the Engineering Design Classroom Part A: Instrument Development. American Society for Engineering Education Conference Proceedings 2007, AC 2007-2261. 3 Mulford, D. R.; Robinson, W. R., An Inventory for Alternate Conceptions among First-Semester General Chemistry Students. Journal of Chemical Education 2002, 79 (6), 739-744. 4 Gulacar, O.; Overton, T. L.; Bowman, C. R.; Fynewever, H., A novel code system for revealing sources of students' difficulties with stoichiometry. Chemistry Education Research and Practice 2013, 14 (4), 507-515.
Yssels, H., & Crowder, M., & Gulacar, O., & Choi, J. H. (2018, June), An In-depth Analysis of Open-ended Biomedical Engineering Design Problems and the Role of Metacognition in Their Solutions Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--29788
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015