Salt Lake City, Utah
June 23, 2018
June 23, 2018
July 27, 2018
Educational Research and Methods
Measuring Engineering Students’ Metacognition with a Think-Aloud Protocol
This evidence-based practice paper describes one method of measuring engineering students’ use of metacognition while problem solving. The capacity for life-long learning is critical for engineering practice and according to the Accreditation Board for Engineering and Technology (ABET) is a required outcome for engineering accreditation. Metacognition, defined as cognition of one’s cognitions, is a higher-order thinking skill and provides the key to developing life-long learning skills necessary for ABET and for an effective work career, yet it is rarely integrated into engineering education. At Iron Range Engineering (IRE), an innovative, problem-based-learning (PBL) engineering program in Virginia, Minnesota, students learn about and apply metacognitive skills. In our NSF-funded project, we are (1) identifying and understanding the metacognitive skills students develop and use during their preparation as engineers in a PBL program and (2) examining whether the preparation of students in the PBL program (particularly in the area of metacognition) gives them a “leg up” in their transition to the engineering workforce. Our mixed-methods study consists of student interview data supplemented with interviews of recent IRE graduates employed as engineers and of employers of the IRE graduates to better understand the effect of this unique undergraduate program on student preparedness for the engineering workforce. In addition, we are conducting think-aloud protocols during student solving of an open-ended design problem.
In this paper, we focus on the think-aloud data we have gathered during this project. We describe our process of gathering think-aloud data from students, as well as our coding structure, which categorizes utterances as metacognitive or not, and, if metacognitive, whether the utterance is a type of metacognitive monitoring or metacognitive control. In addition, we detail our coding process and measures of inter-rater reliability. Finally, we share results to date, including comparisons of the think-aloud data from students about to graduate from the program with students entering the program. The think-alouds from entering students and graduating students are showing marked differences. Our intent is to demonstrate the presence of critical metacognitive processes during problem solving as students progress through their engineering education.
Plumb, C., & Marra, R. M., & Hacker, D. J., & Dunlosky, J. (2018, June), Measuring Engineering Students’ Metacognition with a Think-Aloud Protocol Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. https://peer.asee.org/30796
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015