Using Scaffolded Exams and Post-Exam Reflection to Foster Students’ Metacognitive Regulation of Learning in a Mechanics of Materials Class

Huihui Qi; Isabella Fiorini; Edward Zhou Yang Yu; Richard Eugene Vallejo; Zongnan Wang; Trevor Keoki Oshiro; Changkai Chen

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Student Self-assessment in Mechanics Courses
Tagged Division: Mechanics Division (MECHS)
Permanent URL: https://peer.asee.org/48240

Request a correction

Paper Authors

biography

Huihui Qi University of California, San Diego

visit author page

Dr.Huihui Qi is an Associate Teaching Professor in the department of Mechanical and Aerospace Engineering, at the University of California San Diego.

visit author page

author page

Isabella Fiorini University of California, San Diego

biography

Edward Zhou Yang Yu University of California, San Diego orcid.org/0009-0008-3429-6774

visit author page

Edward Yu is a third-year undergraduate student at UC San Diego majoring in Aerospace Engineering with a specialization in Astrodynamics. Edward mainly assists with the literature review as well as manuscript writing and editing. In addition to research, Edward spends his time on campus building rockets at the UCSD chapter of Students for the Exploration and Development of Space, where he designs, simulates, and manufactures structural components. Edward plans on pursuing a PhD in a structures-related field, which he hopes to leverage in the development of more cost-effective and useful space vehicles. In his free time, Edward enjoys watching films and picking up a good book.

visit author page

biography

Richard Eugene Vallejo Jr University of California, San Diego

visit author page

Richard Vallejo is a third-year undergraduate student at UC San Diego majoring in Mechanical Engineering with a specialization in Controls and Robotics. Richard assists with concept generation and manufacturing for inquiry-based learning activity models and demonstrations. In addition, Richard is an instructional assistant for Introduction to Mechanical Design at UCSD, leading a lab section through basic CAD, workshop, and technical report and presentation techniques. He plans on pursuing a future career in engineering education at the high school level. His hobbies also include spending time with friends, visiting national parks, and creating small robotics projects.

visit author page

biography

Zongnan Wang University of California, San Diego

visit author page

Zongnan is currently a 3rd-year undergraduate student with a major in mechanical engineering and minor in Data Science. He has been assisting Dr. Huihui Qi at the Engineering Pedagogy and Design Lab, with the work of processing and analyzing data collected from de-identified survey responses as well as coursework evaluations, to achieve various research purpose.

visit author page

author page

Trevor Keoki Oshiro University of California, San Diego

author page

Changkai Chen University of California, San Diego

Download Paper | Permalink

Abstract

Assessment plays a critical role in students' learning, as it provides feedback to both instructors and students about the learning progress, and potential adjustment needed for both teaching and learning strategies. Often, students’ learning strategies are steered by the exams. Some students may have a good understanding of the basic concepts, but not be able to apply them in a complex application situation; some students may have a good conceptual understanding, but lack of rigor on procedural knowledge. Being able to identify such areas of improvement is important for students to adjust their next-step learning approaches. The strength and weakness identification, and adjustment of learning strategies requires effective engagement in metacognition. But it is not rare that students often fail to practice metacognition and reflect on their learning strategies to make the appropriate adjustment as needed. At times, students do not clearly understand whether their unsatisfactory exam results are due to time constraints of the exam, or their knowledge or problem-solving skills need to be improved. Also, some students have a difficult time understanding if they lack understanding of their procedural knowledge or individual concepts, or they lack conditional knowledge-when to use isolated concepts together, or problem-solving skills. Thus, assessment methods that can promote students’ metacognition will be important. Both in-class (closed-book) exam (ICE) and take-home (open-book) exams (THE) have been widely used for engineering classes. Each format provides its advantages and disadvantages. ICEs appear to be a more effective tool for testing the lower levels of Bloom’s taxonomy scale such as remembering , and are associated with less student cheating . On the other hand, THEs appear to be more productive for developing and testing higher levels of the Bloom’s taxonomy scale (e.g. analyzing and evaluating), as the student can invest their energy into deeper engagement and understanding rather than memorizing the material.

The impact of the combination of in-class and take-home exams on helping engineering students’ engagement with metacognition has rarely been studied. In this paper, the impact of the use of a combination of in-class and take-home exams, along with post-exam wrappers on students' metacognition, has been studied. The assessment structure has been implemented into 2 engineering mechanics classes (Statics and Mechanics of Materials) in an public research university in the United States. Between the two classes, a total of 220 students were included in this study. The exams were mindfully designed to serve distinct purposes: the closed book in-class exams were timed, and focused on the isolated concepts without in-depth problem-solving, with no access to books or notes unless necessary equations were given; the open-book take-home exams, on the other hands, were un-timed and aimed to test students higher-order thinking, problem-solving skills using many concepts together. Students’ exam performance (3 midterm quizzes and 1 final exam), exam wrappers' open-ended responses, and end-of-quarter survey results are studied collectively. From the exam wrappers' open-ended responses, and end-of-quarter survey questions, students’ perceptions about how the closed-book in-class and open-book take-home exams help them to identify the strength and weaknesses of their learning progress and their learning strategies differently. Regression analysis is used to understand the relationship between exam performance progress and their perception of the exams’ impact on their metacognition and learning strategies. A comparison of final exams performance (difficulty level calibrated) between the previous years' cohorts and the cohorts with the two-part exams is made, showing the students with the two-part exams performed better than the previous cohorts with only closed-book in-class exam or open-book online timed exams.

Citation
Format

Qi, H., & Fiorini, I., & Yu, E. Z. Y., & Vallejo, R. E., & Wang, Z., & Oshiro, T. K., & Chen, C. (2024, June), Using Scaffolded Exams and Post-Exam Reflection to Foster Students’ Metacognitive Regulation of Learning in a Mechanics of Materials Class Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/48240

TY  - CPAPER
AB  - Assessment plays a critical role in students&#39; learning, as it provides feedback to both instructors and students about the learning progress, and potential adjustment needed for both teaching and learning strategies. Often, students’ learning strategies are steered by the exams. Some students may have a good understanding of the basic concepts, but not be able to apply them in a complex application situation; some students may have a good conceptual understanding, but lack of rigor on procedural knowledge. Being able to identify such areas of improvement is important for students to adjust their next-step learning approaches.
The strength and weakness identification, and adjustment of learning strategies requires effective engagement in metacognition. But it is not rare that students often fail to practice metacognition and reflect on their learning strategies to make the appropriate adjustment as needed. At times, students do not clearly understand whether their unsatisfactory exam results are due to time constraints of the exam, or their knowledge or problem-solving skills need to be improved. Also, some students have a difficult time understanding if they lack understanding of their procedural knowledge or individual concepts, or they lack conditional knowledge-when to use isolated concepts together, or problem-solving skills. Thus, assessment methods that can promote students’ metacognition will be important.
Both in-class (closed-book) exam (ICE) and take-home (open-book) exams (THE) have been widely used for engineering classes. Each format provides its advantages and disadvantages. ICEs appear to be a more effective tool for testing the lower levels of Bloom’s taxonomy scale such as remembering , and are associated with less student cheating . On the other hand, THEs appear to be more productive for developing and testing higher levels of the Bloom’s taxonomy scale (e.g. analyzing and evaluating), as the student can invest their energy into deeper engagement and understanding rather than memorizing the material.

The impact of the combination of in-class and take-home exams on helping engineering students’ engagement with metacognition has rarely been studied. In this paper, the impact of the use of a combination of in-class and take-home exams, along with post-exam wrappers on students&#39; metacognition, has been studied. The assessment structure has been implemented into 2 engineering mechanics classes (Statics and Mechanics of Materials) in an public research university in the United States. Between the two classes, a total of 220 students were included in this study. The exams were mindfully designed to serve distinct purposes: the closed book in-class exams were timed, and focused on the isolated concepts without in-depth problem-solving, with no access to books or notes unless necessary equations were given; the open-book take-home exams, on the other hands, were un-timed and aimed to test students higher-order thinking, problem-solving skills using many concepts together.
Students’ exam performance (3 midterm quizzes and 1 final exam), exam wrappers&#39; open-ended responses, and end-of-quarter survey results are studied collectively. From the exam wrappers&#39; open-ended responses, and end-of-quarter survey questions, students’ perceptions about how the closed-book in-class and open-book take-home exams help them to identify the strength and weaknesses of their learning progress and their learning strategies differently. Regression analysis is used to understand the relationship between exam performance progress and their perception of the exams’ impact on their metacognition and learning strategies. A comparison of final exams performance (difficulty level calibrated) between the previous years&#39; cohorts and the cohorts with the two-part exams is made, showing the students with the two-part exams performed better than the previous cohorts with only closed-book in-class exam or open-book online timed exams.
AU  - Huihui Qi
AU  - Isabella Fiorini
AU  - Edward Zhou Yang Yu
AU  - Richard Eugene Vallejo Jr
AU  - Zongnan Wang
AU  - Trevor Keoki Oshiro
AU  - Changkai Chen
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - Using Scaffolded Exams and Post-Exam Reflection to Foster Students’ Metacognitive Regulation of Learning in a Mechanics of Materials Class
UR  - https://peer.asee.org/48240
ER  -