Measuring the Pedagogical Impact on Undergraduate Students through Frequent, Low-Stakes Pre- and Post-Lecture Self-Assessments

Reem Khojah; Josephine Relaford-Doyle

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Biomedical Engineering Division (BED) Technical Session 3
Tagged Division: Biomedical Engineering Division (BED)
Permanent URL: https://peer.asee.org/47767

Request a correction

Paper Authors

biography

Reem Khojah University of California, San Diego

visit author page

Reem Khojah serves as an assistant teaching professor in the Shu Chien-Gene Lay Department of Bioengineering at the University of California, San Diego. With experience in instructing bioengineering at introductory and graduate levels, she actively contributes to enhancing accessibility to research tools for undergraduate research experiences. Her primary focus is on optimizing engineering education through data-driven pre-and post-lecture formative assessments and designing AI-proof assignments. Her educational background includes a Ph.D. in Bioengineering from the University of California, Los Angeles. Reem has also engaged in post-doctoral research at the University of California, Santa Cruz, and the University of California, Irvine.

visit author page

biography

Josephine Relaford-Doyle University of California, San Diego

visit author page

Dr. Josephine Relaford-Doyle is an Education Specialist in the Teaching + Learning Commons at UC San Diego, where she works in faculty development and instructional support. She holds a PhD in Cognitive Science and has researched STEM learning at both the K12 and post-secondary levels.

visit author page

Download Paper | Permalink

Abstract

Formative assessment is a crucial tool in engineering education, known for enhancing learning outcomes. Recent research has indicated that timely, aligned assessments with course learning outcomes (CLOs) can substantially increase student learning [1,2]. These assessments serve the dual purpose of identifying challenges in student learning and facilitating timely, targeted interventions by instructors, which is particularly important in the context of interdisciplinary bioengineering education [3,4]. Given the interdisciplinary nature of bioengineering, continuous engagement is necessary for a comprehensive understanding of essential engineering concepts and their practical applications, upon which subsequent lectures build [5]. To tackle these challenges, studies have shown that frequent, voluntary, and low-risk formative assessments, such as clickers, provide students with regular feedback that enhances their learning [6,7]

This study builds upon these principles by implementing real-time formative assessments in every lecture while systematically measuring student learning based on Bloom's taxonomy [8].

Method

To assess teaching effectiveness in graduate-level bioengineering education, low-stakes pre/post-lecture formative assessments were introduced. In each class, participants voluntarily completed these assessments within a 5-minute window at the beginning and end of the lecture, ensuring anonymity and a non-graded environment. The pre-lecture assessment established a knowledge baseline, while the post-lecture assessment gauged CLO achievement. These assessments were facilitated through Poll Everywhere, engaging students across Bloom's Taxonomy levels, including "remember," "understand," "apply," and "analyze." The "evaluate" and "create" levels were excluded due to time constraints, as they were already covered in homework assignments. By comparing pre- and post-lecture responses, normalized results were obtained, enabling the direct quantification of student learning and cognitive growth after the intervention. At the end of the course, participants were invited to volunteer answers to a comprehensive questionnaire to assess the usefulness of the pre/post-lecture assessments and to identify specific areas of benefit.

Results

This data-driven approach provided valuable insights into lecture effectiveness across various Bloom's Taxonomy levels. Nearly 90% of students voluntarily participated. In the first two Bloom's Taxonomy levels, students exhibited high scores, with significant improvements. Pre-lecture, 29% of students demonstrated an understanding of the material, which increased to 63% post-lecture, indicating a 34% improvement in comprehension. In the application level, the pre-lecture score was 23%, which rose to 52% post-lecture, reflecting a 29% increase in the application of knowledge. Furthermore, in the analyzing level, the pre-lecture score was 23%, and it improved to 51% post-lecture, with 28% of students demonstrating the ability to analyze the material. These results shed light on the impact of the lecture on students' cognitive development at various Bloom's Taxonomy levels, with an average improvement of 32% across the class.

Discussion

Our data categorizes pre/post-lecture assessments by Bloom's Taxonomy levels, aligning with cognitive demands and progression. As we ascend the hierarchy, students engage in more complex cognitive processes within limited timeframes, which may have contributed to the decrease in the average score.

Post-lecture, students anonymously shared their views on the lecture pace and the aspects they found interesting or confusing. This feedback empowered instructors to address areas of difficulty, a strategy well-received by students, with approximately 80% reporting improved teaching, and 11% expressing neutrality. Additionally, 66% of students agreed that this approach encouraged dialogue and positive interaction with the instructor. Our approach effectively dispelled confusion and facilitated student questions, evaluating the impact of instructional strategies and fostering interdisciplinary education.

Conclusion

Our low-stakes pre/post-lecture formative assessment approach offers tailored instruction and learning tools for interdisciplinary topics. It establishes an ongoing feedback loop, guiding lecture customization and the development of learning tools, creating a data-driven, equitable learning environment. Our goal is to establish an educational framework that resonates with the intricacies of bioengineering, fostering inclusivity and impactful learning. Ultimately, this approach aims to elevate pedagogical standards and provide guidance to educators navigating interdisciplinary education.

Citation
Format

Khojah, R., & Relaford-Doyle, J. (2024, June), Measuring the Pedagogical Impact on Undergraduate Students through Frequent, Low-Stakes Pre- and Post-Lecture Self-Assessments Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/47767

TY - CPAPER
AB - Formative assessment is a crucial tool in engineering education, known for enhancing learning outcomes. Recent research has indicated that timely, aligned assessments with course learning outcomes (CLOs) can substantially increase student learning [1,2]. These assessments serve the dual purpose of identifying challenges in student learning and facilitating timely, targeted interventions by instructors, which is particularly important in the context of interdisciplinary bioengineering education [3,4]. Given the interdisciplinary nature of bioengineering, continuous engagement is necessary for a comprehensive understanding of essential engineering concepts and their practical applications, upon which subsequent lectures build [5]. To tackle these challenges, studies have shown that frequent, voluntary, and low-risk formative assessments, such as clickers, provide students with regular feedback that enhances their learning [6,7]

This study builds upon these principles by implementing real-time formative assessments in every lecture while systematically measuring student learning based on Bloom&#39;s taxonomy [8].

Method

To assess teaching effectiveness in graduate-level bioengineering education, low-stakes pre/post-lecture formative assessments were introduced. In each class, participants voluntarily completed these assessments within a 5-minute window at the beginning and end of the lecture, ensuring anonymity and a non-graded environment. The pre-lecture assessment established a knowledge baseline, while the post-lecture assessment gauged CLO achievement. These assessments were facilitated through Poll Everywhere, engaging students across Bloom&#39;s Taxonomy levels, including &quot;remember,&quot; &quot;understand,&quot; &quot;apply,&quot; and &quot;analyze.&quot; The &quot;evaluate&quot; and &quot;create&quot; levels were excluded due to time constraints, as they were already covered in homework assignments. By comparing pre- and post-lecture responses, normalized results were obtained, enabling the direct quantification of student learning and cognitive growth after the intervention. At the end of the course, participants were invited to volunteer answers to a comprehensive questionnaire to assess the usefulness of the pre/post-lecture assessments and to identify specific areas of benefit.

Results

This data-driven approach provided valuable insights into lecture effectiveness across various Bloom&#39;s Taxonomy levels. Nearly 90% of students voluntarily participated. In the first two Bloom&#39;s Taxonomy levels, students exhibited high scores, with significant improvements. Pre-lecture, 29% of students demonstrated an understanding of the material, which increased to 63% post-lecture, indicating a 34% improvement in comprehension. In the application level, the pre-lecture score was 23%, which rose to 52% post-lecture, reflecting a 29% increase in the application of knowledge. Furthermore, in the analyzing level, the pre-lecture score was 23%, and it improved to 51% post-lecture, with 28% of students demonstrating the ability to analyze the material. These results shed light on the impact of the lecture on students&#39; cognitive development at various Bloom&#39;s Taxonomy levels, with an average improvement of 32% across the class.

Discussion

Our data categorizes pre/post-lecture assessments by Bloom&#39;s Taxonomy levels, aligning with cognitive demands and progression. As we ascend the hierarchy, students engage in more complex cognitive processes within limited timeframes, which may have contributed to the decrease in the average score.

Post-lecture, students anonymously shared their views on the lecture pace and the aspects they found interesting or confusing. This feedback empowered instructors to address areas of difficulty, a strategy well-received by students, with approximately 80% reporting improved teaching, and 11% expressing neutrality. Additionally, 66% of students agreed that this approach encouraged dialogue and positive interaction with the instructor. Our approach effectively dispelled confusion and facilitated student questions, evaluating the impact of instructional strategies and fostering interdisciplinary education.

Conclusion

Our low-stakes pre/post-lecture formative assessment approach offers tailored instruction and learning tools for interdisciplinary topics. It establishes an ongoing feedback loop, guiding lecture customization and the development of learning tools, creating a data-driven, equitable learning environment. Our goal is to establish an educational framework that resonates with the intricacies of bioengineering, fostering inclusivity and impactful learning. Ultimately, this approach aims to elevate pedagogical standards and provide guidance to educators navigating interdisciplinary education.

AU - Reem Khojah
AU - Josephine Relaford-Doyle
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Measuring the Pedagogical Impact on Undergraduate Students through Frequent, Low-Stakes Pre- and Post-Lecture Self-Assessments
UR - https://peer.asee.org/47767
ER -