Impact of Two Reflective Practices in an Engineering Laboratory Course using Standards-based Grading

Casey Ankeny; Amy Adkins; David O'Neill

Download Paper | Permalink

Conference: 2022 ASEE Annual Conference & Exposition
Location: Minneapolis, MN
Publication Date: August 23, 2022
Start Date: June 26, 2022
End Date: June 29, 2022
Conference Session: Biomedical Engineering Division: Developing Lab and Research Skills for BioE/BME Students
Page Count: 13
DOI: 10.18260/1-2--40716
Permanent URL: https://peer.asee.org/40716
Download Count: 236

Paper Authors

biography

Casey Ankeny Northwestern University

visit author page

Casey J. Ankeny, PhD is an Associate Professor of Instruction at Northwestern University. Casey received her bachelor’s degree in Biomedical Engineering from the University of Virginia and her doctorate degree in Biomedical Engineering from Georgia Institute of Technology and Emory University where she studied the role of shear stress in aortic valve disease. Currently, she is investigating cyber-based student engagement strategies in flipped and traditional biomedical engineering courses. She aspires to understand and improve student attitude, achievement, and persistence in student-centered courses.

visit author page

biography

Amy Adkins Northwestern University

visit author page

Amy N. Adkins is an Assistant Teaching Professor in the Joint Department of Biomedical Engineering at University of North Carolina (UNC) and North Carolina State University (NCSU). She received her Ph.D. and M.S. in Biomedical Engineering from Northwestern University and her B.S. in Engineering Science from St. Mary's University in San Antonio. Her technical research which relates to her PhD Dissertation is focused on utilizing novel imaging techniques to quantifying adaptation of muscle structure in humans. She also desires to implement innovative teaching, mentoring, and hands-on problem solving to develop students’ deep understanding of engineering principles and to inspire them to tackle real-world problems which can aid human health.

visit author page

biography

David O'Neill Northwestern University

visit author page

David O'Neill is an Associate Professor of Instruction and the Michael Jaharis Director of Experiential Learning for the Biomedical Engineering Department at Northwestern University. He read Engineering Science at University College, Oxford, receiving his M.Eng. and D.Phil. degrees. During his doctoral studies in the engineering department and post-doctoral work in the department of physiology, anatomy and genetics, he taught undergraduate tutorials for Keble, New, University, and Harris Manchester Colleges, was College Lecturer for New College and a Senior College Lecturer in Engineering Science for Keble College.
Current projects and interests include implementing mastery-based grading schemes, and how cognitive load can be affected by vertical curricula and the level of synopticism throughout the biomedical engineering degree path .

visit author page

Download Paper | Permalink

Abstract

Introduction:

Reflective practices have been used in conjunction with many evidence-based strategies, including standards-based grading [1] to tailor student learning and efforts and are known to increase student awareness about learning [2]. This study investigated the implementation of different reflective practices in a laboratory course that uses standards-based grading. We hypothesized that reflecting while completing deliverables in addition to post-submission reflection would enhance learning as measured by a standards-based grading system when compared to only reflecting post-submission.

Methods:

This study was conducted over two terms. During the first term students completed short post-submission reflective surveys based on work by Diefes-Dux, et al., [3-4] before and after receiving instructional feedback on the submitted assignment. In the second term, students were additionally asked to reflect while doing in the form of laboratory notebooks. The notebooks were graded and students were provided detailed feedback from the instructional team. The laboratory course consisted of two modules (M1 and M2) and deliverables for each module included both a draft and full report (team assignments).

The effectiveness of reflection was assessed by grading the post-submission surveys similar to previous work [4-5]. Students also participated in a post-course attitudinal survey consisting of questions regarding the value of reflections with SBG as well as questions about their engagement with the reflective practices [6].

Results and Discussion:

When analyzing report scores, we saw a statistically significant improvement for report scores from M1 to M2 (p<0.05) in the first term; however, we did not see a significant difference in the second term (p=0.83). This could be due to notebooks in M1 accelerating student learning towards mastery before report submission as noted by higher report scores in the second term compared to the first. During both terms, we saw a positive correlative trend between individual reflection survey scores and final exam scores [1st term: 0.64 (p=0.02) and 2nd term: 0.42 (p=0.1)] which could indicate: 1) strong students typically score highly on all assignments or 2) reflecting deeply aids individual performance in final exams.

We found a mildly positive correlative trend between reflection survey scores and lab report assignments for the first term (0.24; p=0.46; n=12) and a mildly negative correlative trend for the second term (-0.37; p=0.16; n=16) though neither were statistically significant. In the future, we will increase sample size to that required to find a large effect size (n=29). Further, these results informed future work targeting implementation of individual deliverables rather than group reports.

The reflection engagement survey showed students do reflect on SBG and incorporate their reflections into the next iteration of the assignment and had a general positive attitude regarding their implementation; however, we saw a disconnect between SBG reflections and students’ perceived utility with exam preparation.

Conclusion:

Limitations of the study include small class sizes in both course offerings, <100% participation in the study, and with two modules, students had a limited number of attempts to demonstrate mastery. Assessing intervention effectiveness was complicated by a mix of individual reflective practices and team-based lab reports.

Our initial results showed that reflection may aid student mastery. Further these results directed the design of a future study to rigorously answer how individual reflective techniques translate to individual mastery of course standards.

Citation
Format

Ankeny, C., & Adkins, A., & O'Neill, D. (2022, August), Impact of Two Reflective Practices in an Engineering Laboratory Course using Standards-based Grading Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. 10.18260/1-2--40716

TY - CPAPER
AB - Introduction:

Reflective practices have been used in conjunction with many evidence-based strategies, including standards-based grading [1] to tailor student learning and efforts and are known to increase student awareness about learning [2]. This study investigated the implementation of different reflective practices in a laboratory course that uses standards-based grading. We hypothesized that reflecting while completing deliverables in addition to post-submission reflection would enhance learning as measured by a standards-based grading system when compared to only reflecting post-submission.

Methods:

This study was conducted over two terms. During the first term students completed short post-submission reflective surveys based on work by Diefes-Dux, et al., [3-4] before and after receiving instructional feedback on the submitted assignment. In the second term, students were additionally asked to reflect while doing in the form of laboratory notebooks. The notebooks were graded and students were provided detailed feedback from the instructional team. The laboratory course consisted of two modules (M1 and M2) and deliverables for each module included both a draft and full report (team assignments).

The effectiveness of reflection was assessed by grading the post-submission surveys similar to previous work [4-5]. Students also participated in a post-course attitudinal survey consisting of questions regarding the value of reflections with SBG as well as questions about their engagement with the reflective practices [6].

Results and Discussion:

When analyzing report scores, we saw a statistically significant improvement for report scores from M1 to M2 (p<0.05) in the first term; however, we did not see a significant difference in the second term (p=0.83). This could be due to notebooks in M1 accelerating student learning towards mastery before report submission as noted by higher report scores in the second term compared to the first.
During both terms, we saw a positive correlative trend between individual reflection survey scores and final exam scores [1st term: 0.64 (p=0.02) and 2nd term: 0.42 (p=0.1)] which could indicate: 1) strong students typically score highly on all assignments or 2) reflecting deeply aids individual performance in final exams.

We found a mildly positive correlative trend between reflection survey scores and lab report assignments for the first term (0.24; p=0.46; n=12) and a mildly negative correlative trend for the second term (-0.37; p=0.16; n=16) though neither were statistically significant. In the future, we will increase sample size to that required to find a large effect size (n=29). Further, these results informed future work targeting implementation of individual deliverables rather than group reports.

The reflection engagement survey showed students do reflect on SBG and incorporate their reflections into the next iteration of the assignment and had a general positive attitude regarding their implementation; however, we saw a disconnect between SBG reflections and students’ perceived utility with exam preparation.

Conclusion:

Limitations of the study include small class sizes in both course offerings, <100% participation in the study, and with two modules, students had a limited number of attempts to demonstrate mastery. Assessing intervention effectiveness was complicated by a mix of individual reflective practices and team-based lab reports.

Our initial results showed that reflection may aid student mastery. Further these results directed the design of a future study to rigorously answer how individual reflective techniques translate to individual mastery of course standards.

AU - Casey Ankeny
AU - Amy Adkins
AU - David O'Neill
CY - Minneapolis, MN
DA - 2022/08/23
PB - ASEE Conferences
TI - Impact of Two Reflective Practices in an Engineering Laboratory Course using Standards-based Grading
UR - https://peer.asee.org/40716
DO - 10.18260/1-2--40716
ER -