WIP: Comparison of a Standards-based Assessment to a Traditional, Summative Rubric in a Biomedical Engineering Laboratory

Casey Jane Ankeny; David P. O'Neill; Lisa Beckmann

Download Paper | Permalink

Conference: 2020 ASEE Virtual Annual Conference Content Access
Location: Virtual On line
Publication Date: June 22, 2020
Start Date: June 22, 2020
End Date: June 26, 2021
Conference Session: Laboratory Learning in Biomedical Engineering (Works in Progress) - June 24th
Tagged Division: Biomedical Engineering
Page Count: 5
DOI: 10.18260/1-2--35529
Permanent URL: https://peer.asee.org/35529
Download Count: 449

Paper Authors

biography

Casey Jane Ankeny Northwestern University

visit author page

Casey J. Ankeny, PhD is an Assistant Professor of Instruction at Northwestern University. Casey received her bachelor’s degree in Biomedical Engineering from the University of Virginia in 2006 and her doctorate degree in Biomedical Engineering from Georgia Institute of Technology and Emory University in 2012 where she studied the role of shear stress in aortic valve disease. Currently, she is investigating cyber-based student engagement strategies in flipped and traditional biomedical engineering courses. She aspires to understand and improve student attitude, achievement, and persistence in student-centered courses.

visit author page

author page

David P. O'Neill Northwestern University

author page

Lisa Beckmann Northwestern University

Download Paper | Permalink

Abstract

Standards-based grading (SBG) is a form of formative assessment that allows for tracking of objective mastery throughout the curriculum and allows for just-in-time feedback. Although widely implemented in K-12, we are beginning to assess its use in engineering in higher education (Carberry, et al. 2016). Here, we hypothesize that SBG implementation will result in greater attainment of learning objectives, a high-level of persistence, and positive attitude.

Recently, we described SBG implementation in an experimental design course focusing on problem solving skills. Here, we will describe our assessment of its implementation. We measured persistence through attendance and student attitude through value and confidence surveys (Carberry, et al. 2013 and Blumer). To better understand achievement, we evaluated standards mastery before and after implementation of the SBG rubric. We reviewed all reports for the Winter 2019 (W19; n=3) offering as well as an analogous number of reports from Spring 2018 (SP18, n=3) using both the traditional, summative rubric (TRAD) from SP18 and the SBG rubric from W19. Additionally, we compared the instructor’s evaluation to the TA’s evaluation.

Briefly, in W19, we found a high persistence level and student motivation for learning when using the SBG rubric. Further, the intervention was not costly in terms of time and emotion and students reported increased confidence in scientific literacy. Students met mastery of each standard. When assessing SP18 and W19 student reports against both the TRAD rubric and the SBG rubric, six findings arose. The TA and instructor scores were inconsistent (p=0.03), thus, calling for grader calibration. Regardless of rubric given in the quarter, students scored higher with the SBG rubric than the TRAD rubric, thereby, alleviating concerns that grades will decrease with SBG implementation (p=0.01). Anecdotally, students that received the TRAD rubric appeared to score similarly on the SBG rubric as those given the SBG rubric. One possible explanation is that the TRAD rubric explicitly requested items inherently required to master the standard. It is important to note that the TRAD rubric removed the necessity for students to determine elements to include. Nonetheless, this finding supports call for reflection on grading to enhance future mastery (Diefes-Dux 2016). Student weaknesses across quarters were the same irrespective of the grading rubric (in the areas of “problem identification” and “interpretation”) and highlighted future teaching points. Students with the TRAD rubric appeared to be better at discussing signal processing and elements of a well-designed experiment. One explanation is that these elements were explicitly stated on the TRAD rubric. Observationally, students with the SBG rubric appeared to have a more coherent report due to the emphasis on communication in the SBG rubric.

These findings will drive future implementation changes in the course, including grader calibration and reflection. We will also work to implement SBG in a laboratory module in the next course in the sequence. These improvements have the potential to increase student attitude and achievement levels not only within this course but within subsequent courses. Further, it allows for increased student awareness of their developing skill set.

Citation
Format

Ankeny, C. J., & O'Neill, D. P., & Beckmann, L. (2020, June), WIP: Comparison of a Standards-based Assessment to a Traditional, Summative Rubric in a Biomedical Engineering Laboratory Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--35529

TY - CPAPER
AB - Standards-based grading (SBG) is a form of formative assessment that allows for tracking of objective mastery throughout the curriculum and allows for just-in-time feedback. Although widely implemented in K-12, we are beginning to assess its use in engineering in higher education (Carberry, et al. 2016). Here, we hypothesize that SBG implementation will result in greater attainment of learning objectives, a high-level of persistence, and positive attitude.

Recently, we described SBG implementation in an experimental design course focusing on problem solving skills. Here, we will describe our assessment of its implementation. We measured persistence through attendance and student attitude through value and confidence surveys (Carberry, et al. 2013 and Blumer). To better understand achievement, we evaluated standards mastery before and after implementation of the SBG rubric. We reviewed all reports for the Winter 2019 (W19; n=3) offering as well as an analogous number of reports from Spring 2018 (SP18, n=3) using both the traditional, summative rubric (TRAD) from SP18 and the SBG rubric from W19. Additionally, we compared the instructor’s evaluation to the TA’s evaluation.

Briefly, in W19, we found a high persistence level and student motivation for learning when using the SBG rubric. Further, the intervention was not costly in terms of time and emotion and students reported increased confidence in scientific literacy. Students met mastery of each standard. When assessing SP18 and W19 student reports against both the TRAD rubric and the SBG rubric, six findings arose. The TA and instructor scores were inconsistent (p=0.03), thus, calling for grader calibration. Regardless of rubric given in the quarter, students scored higher with the SBG rubric than the TRAD rubric, thereby, alleviating concerns that grades will decrease with SBG implementation (p=0.01). Anecdotally, students that received the TRAD rubric appeared to score similarly on the SBG rubric as those given the SBG rubric. One possible explanation is that the TRAD rubric explicitly requested items inherently required to master the standard. It is important to note that the TRAD rubric removed the necessity for students to determine elements to include. Nonetheless, this finding supports call for reflection on grading to enhance future mastery (Diefes-Dux 2016). Student weaknesses across quarters were the same irrespective of the grading rubric (in the areas of “problem identification” and “interpretation”) and highlighted future teaching points. Students with the TRAD rubric appeared to be better at discussing signal processing and elements of a well-designed experiment. One explanation is that these elements were explicitly stated on the TRAD rubric. Observationally, students with the SBG rubric appeared to have a more coherent report due to the emphasis on communication in the SBG rubric.

These findings will drive future implementation changes in the course, including grader calibration and reflection. We will also work to implement SBG in a laboratory module in the next course in the sequence. These improvements have the potential to increase student attitude and achievement levels not only within this course but within subsequent courses. Further, it allows for increased student awareness of their developing skill set.

AU - Casey Jane Ankeny
AU - David P. O'Neill
AU - Lisa Beckmann
CY - Virtual On line
DA - 2020/06/22
PB - ASEE Conferences
TI - WIP: Comparison of a Standards-based Assessment to a Traditional, Summative Rubric in a Biomedical Engineering Laboratory
UR - https://peer.asee.org/35529
DO - 10.18260/1-2--35529
ER -