Chasing assessment: The faculty experience of trying to implement evidence based practices well

Todd M. Fernandez; David Edward Torello

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: MECH - Technical Session 14: Advanced Pedagogical Techniques
Tagged Division: Mechanical Engineering Division (MECH)
Tagged Topic: Diversity
Permanent URL: https://peer.asee.org/48456

Request a correction

Paper Authors

biography

Todd M. Fernandez Georgia Institute of Technology

visit author page

Todd is a lecturer in the Wallace H. Coulter Department of Biomedical Engineering at Georgia Institute of Technology. His research interests are engineering students beliefs about knowledge and education and how those beliefs interact with learning engineering.

visit author page

biography

David Edward Torello Georgia Institute of Technology orcid.org/0000-0001-8565-5038

visit author page

Dr. David Torello graduated with his B.S. in mechanical engineering from UC Berkeley and his M.S. and Ph.D. from the Georgia Institute of Technology. He is currently non-tenure track faculty in mechanical engineering at Georgia Tech, lecturing in mechanic

visit author page

Download Paper | Permalink

Abstract

Engineering faculty are regularly encouraged to improve their courses as part of improving engineering education. Improvement efforts are often measured through an increase in the use of evidence-based practices. While inarguably important, achieving effective and high quality implementations of evidence based practices is not automatic or inherent in adopting their underlying principles. Oftentimes, it requires an iterative process of learning design - but that process is far less visible than implementation metrics. As both practitioners and researchers, we (the authors) see the process, tools, and even the existence of iterative implementations of evidence-based practices as useful to others. While there is a body of work on how to train, resource, or support faculty in changing their course, far less has focused on the experience faculty have in doing so. What happens when a change does not work as anticipated? What is the faculty experience of engaging in iterative changes to create robust implementations of evidence-based practices? How are evidence-based practices different once the novelty of a new change fades?

Our paper seeks to make visible the challenge, process, and results of iterating on initial implementations of evidence based practices. The motivation for this project was the instructor’s (i.e., lead author’s) dis-satisfaction with grading in a middle years rigid body dynamics course. He had implemented a new grading system two years prior that shifted from points to specifications-based grading. While happy with the switch to specifications grading, details of the initial implementation were resulting in untenable workloads and frustration for both students and the instructional team. That ongoing observation led to a process of observing and revising his initial implementation.

The full paper will present an autoethnographic study of the instructor’s experience and reasoning with revising his implementation of specifications grading. The study will use data from an extended ethnographic interview of the lead author (by the second author) as well as artifacts from the various iterations of the course grading scheme. We will focus on a set of longitudinal experiences that the faculty member undergoes after an initial large change in a course is made. Our hope is that the results will be useful for both educators and researchers. We expect that educators will identify with our results in ways that allow them to articulate and make sense of the curricular change as an iterative process, as opposed to a one time event that works or does not. We expect that researchers will find value in documentation of the experiences of faculty who live with and maintain course changes in the long term. Better understanding that experience will aid in developing resources to support faculty in refining and sustaining major changes to their courses, improving the outcomes of change efforts.

Citation
Format

Fernandez, T. M., & Torello, D. E. (2024, June), Chasing assessment: The faculty experience of trying to implement evidence based practices well Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/48456

TY - CPAPER
AB - Engineering faculty are regularly encouraged to improve their courses as part of improving engineering education. Improvement efforts are often measured through an increase in the use of evidence-based practices. While inarguably important, achieving effective and high quality implementations of evidence based practices is not automatic or inherent in adopting their underlying principles. Oftentimes, it requires an iterative process of learning design - but that process is far less visible than implementation metrics. As both practitioners and researchers, we (the authors) see the process, tools, and even the existence of iterative implementations of evidence-based practices as useful to others. While there is a body of work on how to train, resource, or support faculty in changing their course, far less has focused on the experience faculty have in doing so. What happens when a change does not work as anticipated? What is the faculty experience of engaging in iterative changes to create robust implementations of evidence-based practices? How are evidence-based practices different once the novelty of a new change fades?

Our paper seeks to make visible the challenge, process, and results of iterating on initial implementations of evidence based practices. The motivation for this project was the instructor’s (i.e., lead author’s) dis-satisfaction with grading in a middle years rigid body dynamics course. He had implemented a new grading system two years prior that shifted from points to specifications-based grading. While happy with the switch to specifications grading, details of the initial implementation were resulting in untenable workloads and frustration for both students and the instructional team. That ongoing observation led to a process of observing and revising his initial implementation.

The full paper will present an autoethnographic study of the instructor’s experience and reasoning with revising his implementation of specifications grading. The study will use data from an extended ethnographic interview of the lead author (by the second author) as well as artifacts from the various iterations of the course grading scheme. We will focus on a set of longitudinal experiences that the faculty member undergoes after an initial large change in a course is made. Our hope is that the results will be useful for both educators and researchers. We expect that educators will identify with our results in ways that allow them to articulate and make sense of the curricular change as an iterative process, as opposed to a one time event that works or does not. We expect that researchers will find value in documentation of the experiences of faculty who live with and maintain course changes in the long term. Better understanding that experience will aid in developing resources to support faculty in refining and sustaining major changes to their courses, improving the outcomes of change efforts.

AU - Todd M. Fernandez
AU - David Edward Torello
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Chasing assessment: The faculty experience of trying to implement evidence based practices well
UR - https://peer.asee.org/48456
ER -