Developing a Grounded Framework for Implementing Ungrading in a Disciplinary Context

Sarah Marie Coppola; Jennifer A. Turns

Download Paper | Permalink

Conference: 2023 ASEE Annual Conference & Exposition
Location: Baltimore , Maryland
Publication Date: June 25, 2023
Start Date: June 25, 2023
End Date: June 28, 2023
Conference Session: Faculty Development Division (FDD) Technical Session 8
Tagged Division: Faculty Development Division (FDD)
Tagged Topic: Diversity
Page Count: 13
DOI: 10.18260/1-2--43037
Permanent URL: https://peer.asee.org/43037
Download Count: 236

Paper Authors

biography

Sarah Marie Coppola University of Washington

visit author page

Sarah Coppola is an Assistant Teaching Professor the Department of Human Centered Design & Engineering at the University of Washington. Dr. Coppola is an educator and researcher whose work focuses on how technology and systems design affects people’s performance and health.

Coppola's research explores bias in technology and how to measure and quantify its impact. She has studied sex/gender differences caused by interface design, healthcare sociotechnical systems, and inclusive pedagogy.

visit author page

biography

Jennifer A. Turns University of Washington

visit author page

Dr. Jennifer Turns is a full professor in the Human Centered Design & Engineering Department in the College of Engineering at the University of Washington. Engineering education is her primary area of scholarship, and has been throughout her career. In her work, she currently focuses on the role of reflection in engineering student learning and the relationship of research and practice in engineering education. In recent years, she has been the co-director of the Consortium to Promote Reflection in Engineering Education (CPREE, funded by the Helmsley Charitable Trust), a member of the governing board for the International Research in Engineering Education Network, and an Associate Editor for the Journal of Engineering Education. Dr. Turns has published over 175 journal and conference papers on topics related to engineering education.

visit author page

Download Paper | Permalink

Abstract

Recent scholarship on grading suggests that grading can be a site of pedagogical innovation (e.g., Feldman, 2018), and ungrading is an example of such an innovation (Kohn and Blum, 2020). According to the Lafayette Center for Teaching and Learning, ungrading can be understood as “an umbrella term for any assessment that decenters the action of an instructor assigning a summary grade to student work.” A full program of research around ungrading would include research on how students experience ungrading, the way ungrading operates in different disciplinary contexts, how ungrading can support DEI goals, and the kinds of risks ungrading might involve. Because these questions all live on top of implementations of ungrading, and because implementing ungrading is non-negligible, we see an opportunity for research that can guide implementation of ungrading and thus enable other scholarship. In this work, we ask: what do practicing educators vary in order to adapt ungrading to their unique educational contexts. This question is similar to questions like how do practicing educators go about configuring ungrading to their unique contexts or what do naturally occurring implementations of ungrading reveal about the design space known. We ask the question as we do in order to situate our work as a research through design effort, specifically the approach advocated by Gaver (2012) in which a set of design solutions are interrogated to determine their invariances as well as the dimensions of variation. In framing our effort as research through design, we see educators as designers and their work of implementation as a form of design. We further highlight that our question about practicing educators and their unique educational contexts takes advantage of a problem solving orientation of educators to adapt their pedagogical choices when problems arise and the naturally occurring variation in their pedagogical contexts. We will focus on practicing educators in one disciplinary context who have recently used ungrading in a class assigned to them. Our process consists of two phases of data collection and analysis. In the first phase, participating educators are asked to provide a modest (500 word), unstructured, narrative description of how they went about ungrading. We will analyze these narratives inductively. We will start by open coding for invariances (what everyone did) and as well as places where educators took different paths, and then by organizing these open codes into a parsimonious set of dimensions of variation. For the second phase, we will start by organizing the narrative information about each instance of ungrading along the identified dimensions of variation and then invite each educator to additionally elaborate their instance along each dimension. We will then analyze the descriptions for each dimension in order to better understand the choices and the considerations associated with each dimension. The result will be a framework consisting of the dimensions of variation and a deep description of each dimension. The contribution of our work will be an empirically derived, grounded framework capturing key issues in how ungrading may be implemented. This framework is grounded in the sense that it is based on naturalistic instances of ungrading followed by an inductive process to determine what goes in the framework. Our framework contribution will position several communities to proceed. Educators may use this framework to guide their decision making as they go about implementing ungrading. Researchers studying questions about effectiveness, experiences, benefits and mechanisms of ungrading will be able to use the framework to be precise about the instantiations of ungrading with which they are working. Further, the framework may inspire design of new supports for ungrading and identification of new research questions. Thus, we see our work as an enabling contribution.

Gaver, W. (2012). What should we expect from research through design?. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 937-946). Kohn, Alfie, and Susan D. Blum (2020). Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). West Virginia University Press.. Feldman, J. (2018). Grading for equity: What it is, why it matters, and how it can transform schools and classrooms. Corwin Press.

Citation
Format

Coppola, S. M., & Turns, J. A. (2023, June), Developing a Grounded Framework for Implementing Ungrading in a Disciplinary Context Paper presented at 2023 ASEE Annual Conference & Exposition, Baltimore , Maryland. 10.18260/1-2--43037

TY - CPAPER
AB - Recent scholarship on grading suggests that grading can be a site of pedagogical innovation (e.g., Feldman, 2018), and ungrading is an example of such an innovation (Kohn and Blum, 2020). According to the Lafayette Center for Teaching and Learning, ungrading can be understood as “an umbrella term for any assessment that decenters the action of an instructor assigning a summary grade to student work.” A full program of research around ungrading would include research on how students experience ungrading, the way ungrading operates in different disciplinary contexts, how ungrading can support DEI goals, and the kinds of risks ungrading might involve. Because these questions all live on top of implementations of ungrading, and because implementing ungrading is non-negligible, we see an opportunity for research that can guide implementation of ungrading and thus enable other scholarship.

In this work, we ask: what do practicing educators vary in order to adapt ungrading to their unique educational contexts. This question is similar to questions like how do practicing educators go about configuring ungrading to their unique contexts or what do naturally occurring implementations of ungrading reveal about the design space known. We ask the question as we do in order to situate our work as a research through design effort, specifically the approach advocated by Gaver (2012) in which a set of design solutions are interrogated to determine their invariances as well as the dimensions of variation. In framing our effort as research through design, we see educators as designers and their work of implementation as a form of design. We further highlight that our question about practicing educators and their unique educational contexts takes advantage of a problem solving orientation of educators to adapt their pedagogical choices when problems arise and the naturally occurring variation in their pedagogical contexts.

We will focus on practicing educators in one disciplinary context who have recently used ungrading in a class assigned to them. Our process consists of two phases of data collection and analysis. In the first phase, participating educators are asked to provide a modest (500 word), unstructured, narrative description of how they went about ungrading. We will analyze these narratives inductively. We will start by open coding for invariances (what everyone did) and as well as places where educators took different paths, and then by organizing these open codes into a parsimonious set of dimensions of variation. For the second phase, we will start by organizing the narrative information about each instance of ungrading along the identified dimensions of variation and then invite each educator to additionally elaborate their instance along each dimension. We will then analyze the descriptions for each dimension in order to better understand the choices and the considerations associated with each dimension. The result will be a framework consisting of the dimensions of variation and a deep description of each dimension.

The contribution of our work will be an empirically derived, grounded framework capturing key issues in how ungrading may be implemented. This framework is grounded in the sense that it is based on naturalistic instances of ungrading followed by an inductive process to determine what goes in the framework. Our framework contribution will position several communities to proceed. Educators may use this framework to guide their decision making as they go about implementing ungrading. Researchers studying questions about effectiveness, experiences, benefits and mechanisms of ungrading will be able to use the framework to be precise about the instantiations of ungrading with which they are working. Further, the framework may inspire design of new supports for ungrading and identification of new research questions. Thus, we see our work as an enabling contribution.

Gaver, W. (2012). What should we expect from research through design?. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 937-946).
Kohn, Alfie, and Susan D. Blum (2020). Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). West Virginia University Press..
Feldman, J. (2018). Grading for equity: What it is, why it matters, and how it can transform schools and classrooms. Corwin Press.

AU - Sarah Marie Coppola
AU - Jennifer A. Turns
CY - Baltimore , Maryland
DA - 2023/06/25
PB - ASEE Conferences
TI - Developing a Grounded Framework for Implementing Ungrading in a Disciplinary Context
UR - https://peer.asee.org/43037
DO - 10.18260/1-2--43037
ER -