Visualizing TDOP+ Classroom Observation Data using Dashboards

Allison Van Beek; Susan McCahan

Download Paper | Permalink

Conference: 2025 ASEE Annual Conference & Exposition
Location: Montreal, Quebec, Canada
Publication Date: June 22, 2025
Start Date: June 22, 2025
End Date: August 15, 2025
Conference Session: ERM Technical Session: Improving Assessment in Engineering Education
Tagged Division: Educational Research and Methods Division (ERM)
Page Count: 13
DOI: 10.18260/1-2--57333
Permanent URL: https://peer.asee.org/57333
Download Count: 6

Paper Authors

biography

Allison Van Beek University of Toronto

visit author page

Allison (she/her) is a PhD candidate in Engineering Education through the Department of Mechanical and Industrial Engineering. Her supervisor is Prof. Susan McCahan.

visit author page

biography

Susan McCahan University of Toronto

visit author page

Susan McCahan is a Professor in the Department of Mechanical and Industrial Engineering at the University of Toronto. She currently holds the positions of Vice-Provost, Innovations in Undergraduate Education and Associate Vice-President & Vice Provost, Digital Strategies.

visit author page

Download Paper | Permalink

Abstract

Classroom observation protocols are used to generate rich qualitative and quantitative data that illustrate many aspects of teaching practice and pedagogy. However, each observation can generate hundreds of rows of transactional data. Making sense of this volume of data is a challenge because there is no standardized mechanism to visualize this data in a way that would turn it into meaningful information. As a result, instructors and teaching teams who want to learn from observation protocols can struggle to incorporate observational data into their teaching practice. In addition, there is a missed opportunity to use information that could inform institutions about how classroom space (e.g., active learning space) is utilized. This type of aggregated observation data could be particularly useful to enhance instructional space design, as well as to support other systemic decision making.

We saw an opportunity to address this challenge by building descriptive activity dashboards that productively visualize large datasets, enabling users and administrators to identify patterns and draw conclusions. For this proof-of-concept, we use a large dataset generated from classroom observations gathered using the TDOP+ protocol. The TDOP+ was used to observe teaching behaviors across 10 STEM courses in 2020 and 4 in 2023. In each course, three class sessions were observed, and activity was categorized according to teaching dimension (e.g., instructional technology, pedagogical strategies, cognitive engagement, etc.).

Through the dashboard, we observe patterns of behaviour, including how instructors move through the space as well as how instructors take on new and expanded roles while teaching (e.g., acting as technical support during the class session). The dashboards also provide insight into how tools are used – and not used – during class sessions. We are looking at how and when tools both built-in (e.g., displays, table microphones), as well as brought-in by the teaching teams (e.g., laptops, tablets, handouts) are used to enable (or disable) different types of teaching activities.

While the dashboard will be made available to instructors, our project goal is to provide engineering education researchers, educational technologists, and classroom designers with a tool for examining classroom practices. By identifying patterns in tool use (e.g., abandonment, misuse), physical movement in the space, and the division of labour across different teaching team structures and courses, it is possible to gain a generalizable, holistic understanding of classroom orchestration. With this greater understanding, we aim to inform design decisions to enhance the use and utilization of active learning classrooms in engineering.

Citation
Format

Van Beek, A., & McCahan, S. (2025, June), Visualizing TDOP+ Classroom Observation Data using Dashboards Paper presented at 2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada . 10.18260/1-2--57333

TY - CPAPER
AB - Classroom observation protocols are used to generate rich qualitative and quantitative data that illustrate many aspects of teaching practice and pedagogy. However, each observation can generate hundreds of rows of transactional data. Making sense of this volume of data is a challenge because there is no standardized mechanism to visualize this data in a way that would turn it into meaningful information. As a result, instructors and teaching teams who want to learn from observation protocols can struggle to incorporate observational data into their teaching practice. In addition, there is a missed opportunity to use information that could inform institutions about how classroom space (e.g., active learning space) is utilized. This type of aggregated observation data could be particularly useful to enhance instructional space design, as well as to support other systemic decision making.

We saw an opportunity to address this challenge by building descriptive activity dashboards that productively visualize large datasets, enabling users and administrators to identify patterns and draw conclusions. For this proof-of-concept, we use a large dataset generated from classroom observations gathered using the TDOP+ protocol. The TDOP+ was used to observe teaching behaviors across 10 STEM courses in 2020 and 4 in 2023. In each course, three class sessions were observed, and activity was categorized according to teaching dimension (e.g., instructional technology, pedagogical strategies, cognitive engagement, etc.).

Through the dashboard, we observe patterns of behaviour, including how instructors move through the space as well as how instructors take on new and expanded roles while teaching (e.g., acting as technical support during the class session). The dashboards also provide insight into how tools are used – and not used – during class sessions. We are looking at how and when tools both built-in (e.g., displays, table microphones), as well as brought-in by the teaching teams (e.g., laptops, tablets, handouts) are used to enable (or disable) different types of teaching activities.

While the dashboard will be made available to instructors, our project goal is to provide engineering education researchers, educational technologists, and classroom designers with a tool for examining classroom practices. By identifying patterns in tool use (e.g., abandonment, misuse), physical movement in the space, and the division of labour across different teaching team structures and courses, it is possible to gain a generalizable, holistic understanding of classroom orchestration. With this greater understanding, we aim to inform design decisions to enhance the use and utilization of active learning classrooms in engineering.
AU - Allison Van Beek
AU - Susan McCahan
CY - Montreal, Quebec, Canada
DA - 2025/06/22
PB - ASEE Conferences
TI - Visualizing TDOP+ Classroom Observation Data using Dashboards
UR - https://peer.asee.org/57333
DO - 10.18260/1-2--57333
ER -