Work in Progress: Investigating the Effectiveness of an Orchestration Tool on the Nature of Students’ Collaborative Interactions During Group Work

Taylor Tucker; LuEttaMae Lawrence; Emma Mercier

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: Studies of Student Teams and Student Interactions
Tagged Division: Educational Research and Methods
Page Count: 7
DOI: 10.18260/1-2--38174
Permanent URL: https://peer.asee.org/38174
Download Count: 310

Paper Authors

biography

Taylor Tucker University of Illinois at Urbana - Champaign

visit author page

Taylor Tucker graduated from the University of Illinois at Urbana-Champaign with a Bachelor’s degree in engineering mechanics. She is now pursuing a master’s degree at UIUC through the Digital Environments for Learning, Teaching, and Agency program in the department of Curriculum and Instruction. She is interested in design thinking as it applies to engineering settings and lends her technical background to her research with the Collaborative Learning Lab, exploring ways to to promote collaborative problem solving in engineering education and provide students with team design experiences that mimic authentic work in industry.

visit author page

biography

LuEttaMae Lawrence Carnegie Mellon University

visit author page

LuEttaMae Lawrence is a Postdoc Fellow at Carnegie Mellon at the Human-Computer Interaction Institute. She received her PhD in Curriculum and Instruction from the University of Illinois at Urbana-Champaign and her BFA in Graphic Design from Iowa State University. As a learning scientist and design researcher, Lu studies co-design processes to build educational technology and investigates how designs are embedded in authentic learning contexts. To do this work, she integrates methods from design, human-computer interaction, and education to understand how collaborative discourse and learning occur.

visit author page

biography

Emma Mercier

visit author page

Emma Mercier is an associate professor in Curriculum and Instruction at the University of Illinois Urbana-Champaign. Her work focuses on collaborative learning in classrooms, and in particular, the use of technology for teachers and students during collaborative learning. Most recently Mercier's projects have focused on collaborative learning in required undergraduate engineering courses.

visit author page

Download Paper | Permalink

Abstract

WIP: Investigating the Effectiveness of an Orchestration Tool on the Nature of Students’ Collaborative Interactions During Group Work

The role of the instructor is characterized by different pedagogies as both “guide on the side” and “sage on the stage.” In the context of collaborative problem solving (CPS), a form of pedagogy that has become increasingly common in engineering courses, it is necessary for instructors to reactively provide guidance for students’ conceptual understanding while simultaneously supporting their interactions, rather than solely providing content-focused instruction at their chosen pace. This can be done through many means, such as classroom affordances, task structures, and face-to-face instructor interventions and feedback. However, supporting CPS remains challenging, especially for novice instructors (e.g. teaching assistants; TAs) who often teach introductory engineering courses. This work-in-progress study investigates the effectiveness of instructor interventions by evaluating the influence of an orchestration tool deployed while students collaboratively solved ill-structured engineering design tasks in real time.

Orchestration technology is developed to support instructors in managing real-time factors within a classroom. Researchers have developed supportive tools that help teachers identify crucial learning moments and provide actionable information. In order to better support TAs, who do not always have the resources or training to facilitate CPS, we developed an orchestration tool that provided strategies to strengthen TAs’ interventions with groups and support students’ collaborative interactions in real time. We deployed our tool in collaborative discussion sections that were part of a required undergraduate engineering course; these sections were taught by graduate and undergraduate TAs who had little or no experience teaching CPS. In this work, we explore TAs’ interactions with the orchestration tool and examine ways in which 1) the tool impacted instructor interventions and 2) instructors interventions impacted students’ group work.

This study evaluates the results of implementing our tool in three discussion sections held over two weeks. Across these sections, a total of 26 groups of three to four undergraduate engineering students worked to solve collaborative, ill-structured tasks. The tool was provided to all TAs as a resource that enabled them to identify groups needing collaborative support while providing suggestions for how to attend these needs; both interventions that were and were not guided by the tool were analyzed to understand its effectiveness. Our initial findings indicate that when TAs used the tool, they were more likely to monitor a group before intervening and to initiate the intervention themselves, suggesting that the tool effectively directed TAs’ attention toward groups. In contrast, without the tool, instructors were more likely to end their intervention by checking the groups’ understanding. Our continued analysis will seek to understand how these factors affect groups’ collaborative interactions. Our findings respond to calls to implement CPS in engineering courses by understanding if and how orchestration technology aided TAs in facilitating this form of pedagogy. By investigating how an orchestration tool supported TAs and groups of students, our analysis will provide recommendations for how similar tools can be developed to further promote this form of learning.

Citation
Format

Tucker, T., & Lawrence, L., & Mercier, E. (2021, July), Work in Progress: Investigating the Effectiveness of an Orchestration Tool on the Nature of Students’ Collaborative Interactions During Group Work Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--38174

TY  - CPAPER
AB  - WIP: Investigating the Effectiveness of an Orchestration Tool on the Nature of Students’ Collaborative Interactions During Group Work

The role of the instructor is characterized by different pedagogies as both “guide on the side” and “sage on the stage.” In the context of collaborative problem solving (CPS), a form of pedagogy that has become increasingly common in engineering courses, it is necessary for instructors to reactively provide guidance for students’ conceptual understanding while simultaneously supporting their interactions, rather than solely providing content-focused instruction at their chosen pace. This can be done through many means, such as classroom affordances, task structures, and face-to-face instructor interventions and feedback. However, supporting CPS remains challenging, especially for novice instructors (e.g. teaching assistants; TAs) who often teach introductory engineering courses. This work-in-progress study investigates the effectiveness of instructor interventions by evaluating the influence of an orchestration tool deployed while students collaboratively solved ill-structured engineering design tasks in real time.

Orchestration technology is developed to support instructors in managing real-time factors within a classroom. Researchers have developed supportive tools that help teachers identify crucial learning moments and provide actionable information. In order to better support TAs, who do not always have the resources or training to facilitate CPS, we developed an orchestration tool that provided strategies to strengthen TAs’ interventions with groups and support students’ collaborative interactions in real time. We deployed our tool in collaborative discussion sections that were part of a required undergraduate engineering course; these sections were taught by graduate and undergraduate TAs who had little or no experience teaching CPS. In this work, we explore TAs’ interactions with the orchestration tool and examine ways in which 1) the tool impacted instructor interventions and 2) instructors interventions impacted students’ group work. 

This study evaluates the results of implementing our tool in three discussion sections held over two weeks. Across these sections, a total of 26 groups of three to four undergraduate engineering students worked to solve collaborative, ill-structured tasks. The tool was provided to all TAs as a resource that enabled them to identify groups needing collaborative support while providing suggestions for how to attend these needs; both interventions that were and were not guided by the tool were analyzed to understand its effectiveness. Our initial findings indicate that when TAs used the tool, they were more likely to monitor a group before intervening and to initiate the intervention themselves, suggesting that the tool effectively directed TAs’ attention toward groups. In contrast, without the tool, instructors were more likely to end their intervention by checking the groups’ understanding. Our continued analysis will seek to understand how these factors affect groups’ collaborative interactions. Our findings respond to calls to implement CPS in engineering courses by understanding if and how orchestration technology aided TAs in facilitating this form of pedagogy. By investigating how an orchestration tool supported TAs and groups of students, our analysis will provide recommendations for how similar tools can be developed to further promote this form of learning.



AU  - Taylor Tucker
AU  - LuEttaMae Lawrence
AU  - Emma Mercier
CY  - Virtual Conference
DA  - 2021/07/26
PB  - ASEE Conferences
TI  - Work in Progress: Investigating the Effectiveness of an Orchestration Tool on the Nature of Students’ Collaborative Interactions During Group Work
UR  - https://peer.asee.org/38174
DO  - 10.18260/1-2--38174
ER  -