Multiple Perspectives on Assessing Student Team Dynamics Using CATME in a First-year Engineering Course

Rui Li

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: First-Year Programs Division Technical Session 3: Teamwork
Tagged Division: First-Year Programs Division (FYP)
Permanent URL: https://peer.asee.org/47788

Request a correction

Paper Authors

biography

Rui Li New York University

visit author page

Dr. Li earned his master's degree in Chemical Engineering in 2009 from the Imperial College of London and his doctoral degree in 2020 from the University of Georgia, College of Engineering.

visit author page

Download Paper | Permalink

Abstract

The importance of teamwork is undeniable in industries relating to new product development, and engineering professionals almost always collaborate in project teams composed of people from various, complementary engineering backgrounds. As directed by the project manager or team leader, the project team is responsible for completing required tasks and for producing required deliverables. Due to the multidisciplinary of the engineering profession, professionals from various engineering disciplines must collaborate with each other to develop a good quality product with the highest efficiency. [a] Team collaboration is based on individual participation, to meet a specific goal. It is important to use tools that would be effective in producing successful results. It is therefore important to monitor and maximize team dynamics. The web-based program. Comprehensive Assessment of Team Member Effectiveness (CATME), developed by Purdue University, allows for peer-to-peer evaluations. This tool teaches students to give constructive and effective feedback to each other and to help them work together to achieve a successful engineering design project. By using CATME, the students are introduced to collaboratively working together and to help evaluate or critique each other. Meanwhile, the course instructors are available to provide guidance and to review students’ comments.

In a large private university, 600 students take first-year courses annually. It is time-consuming for instructors to analyze over 3000 peer ratings and comments. Given that the student teams would require fast response from the instructor, it is necessary to introduce a new framework: project mentorship, additional evaluation dimensions, and confidential statements to the instructors.

One part of the framework is to introduce the assistant roles into the CATME evaluation process, i.e. the project mentor. The responsibilities of project mentors are to provide technical support as well as set the benchmark requirements. The project mentors would be able to access the team performance together with the instructor and provide their perspectives on dealing with team issues. In this way, the mentors would speed up the instructor’s workflow.

It was found that additional evaluation dimensions allow the users to gain more insights into the team dynamics. The three new dimensions were psychological safety, team interdependence, and team satisfaction. One dimension of psychological safety was applied in Peer Evaluation I. As it is the early and storming stage of team formation, it is critical to check if students feel comfortable within the team. Another dimension of team independence was applied in Peer Evaluation II to evaluate the team synergy, then the dimension of the team satisfaction was placed in Peer Evaluation III to analyze the overall teamwork experience.

During the peer evaluations, some of the students prefer to directly message in fear of escalated interpersonal conflicts. CATME has a feature named confidential statements, which allows students to share additional thoughts or suggestions. The instructor could have more insights into the team issues such as potential cliques formed within the team. This new framework was implemented in the summer semester of 2023. The team peer ratings and evaluations will be compared with last year’s data. It is worth noting that the student teams generally have more positive feedback on the teamwork experience after implementing the new framework.

Citation
Format

Li, R. (2024, June), Multiple Perspectives on Assessing Student Team Dynamics Using CATME in a First-year Engineering Course Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/47788

TY - CPAPER
AB -
The importance of teamwork is undeniable in industries relating to new product development, and engineering professionals almost always collaborate in project teams composed of people from various, complementary engineering backgrounds. As directed by the project manager or team leader, the project team is responsible for completing required tasks and for producing required deliverables. Due to the multidisciplinary of the engineering profession, professionals from various engineering disciplines must collaborate with each other to develop a good quality product with the highest efficiency.
[a]
Team collaboration is based on individual participation, to meet a specific goal. It is important to use tools that would be effective in producing successful results. It is therefore important to monitor and maximize team dynamics. The web-based program. Comprehensive Assessment of Team Member Effectiveness (CATME), developed by Purdue University, allows for peer-to-peer evaluations. This tool teaches students to give constructive and effective feedback to each other and to help them work together to achieve a successful engineering design project. By using CATME, the students are introduced to collaboratively working together and to help evaluate or critique each other. Meanwhile, the course instructors are available to provide guidance and to review students’ comments.

In a large private university, 600 students take first-year courses annually. It is time-consuming for instructors to analyze over 3000 peer ratings and comments. Given that the student teams would require fast response from the instructor, it is necessary to introduce a new framework: project mentorship, additional evaluation dimensions, and confidential statements to the instructors.

One part of the framework is to introduce the assistant roles into the CATME evaluation process, i.e. the project mentor. The responsibilities of project mentors are to provide technical support as well as set the benchmark requirements. The project mentors would be able to access the team performance together with the instructor and provide their perspectives on dealing with team issues. In this way, the mentors would speed up the instructor’s workflow.

It was found that additional evaluation dimensions allow the users to gain more insights into the team dynamics. The three new dimensions were psychological safety, team interdependence, and team satisfaction. One dimension of psychological safety was applied in Peer Evaluation I. As it is the early and storming stage of team formation, it is critical to check if students feel comfortable within the team. Another dimension of team independence was applied in Peer Evaluation II to evaluate the team synergy, then the dimension of the team satisfaction was placed in Peer Evaluation III to analyze the overall teamwork experience.

During the peer evaluations, some of the students prefer to directly message in fear of escalated interpersonal conflicts. CATME has a feature named confidential statements, which allows students to share additional thoughts or suggestions. The instructor could have more insights into the team issues such as potential cliques formed within the team. This new framework was implemented in the summer semester of 2023. The team peer ratings and evaluations will be compared with last year’s data. It is worth noting that the student teams generally have more positive feedback on the teamwork experience after implementing the new framework.

AU - Rui Li
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Multiple Perspectives on Assessing Student Team Dynamics Using CATME in a First-year Engineering Course
UR - https://peer.asee.org/47788
ER -