Using Sentiment Analysis to Evaluate First-year Engineering Students Teamwork Textual Feedback

Abdulrahman Alsharif; Andrew Katz; David Knight; Saleh Alatwah

Download Paper | Permalink

Conference: 2022 ASEE Annual Conference & Exposition
Location: Minneapolis, MN
Publication Date: August 23, 2022
Start Date: June 26, 2022
End Date: June 29, 2022
Conference Session: First-Year Programs Division Technical Session 7: Teamwork, Reflection, and Wellness
Page Count: 20
DOI: 10.18260/1-2--41460
Permanent URL: https://peer.asee.org/41460
Download Count: 454

Request a correction

Abstract

Abstract Sentiment analysis (SA) is used in multiple disciplines to evaluate textual data and has become a popular topic in educational research, with a growing body of published work. SA has been employed in educational research to investigate student satisfaction, attitudes, topics of concern, or evaluating instructors' teaching performance. However, there has been little discussion of applying SA as an assessment approach to evaluate teamwork textual feedback (i.e., students rate their teammates by writing comments on them) in engineering. The purpose of this research is to investigate the possibility of using SA as a method for evaluating collaborative textual feedback (e.g., comments) from students and to assist teachers in evaluating teamwork dynamics in their classrooms.

Teamwork is a key skill in engineering. With the rising complexity and magnitude of the challenges engineers handle, teamwork has become increasingly important. This is reflected in the Accreditation Board for Engineering and Technology accreditation student outcome criteria 3.5, which specifically highlights an ability to effectively function on teams. The importance of teamwork is also reflected in research showing that when organizations are trying to bring fresh talent into their firm, potential industry employers search for people who can effectively work in engineering teams. Engineering education literature further demonstrates the importance and the responsibility of faculty involvement in the development of effective teamwork. To assess teamwork functionality, instructors can distribute a survey among teams for team members to provide feedback about each other. This kind of feedback is helpful not only for that specific team and class but also for identifying broader, systematic trends in engineering student teams. Often the textual feedback is gathered at the end of a semester, and evaluating these responses can identify useful insights for improving teaching approaches. Unfortunately, in many cases, such surveys also can go underutilized. Large amounts of textual data often are not compatible with traditional analytic methodologies, but we claim that these huge amounts of textual data have the potential to deliver unique insights to educators and researchers.

We investigate SA as a potential method for analyzing a large corpus of student feedback responses about their team members and test this concept using a sample from 53,088 student responses from a first-year engineering course. The purpose of this research is to propose SA as an assessment approach to evaluate students’ teamwork comments. According to research, potential issues that first-year engineering students frequently face when working in teams include teammates not performing their share of their work, tardy teammates, domineering teammates, and some team members being excluded from major tasks. SA has the potential of identifying team members' biases, and it can provide quick feedback and provide real-time insights on the teamwork environment (e.g., positive, neutral, or negative environment). Research shows that conflicts between team members are common, and constructive feedback is critical in the development of students both as individuals and teammates. Furthermore, insights from SA have the ability to augment instructor evaluations of teamwork. By analyzing survey comments based on the comment writer and the individual about whom the comment was made.

We provide descriptive statistics about the proportions of sentiments conveyed in students’ comments (e.g., positive, neutral, or negative comments) split by demographic characteristics (e.g., gender, race, and nationality) of both the student making the comment and the student about whom the comment was made. Our findings suggest that sentiment analysis can assist instructors in comprehending collaborative dynamics more quickly, providing them with a better sense of how relationships are developing in their classroom teams. SA can also be used to identify positive, neutral, and negative words used by students, allowing the teacher to take measures based on the feedback and assist students to work in a more productive learning environment. For future work, we recommend that SA can be employed to evaluate other teamwork settings in engineering classrooms (e.g., design courses) and could be used to explore long textual feedback (e.g., journal reflections).

Citation
Format

Alsharif, A., & Katz, A., & Knight, D., & Alatwah, S. (2022, August), Using Sentiment Analysis to Evaluate First-year Engineering Students Teamwork Textual Feedback Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. 10.18260/1-2--41460

TY - CPAPER
AB - Abstract
Sentiment analysis (SA) is used in multiple disciplines to evaluate textual data and has become a popular topic in educational research, with a growing body of published work. SA has been employed in educational research to investigate student satisfaction, attitudes, topics of concern, or evaluating instructors' teaching performance. However, there has been little discussion of applying SA as an assessment approach to evaluate teamwork textual feedback (i.e., students rate their teammates by writing comments on them) in engineering. The purpose of this research is to investigate the possibility of using SA as a method for evaluating collaborative textual feedback (e.g., comments) from students and to assist teachers in evaluating teamwork dynamics in their classrooms.

Teamwork is a key skill in engineering. With the rising complexity and magnitude of the challenges engineers handle, teamwork has become increasingly important. This is reflected in the Accreditation Board for Engineering and Technology accreditation student outcome criteria 3.5, which specifically highlights an ability to effectively function on teams. The importance of teamwork is also reflected in research showing that when organizations are trying to bring fresh talent into their firm, potential industry employers search for people who can effectively work in engineering teams. Engineering education literature further demonstrates the importance and the responsibility of faculty involvement in the development of effective teamwork. To assess teamwork functionality, instructors can distribute a survey among teams for team members to provide feedback about each other. This kind of feedback is helpful not only for that specific team and class but also for identifying broader, systematic trends in engineering student teams. Often the textual feedback is gathered at the end of a semester, and evaluating these responses can identify useful insights for improving teaching approaches. Unfortunately, in many cases, such surveys also can go underutilized. Large amounts of textual data often are not compatible with traditional analytic methodologies, but we claim that these huge amounts of textual data have the potential to deliver unique insights to educators and researchers.

We investigate SA as a potential method for analyzing a large corpus of student feedback responses about their team members and test this concept using a sample from 53,088 student responses from a first-year engineering course. The purpose of this research is to propose SA as an assessment approach to evaluate students’ teamwork comments. According to research, potential issues that first-year engineering students frequently face when working in teams include teammates not performing their share of their work, tardy teammates, domineering teammates, and some team members being excluded from major tasks. SA has the potential of identifying team members' biases, and it can provide quick feedback and provide real-time insights on the teamwork environment (e.g., positive, neutral, or negative environment). Research shows that conflicts between team members are common, and constructive feedback is critical in the development of students both as individuals and teammates. Furthermore, insights from SA have the ability to augment instructor evaluations of teamwork. By analyzing survey comments based on the comment writer and the individual about whom the comment was made.

We provide descriptive statistics about the proportions of sentiments conveyed in students’ comments (e.g., positive, neutral, or negative comments) split by demographic characteristics (e.g., gender, race, and nationality) of both the student making the comment and the student about whom the comment was made. Our findings suggest that sentiment analysis can assist instructors in comprehending collaborative dynamics more quickly, providing them with a better sense of how relationships are developing in their classroom teams. SA can also be used to identify positive, neutral, and negative words used by students, allowing the teacher to take measures based on the feedback and assist students to work in a more productive learning environment. For future work, we recommend that SA can be employed to evaluate other teamwork settings in engineering classrooms (e.g., design courses) and could be used to explore long textual feedback (e.g., journal reflections).

AU - Abdulrahman Alsharif
AU - Andrew Katz
AU - David Knight
AU - Saleh Alatwah
CY - Minneapolis, MN
DA - 2022/08/23
PB - ASEE Conferences
TI - Using Sentiment Analysis to Evaluate First-year Engineering Students Teamwork Textual Feedback
UR - https://peer.asee.org/41460
DO - 10.18260/1-2--41460
ER -

Using Sentiment Analysis to Evaluate First-year Engineering Students Teamwork Textual Feedback

Paper Authors

Abdulrahman Alsharif Virginia Polytechnic Institute and State University

Andrew Katz Virginia Polytechnic Institute and State University

David Knight Virginia Polytechnic Institute and State University

Saleh Alatwah

Abstract

Citation

APA

APA - LaTeX bibitem

MLA

MLA - LaTeX bibitem

Bibtex

EndNote - RIS