Kardam, N., & Wilson, D., & Makhsous, S. (2024, June), A Comparative Analysis of Natural Language Processing Techniques for Analyzing Student Feedback about TA Support  Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--46425

\bibitem{asee_peer_46425}
  Kardam, N., \& Wilson, D., \& Makhsous, S. (2024, June), \emph{A Comparative Analysis of Natural Language Processing Techniques for Analyzing Student Feedback about TA Support}
  Paper presented at 2024 ASEE Annual Conference \& Exposition, Portland, Oregon.
  10.18260/1-2--46425

Neha Kardam, Denise Wilson, and Sep Makhsous.  "A Comparative Analysis of Natural Language Processing Techniques for Analyzing Student Feedback about TA Support".  2024 ASEE Annual Conference & Exposition, Portland, Oregon, 2024, June.  ASEE Conferences, 2024.  https://peer.asee.org/46425 Internet.  07 Jan, 2025

\bibitem{asee_peer_46425}
  Neha Kardam, Denise Wilson, and Sep Makhsous.
  "A Comparative Analysis of Natural Language Processing Techniques for Analyzing Student Feedback about TA Support".
  \emph{2024 ASEE Annual Conference \& Exposition, Portland, Oregon, 2024, June}.
  ASEE Conferences, 2024.
  https://peer.asee.org/46425 Internet.  07 Jan, 2025

@INPROCEEDINGS{asee_peer_46425
author = "Neha Kardam, Denise Wilson, and Sep Makhsous"
title = "A Comparative Analysis of Natural Language Processing Techniques for Analyzing Student Feedback about TA Support"
booktitle = "2024 ASEE Annual Conference \& Exposition"
year = "2024"
month = "June"
address = "Portland, Oregon"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/46425}
number = {10.18260/1-2--46425}
}

TY  - CPAPER
AB  - This paper advances the exploration of Natural Language Processing (NLP) for automated coding and analysis of short-answer, text-based data collected from student feedback. Specifically, it assesses student preferences for Teaching Assistant (TA) support in engineering courses at a large public research university. This work complements existing research with an in-depth comparative analysis of NLP approaches to examining qualitative data within the realm of engineering education, utilizing survey data (training set = 1359, test set = 341) collected from 2017 to 2022. The challenges and intricacies of multiple types of classification errors emerging from five NLP methods are highlighted: Latent Dirichlet Allocation (LDA), Non-Negative Matrix Factorization (NMF), BERTopic, Latent Semantic Analysis (LSA), and Principal Component Analysis (PCA). These results are compared with results from traditional thematic analysis conducted by a domain expert to assess their efficacy. Two principal findings emerged for TA teaching practice and for the use of NLP in education research. Firstly, the conclusions derived from each coding technique are consistent, demonstrating that students want, in order of priority, extensive TA-student interactions, problem-solving support, and experiential/laboratory learning support from TAs. Secondly, the research offers insights into the effectiveness of NMF and PCA in processing educational survey data. A comparison of NMF and PCA topic models, based on accuracy, precision, recall, and F1 scores, reveals that while PCA outperforms NMF in terms of precision (identifying truly relevant topics), NMF excels in recall (capturing a broader range of student responses). 
AU  - Neha Kardam
AU  - Denise Wilson
AU  - Sep Makhsous
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - A Comparative Analysis of Natural Language Processing Techniques for Analyzing Student Feedback about TA Support
UR  - https://peer.asee.org/46425
DO  - 10.18260/1-2--46425
ER  -