Alsharif, A., & Katz, A. (2024, June), From Manual Coding to Machine Understanding: Students' Feedback Analysis  Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--47481

\bibitem{asee_peer_47481}
  Alsharif, A., \& Katz, A. (2024, June), \emph{From Manual Coding to Machine Understanding: Students' Feedback Analysis}
  Paper presented at 2024 ASEE Annual Conference \& Exposition, Portland, Oregon.
  10.18260/1-2--47481

Abdulrahman Alsharif and Andrew Katz.  "From Manual Coding to Machine Understanding: Students' Feedback Analysis".  2024 ASEE Annual Conference & Exposition, Portland, Oregon, 2024, June.  ASEE Conferences, 2024.  https://peer.asee.org/47481 Internet.  12 Dec, 2024

\bibitem{asee_peer_47481}
  Abdulrahman Alsharif and Andrew Katz.
  "From Manual Coding to Machine Understanding: Students' Feedback Analysis".
  \emph{2024 ASEE Annual Conference \& Exposition, Portland, Oregon, 2024, June}.
  ASEE Conferences, 2024.
  https://peer.asee.org/47481 Internet.  12 Dec, 2024

@INPROCEEDINGS{asee_peer_47481
author = "Abdulrahman Alsharif and Andrew Katz"
title = "From Manual Coding to Machine Understanding: Students' Feedback Analysis"
booktitle = "2024 ASEE Annual Conference \& Exposition"
year = "2024"
month = "June"
address = "Portland, Oregon"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/47481}
number = {10.18260/1-2--47481}
}

TY  - CPAPER
AB  - In this study, we evaluate the use of generative AI (GAI) models for qualitative coding of open-ended student responses, compared to traditional natural language processing (NLP) methods. The main objective was to explore an in-house GAI method to develop themes from students’ feedback responses. A systematic four-step process of text extraction, embedding, clustering, and code generation was employed on responses from a large engineering course regarding the transition to online learning during COVID-19. A locally deployed GAI model (Dolphin-Mistral 2.6) was used for privacy-preserving text extraction, with the UAE-Angle embedding model enabling the clustering of similar responses. GAI was then leveraged to generate qualitative codes and themes from the clusters. Human evaluation (i.e., human in the loop process) found the GAI-generated codes displayed high similarity to human-generated codes, with minor terminology distinctions. Key themes emphasized the importance of instructor feedback, communication strategies, engagement approaches, and resource accessibility for effective online learning experiences. Treemap visualizations aided the interpretation of the hierarchical code structure. While human input was still required for consolidating overlapping sub-codes, the study demonstrates GAI's potential to semi-automate qualitative coding tasks traditionally performed manually, while ensuring data privacy through local deployments. Future work could explore more advanced GAI models to further streamline the clustering and code generation workflow.
AU  - Abdulrahman Alsharif
AU  - Andrew Katz
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - From Manual Coding to Machine Understanding: Students' Feedback Analysis
UR  - https://peer.asee.org/47481
DO  - 10.18260/1-2--47481
ER  -