Board 63: Work in progress: Uncovering engineering students’ sentiments from weekly reflections using natural language processing

Ahmed Ashraf Butt; Saira Anwar; Muhsin Menekse

Download Paper | Permalink

Conference: 2023 ASEE Annual Conference & Exposition
Location: Baltimore , Maryland
Publication Date: June 25, 2023
Start Date: June 25, 2023
End Date: June 28, 2023
Conference Session: Computers in Education Division (COED) Poster Session
Tagged Division: Computers in Education Division (COED)
Page Count: 6
DOI: 10.18260/1-2--43210
Permanent URL: https://peer.asee.org/43210
Download Count: 205

Paper Authors

author page

Ahmed Ashraf Butt Purdue University at West Lafayette (COE) orcid.org/0000-0003-2047-8493

biography

Saira Anwar Texas A&M University orcid.org/0000-0001-6947-3226

visit author page

Saira Anwar is an Assistant Professor at Department of Multidisciplinary Engineering, Texas A &M University. Dr. Anwar has over 13 years of teaching experience, primarily in the disciplines of engineering education, computer science and software engineering. Her research focuses on studying the unique contribution of different instructional strategies on students' learning and motivation. Also, she is interested in designing interventions that help in understanding conceptually hard concepts in STEM courses. Dr. Anwar is the recipient of the 2020 outstanding researcher award by the School of Engineering Education, Purdue University. Also, she was the recipient of the "President of Pakistan Merit and Talent Scholarship" for her undergraduate studies.

visit author page

biography

Muhsin Menekse Purdue University at West Lafayette (COE)

visit author page

Muhsin Menekse is an Assistant Professor at Purdue University with a joint appointment in the School of Engineering Education and the Department of Curriculum & Instruction. Dr. Menekse's primary research focus is on exploring K-16 students' engagement and learning of engineering and science concepts by creating innovative instructional resources and conducting interdisciplinary quasi-experimental research studies in and out of classroom environments. Dr. Menekse is the recipient of the 2014 William Elgin Wickenden Award by the American Society for Engineering Education. Dr. Menekse also received three Seed-for-Success Awards (in 2017, 2018, and 2019) from Purdue University's Excellence in Research Awards programs in recognition of obtaining three external grants of $1 million or more during each year. His research has been generously funded by grants from the Institute of Education Sciences (IES), Purdue Research Foundation (PRF), and National Science Foundation (NSF).

visit author page

Download Paper | Permalink

Abstract

Understanding students’ sentiments are crucial as they impact their motivation and regulation strategies. However, limited work has been devoted to understanding students’ sentiments with respect to their classroom experiences in STEM courses, particularly in courses with conceptually difficult concepts. In this regard, this work-in-progress study used Natural Language Processing (NLP) algorithm to analyze the sentiments of the engineering students’ written reflections and then understand the change in their sentiments during the semester. In this course, students were introduced to computer programming with MATLAB, which is considered a difficult undertaking for novice learners. Specifically, this study will be guided by two research questions: 1) What kind of student sentiments occur when students reflect on the interesting and confusing aspect of the lecture? and 2) How do students’ sentiments change over the semester? To inform the study, we gathered students’ written reflections using the CourseMIRROR application in the multiple sections of the introductory engineering course (N = 300 students). This application asked students to reflect on the interesting and confusing aspects of the lecture after the end of each lecture throughout the semester. For sentiment analysis, we will use the Valence Aware Dictionary for Sentiment Reasoning (VADER) to assess sentiments in students’ reflections by generating a normalized sentiment score ranging from + 1 (extreme positive) to -1 (extreme negative). Based on the sentiment analysis results, we will employ descriptive statistics to inform the first research question by counting the frequency of the sentiments found in the students’ reflections. For the second research question, we will use one-way repeated measure ANOVA by splitting the lectures (N=21) into three equal time points and seeing the change of sentiment scores across these time points. We hypothesize that the students would have positive sentiments associated with reflections on interesting aspects of the lecture and negative sentiments while reflecting on confusing aspects of the lecture. Due to the nature of the course (i.e., programming concepts become gradually complex), we expect to see declining positive sentiments over time. The findings of this study will provide insights into students’ sentiments and provide suggestions to improve students’ engagement in engineering courses.

Citation
Format

Butt, A. A., & Anwar, S., & Menekse, M. (2023, June), Board 63: Work in progress: Uncovering engineering students’ sentiments from weekly reflections using natural language processing Paper presented at 2023 ASEE Annual Conference & Exposition, Baltimore , Maryland. 10.18260/1-2--43210

TY  - CPAPER
AB  - Understanding students’ sentiments are crucial as they impact their motivation and regulation strategies. However, limited work has been devoted to understanding students’ sentiments with respect to their classroom experiences in STEM courses, particularly in courses with conceptually difficult concepts. In this regard, this work-in-progress study used Natural Language Processing (NLP) algorithm to analyze the sentiments of the engineering students’ written reflections and then understand the change in their sentiments during the semester. In this course, students were introduced to computer programming with MATLAB, which is considered a difficult undertaking for novice learners. Specifically, this study will be guided by two research questions: 1) What kind of student sentiments occur when students reflect on the interesting and confusing aspect of the lecture? and 2) How do students’ sentiments change over the semester? To inform the study, we gathered students’ written reflections using the CourseMIRROR application in the multiple sections of the introductory engineering course (N = 300 students). This application asked students to reflect on the interesting and confusing aspects of the lecture after the end of each lecture throughout the semester. For sentiment analysis, we will use the Valence Aware Dictionary for Sentiment Reasoning (VADER) to assess sentiments in students’ reflections by generating a normalized sentiment score ranging from + 1 (extreme positive) to -1 (extreme negative). Based on the sentiment analysis results, we will employ descriptive statistics to inform the first research question by counting the frequency of the sentiments found in the students’ reflections. For the second research question, we will use one-way repeated measure ANOVA by splitting the lectures (N=21) into three equal time points and seeing the change of sentiment scores across these time points. We hypothesize that the students would have positive sentiments associated with reflections on interesting aspects of the lecture and negative sentiments while reflecting on confusing aspects of the lecture. Due to the nature of the course (i.e., programming concepts become gradually complex), we expect to see declining positive sentiments over time. The findings of this study will provide insights into students’ sentiments and provide suggestions to improve students’ engagement in engineering courses. 
AU  - Ahmed Ashraf Butt
AU  - Saira Anwar
AU  - Muhsin Menekse
CY  - Baltimore , Maryland
DA  - 2023/06/25
PB  - ASEE Conferences
TI  - Board 63: Work in progress: Uncovering engineering students’ sentiments from weekly reflections using natural language processing
UR  - https://peer.asee.org/43210
DO  - 10.18260/1-2--43210
ER  -