Work-in-Progress: Understanding learners' motivation through machine learning analysis on reflection writing

Elizabeth Pluskwik; Yuezhou Wang; Lauren Singelmann

Download Paper | Permalink

Conference: 2022 ASEE Annual Conference & Exposition
Location: Minneapolis, MN
Publication Date: August 23, 2022
Start Date: June 26, 2022
End Date: June 29, 2022
Conference Session: Multidisciplinary Engineering Division Technical Session - Machine Learning, IoT, Writing Center Peer Tutors, Conceptual Modeling
Page Count: 12
DOI: 10.18260/1-2--40487
Permanent URL: https://peer.asee.org/40487
Download Count: 261

Paper Authors

biography

Elizabeth Pluskwik Minnesota State University, Mankato

visit author page

Elizabeth facilitates project-based and co-op based engineering education in the Iron Range Engineering program, Minnesota State University, Mankato. Her specialties leading entrepreneurial mindset in engineering, engineering management, accounting, product-costing, and lean six sigma. Her research interests include motivation to persist in engineering, emotional intelligence, and industry 4.0.

visit author page

biography

Yuezhou Wang Minnesota State University, Mankato

visit author page

Dr. Yuezhou Wang is an associate professor at Minnesota State University, Mankato. After receiving his Ph.D. from University of Minnesota in 2017, he works for the Iron Range Engineering, a project-based learning program. His teaching interests are in areas of materials science, structural analysis, finite element modeling and dynamic systems. His technical research focuses on multiscale modeling on mechanical behavior of nano and granular materials.

visit author page

biography

Lauren Singelmann North Dakota State University

visit author page

Lauren Singelmann earned her Ph.D. from North Dakota State University in Electrical and Computer Engineering and STEM Education in 2022. She is a faculty member for Iron Range Engineering through Minnesota State University, Mankato, and she supports instruction of Innovation-Based Learning courses at multiple institutions. Her research interests include learning analytics, experiential learning, and equitable grading and assessment.

visit author page

Download Paper | Permalink

Abstract

Educational data mining (EDM) is an emerging interdisciplinary field that utilizes a machine learning (ML) algorithm to collect and analyze educational data, aiming to better predict students' performance and retention. In this WIP paper, we report our methodology and preliminary results from utilizing a ML program to assess students’ motivation through their upper-division years in the XYZ project-based learning (PBL) program. ML, or more specifically, the clustering algorithm, opens the door to processing large amounts of student-written artifacts, such as reflection journals, project reports, and written assignments, and then identifies keywords that signal their levels of motivation (i.e., extrinsic vs. intrinsic). These results will be compared against other measures of motivation, including student self-report, faculty observation, and externally validated surveys. As part of a longer-term study, this pilot work sheds light on the key question for student success and retention: how does student motivation evolve through the 3rd and 4th years in college?

The purpose of this research project is to gain insights into learners’ motivation levels and how it evolves during the last two years in college, as well as to extend current Educational Data Mining research and Machine Learning analysis described in the literature. It is significant on two fronts: 1) we will extend the ability of ML in analyzing reflective written artifacts to explore student physiological and emotional development; 2) the longitudinal study will help monitor the progressive change of motivation in college students in a PBL environment.

Preliminary results from an initial pilot study are promising. By analyzing written reflection journal entries from previous students, the ML algorithm has differentiated keywords into three student motivation levels: “high”, “neutral” and “low”. Using supervised classes, for example, the ML algorithm differentiated words in the highly motivated student text such as “team” and “learning”, while the text coded as low motivation included “use”, “pushed” and “nothing”.

For our future research, we aim to create a dictionary that identifies words/phrases related to positive/negative motivation. We will extend the pilot study to a longitudinal evaluation of student motivation over four semesters of engineering education as well as prediction of student success in a PBL environment.

Citation
Format

Pluskwik, E., & Wang, Y., & Singelmann, L. (2022, August), Work-in-Progress: Understanding learners' motivation through machine learning analysis on reflection writing Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. 10.18260/1-2--40487

TY  - CPAPER
AB  - Educational data mining (EDM) is an emerging interdisciplinary field that utilizes a machine learning (ML) algorithm to collect and analyze educational data, aiming to better predict students' performance and retention. In this WIP paper, we report our methodology and preliminary results from utilizing a ML program to assess students’ motivation through their upper-division years in the XYZ project-based learning (PBL) program. ML, or more specifically, the clustering algorithm, opens the door to processing large amounts of student-written artifacts, such as reflection journals, project reports, and written assignments, and then identifies keywords that signal their levels of motivation (i.e., extrinsic vs. intrinsic). These results will be compared against other measures of motivation, including student self-report, faculty observation, and externally validated surveys. As part of a longer-term study, this pilot work sheds light on the key question for student success and retention: how does student motivation evolve through the 3rd and 4th years in college? 

The purpose of this research project is to gain insights into learners’ motivation levels and how it evolves during the last two years in college, as well as to extend current Educational Data Mining research and Machine Learning analysis described in the literature. It is significant on two fronts: 1) we will extend the ability of ML in analyzing reflective written artifacts to explore student physiological and emotional development; 2) the longitudinal study will help monitor the progressive change of motivation in college students in a PBL environment. 

Preliminary results from an initial pilot study are promising.  By analyzing written reflection journal entries from previous students, the ML algorithm has differentiated keywords into three student motivation levels: “high”, “neutral” and “low”. Using supervised classes, for example, the ML algorithm differentiated words in the highly motivated student text such as “team” and “learning”, while the text coded as low motivation included  “use”, “pushed” and “nothing”.  

For our future research,  we aim to create a dictionary that identifies words/phrases related to positive/negative motivation. We will extend the pilot study to a longitudinal evaluation of student motivation over four semesters of engineering education as well as prediction of student success in a PBL environment.
AU  - Elizabeth Pluskwik
AU  - Yuezhou Wang
AU  - Lauren Singelmann
CY  - Minneapolis, MN
DA  - 2022/08/23
PB  - ASEE Conferences
TI  - Work-in-Progress: Understanding learners' motivation through machine learning analysis on reflection writing
UR  - https://peer.asee.org/40487
DO  - 10.18260/1-2--40487
ER  -