Asee peer logo

A Real-Time Model to Assess Student Engagement during Interaction with Intelligent Educational Agents

Download Paper |


2014 ASEE Annual Conference & Exposition


Indianapolis, Indiana

Publication Date

June 15, 2014

Start Date

June 15, 2014

End Date

June 18, 2014



Conference Session

Computer-Based Learning Models

Tagged Division

Computers in Education

Page Count


Page Numbers

24.95.1 - 24.95.11



Permanent URL

Download Count


Request a correction

Paper Authors


LaVonda N. Brown Georgia Institute of Technology

visit author page

LaVonda Brown received her B.S. (2010) in Electronics Engineering from Norfolk State University and M.S. (2012) in Electrical and Computer Engineering from Georgia Institute of Technology. She is currently pursuing a Ph.D. at the GT Human-Automation Systems (HumAnS) Lab. Her research interests include engagement, educational robotics, and socially interactive robots.

visit author page

author page

Ayanna M. Howard Georgia Institute of Technology

Download Paper |


A Real-Time Model to Assess Student Engagement during Interaction with Intelligent Educational AgentsAdaptive learning is an educational method that utilizes computers as an interactive teachingdevice. Intelligent tutoring systems, or educational agents, use adaptive learning techniques toadapt to each student’s needs and learning styles in order to individualize learning. Effectiveeducational agents should accomplish two essential goals during the learning process – 1)monitor engagement of the student during the interaction and 2) apply behavioral strategies tomaintain the student’s attention when engagement decreases. In this paper, we focus on the firstobjective of monitoring student engagement. Most educational agents do not monitorengagement explicitly, but rather assume engagement and adapt their interaction based on thestudent’s responses to questions and tasks. A few advanced methods have begun to incorporatemodels of engagement through vision-based algorithms that assess behavioral cues such as eyegaze, head pose, gestures, and facial expressions. Unfortunately, these methods require a heavycomputation load, memory/storage constraints, and high power consumption. In addition, thesebehavioral cues do not correlate well with achievement of high-cognitive tasks, as we willdiscuss in this paper. As an alternative, our proposed model of engagement uses physical events,such as keyboard and mouse events. This approach requires fewer resources and lower powerconsumption, which is also ideally suited for mobile educational agents such as handheld tabletsand robotic platforms.In this paper, we discuss our engagement model which uses techniques that determine behavioraluser state and correlate these findings to mouse and keyboard events. In particular, we observethree event processes: total time required to answer a question; accuracy of responses; and properfunction executions. We evaluate the correctness of our model based on an investigationinvolving a middle-school after-school program in which a 15-question math exam that increasedin cognitive difficulty is used for assessment. The first 5 questions of the exam are categorized assimple 1-step problems; the next 5 questions require multiple steps; the last 5 questions arecategorized as very difficult (i.e. requiring a high-cognitive load). Eye gaze and head posetechniques are referenced for the baseline metric of engagement. We then conclude theinvestigation with a survey to gather the subject’s perspective of their mental state throughout theexam.We found that our model of engagement is comparable to the eye gaze and head pose techniques.When high-level cognitive thinking is required, our model is more accurate than the eye gaze andhead pose techniques due to the use of outside variables for assistance and non-focused gazesduring questions requiring deep thought. The large time delay associated with the lack of eyecontact between the student and the computer screen causes the aforementioned algorithms toincorrectly declare the subjects as being disengaged. Furthermore, speed and validity ofresponses can help to determine how well the student understands the material, and this isconfirmed through the survey responses and video observations. This additional information willbe used in the future to better integrate instructional scaffolding and adaptation with theeducational agent.

Brown, L. N., & Howard, A. M. (2014, June), A Real-Time Model to Assess Student Engagement during Interaction with Intelligent Educational Agents Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--19987

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015