Asee peer logo

Towards Personalized Performance Feedback: Mining the Dynamics of Facial Keypoint Data in Engineering Lab Environments

Download Paper |


2018 ASEE Mid-Atlantic Section Spring Conference


Washington, District of Columbia

Publication Date

April 6, 2018

Start Date

April 6, 2018

End Date

April 7, 2018

Tagged Topic


Page Count


Permanent URL

Download Count


Request a correction

Paper Authors


Christian Enmanuel Lopez Pennsylvania State University, University Park Orcid 16x16

visit author page

Christian Lopez Bencosme, is currently a Ph.D. student at Harold and Inge Marcus Department of Industrial and Manufacturing Engineering at the Pennsylvania State University. He has worked as an Industrial Engineer in both the Service and Manufacturing sectors before pursuing his Ph.D. His current research focused on the design and optimization of intelligent systems through the acquisition, integration, and mining of large scale, disparate data. He is currently working on a project that ambition to design a system capable of providing students customized motivational stimuli and performance feedback based on their affective states.

visit author page


Conrad Tucker Pennsylvania State University, University Park

visit author page

Dr. Tucker holds a joint appointment as Assistant Professor in Engineering Design and Industrial Engineering at The Pennsylvania State University. He is also affiliate faculty in Computer Science and Engineering. He teaches Introduction to Engineering Design (EDSGN 100) at the undergraduate level and developed and taught a graduate-level course titled Data Mining–Driven Design (EDSGN 561). As part of the Engineering Design Program’s “Summers by Design” (SBD) program, Dr. Tucker supervises students from Penn State during the summer semester in a two-week engineering design program at the École Centrale de Nantes in Nantes, France.

Dr. Tucker is the director of the Design Analysis Technology Advancement (D.A.T.A) Laboratory. His research interests are in formalizing system design processes under the paradigm of knowledge discovery, optimization, data mining, and informatics. His research interests include applications in complex systems design and operation, product portfolio/family design, and sustainable system design optimization in the areas of engineering education, energy generation systems, consumer electronics, environment, and national security.

visit author page

Download Paper |


According to the National Academy of Engineering, the development of personalized learning is one of the grand engineering challenges of the 21st century1. Even though affect-sensitive systems have been used for personalized learning, current systems provide feedback based on predefined relations of affective state and performance. However, studies have shown that the affective state that correlates to good performance could vary between tasks and students. Hence, these systems can only provide accurate performance feedback once the student completed the task at hand. In light of the current methods’ limitation, this work presents a machine learning method for predicting students’ performance by using the dynamics of their facial keypoint data captured while reading the instructions of a task, thus, avoiding the need to infer their affective state. A case study involving 40 students performing tasks in an engineering lab environment is used to validate the proposed method. The results reveal that the proposed model yielded an accuracy of 80%. The results indicate the importance of using students’ facial keypoint data captured while reading the instructions of a task, to predict their performance. This method could be implemented in engineering lab environments to provide real-time feedback to students and advance personalized learning.

Lopez, C. E., & Tucker, C. (2018, April), Towards Personalized Performance Feedback: Mining the Dynamics of Facial Keypoint Data in Engineering Lab Environments Paper presented at 2018 ASEE Mid-Atlantic Section Spring Conference, Washington, District of Columbia.

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015