Asee peer logo

Board # 91 :When to Provide Feedback? Exploring Human-Co-Robot Interactions in Engineering Environments

Download Paper |


2017 ASEE Annual Conference & Exposition


Columbus, Ohio

Publication Date

June 24, 2017

Start Date

June 24, 2017

End Date

June 28, 2017

Conference Session

NSF Grantees Poster Session

Tagged Topic

NSF Grantees Poster Session

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Christian Enmanuel Lopez The Pennsylvania State University Orcid 16x16

visit author page

Christian Lopez Bencosme, is currently a Ph.D. student at Harold and Inge Marcus Department of Industrial and Manufacturing Engineering at the Pennsylvania State University. He has worked as an Industrial Engineer in both the Service and Manufacturing sectors before pursuing his Ph.D. His current research focused on the design and optimization of systems and intelligent assistive technologies through the acquisition, integration, and mining of large scale, disparate data. He is currently working on a project that ambition to design a system capable of providing students customized motivational stimuli and performance feedback based on their affective states.

visit author page


Conrad Tucker Pennsylvania State University, University Park

visit author page

Dr. Tucker holds a joint appointment as Assistant Professor in Engineering Design and Industrial Engineering at The Pennsylvania State University. He is also affiliate faculty in Computer Science and Engineering. He teaches Introduction to Engineering Design (EDSGN 100) at the undergraduate level and developed and taught a graduate-level course titled Data Mining–Driven Design (EDSGN 561). As part of the Engineering Design Program’s “Summers by Design” (SBD) program, Dr. Tucker supervises students from Penn State during the summer semester in a two-week engineering design program at the École Centrale de Nantes in Nantes, France.

Dr. Tucker is the director of the Design Analysis Technology Advancement (D.A.T.A) Laboratory. His research interests are in formalizing system design processes under the paradigm of knowledge discovery, optimization, data mining, and informatics. His research interests include applications in complex systems design and operation, product portfolio/family design, and sustainable system design optimization in the areas of engineering education, energy generation systems, consumer electronics, environment, and national security.

visit author page

Download Paper |


Abstract Co-robots are robots that work alongside their human counterparts towards the successful completion of a task or set of tasks. In the context of engineering education, co-robots have the potential to aid students towards the successful completion of an engineering assignment by providing students with real-time feedback regarding their performance, technique, or safety practices. However, determining when and how to provide feedback that advances learning remains an open research question for human-co-robot interactions. Towards addressing this knowledge gap, this work describes the data types available to both humans and co-robots in the context of engineering education. Furthermore, this works demonstrates how these data types can be potentially utilized to enable co-robot systems to provide feedback that advances students’ learning or task performance.

The authors introduce a case study pertaining the use of a co-robot system capable of capturing students’ facial keypoint and skeletal data, and providing real-time feedback. The co-robot is created using commercially available, off-the-shelf components (e.g., Microsoft Kinect) in order to expand the reach and potential availability of these systems in engineering education. In this work, the facial expressions exhibited by students as they received instructions about how to complete a task and feedback about their subsequent performance on that task are analyzed. This allows the authors to explore the influence that co-robot visual feedback systems have in changing students’ behavior while performing a task. The results suggest that students’ facial keypoint data is statistically significantly different, depending on the feedback provided (p-value<0.005). Moreover, the results suggest there is a statistically significant relationship between students’ facial keypoint data while receiving instructions on how to complete a task, and their subsequent performance on that task (p-value<0.005). These findings suggest that students’ facial keypoint data can be utilized by a co-robot system to learn about the state changes in students, as they complete a task, and provide interventions when certain patterns are discovered that have the potential to reduce students’ learning or task performance.

The outcomes of this paper contribute to advancing the National Academy of Engineering’s Grand Challenge of personalized learning a by demonstrating how students’ facial data can be utilized in an effective manner to advance human-co-robot interactions, and improve the capability of co-robot systems to provide feedback that advances students’ performance.

Lopez, C. E., & Tucker, C. (2017, June), Board # 91 :When to Provide Feedback? Exploring Human-Co-Robot Interactions in Engineering Environments Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--27955

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2017 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015