Asee peer logo

Eye-Track Modeling of Problem-Solving in Virtual Manufacturing Environments

Download Paper |

Conference

2021 ASEE Virtual Annual Conference Content Access

Location

Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

Manufacturing Division Session - Virtual and Augmented Reality

Tagged Division

Manufacturing

Page Count

17

Permanent URL

https://strategy.asee.org/37169

Download Count

186

Request a correction

Paper Authors

biography

Rui Zhu Complex System Monitoring, Modeling and Analysis Laboratory, The Pennsylvania State University, University Park, PA, 16802, USA

visit author page

Rui Zhu is a Ph.D. candidate in the Harold and Inge Marcus Department of Industrial and Manufacturing Engineering at the Pennsylvania State University. Her research interests focus on sensor-based modeling, analysis, and optimization of complex systems, with applications in virtual reality, healthcare, and smart communities.

visit author page

biography

Faisal Aqlan The Pennsylvania State University - Erie Campus Orcid 16x16 orcid.org/0000-0002-0695-5364

visit author page

Dr. Faisal Aqlan is an Associate Professor of Industrial Engineering at The Pennsylvania State University, The Behrend College. He received his Ph.D. in Industrial and Systems Engineering form The State University of New York at Binghamton. His research interests include sensor-based virtual reality for manufacturing and healthcare applications. He is a senior member of the Institute of Industrial and Systems Engineers (IISE) and currently serves as the IISE Vice President of Student Development.

visit author page

biography

Richard Zhao University of Calgary Orcid 16x16 orcid.org/0000-0001-8257-4291

visit author page

Dr. Richard Zhao is an Assistant Professor in the Department of Computer Science at the University of Calgary. He leads the serious games research group, focusing on games for training and education where he utilizes artificial intelligence, virtual reality and eye-tracking technologies for this purpose. He is currently working on a game-focused graduate program at the University of Calgary. He received his M.S. and Ph.D. in Computing Science from the University of Alberta.
Dr. Zhao has served as a program committee member on academic conferences such as the International Conference on the Foundations of Digital Games (FDG), the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE) and the ACM Special Interest Group on Computer Science Education (SIGCSE) Technical Symposium.

visit author page

biography

Hui Yang The Pennsylvania State University

visit author page

Dr. Hui Yang is a Professor in the Harold and Inge Marcus Department of Industrial and Manufacturing Engineering at The Pennsylvania State University, University Park, PA. Dr. Yang's research interests focus on sensor-based modeling and analysis of complex systems for process monitoring, process control, system diagnostics, condition prognostics, quality improvement, and performance optimization. His research program is supported by National Science Foundation (including the prestigious NSF CAREER award), National Institute of Standards and Technology (NIST), Lockheed Martin, NSF center for e-Design, Susan Koman Cancer Foundation, NSF Center for Healthcare Organization Transformation, Institute of Cyberscience, James A. Harley Veterans Hospital, and Florida James and Esther King Biomedical research program. His research group received a number of best paper awards and best poster awards from IISE Annual Conference, IEEE EMBC, IEEE CASE, and INFORMS.

Dr. Yang is the president (2017-2018) of IISE Data Analytics and Information Systems Society, the president (2015-2016) of INFORMS Quality, Statistics and Reliability (QSR) society, and the program chair of 2016 Industrial and Systems Engineering Research Conference (ISERC). He is also an associate editor for IISE Transactions, IEEE Journal of Biomedical and Health Informatics (JBHI), IEEE Transactions on Automation Science and Engineering (TASE), IEEE Robotics and Automation Letters (RA-L), Quality Technology & Quantitative Management, and an Associate Editor for the Proceedings of IEEE CASE, IEEE EMBC, and IEEE BHI.

visit author page

Download Paper |

Abstract

Problem-solving focuses on defining and analyzing problems, then finding viable solutions through an iterative process that requires brainstorming and understanding of what is known and what is unknown in the problem space. With rapid changes of economic landscape in the United States, new types of jobs emerge when new industries are created. Employers report that problem-solving is the most important skill they are looking for in job applicants. However, there are major concerns about the lack of problem-solving skills in engineering students. This lack of problem-solving skills calls for an approach to measure and enhance these skills. In this research, we propose to understand and improve problem-solving skills in engineering education by integrating eye-tracking sensing with virtual reality (VR) manufacturing. First, we simulate a manufacturing system in a VR game environment that we call a VR learning factory. The VR learning factory is built in the Unity game engine with the HTC Vive VR system for navigation and motion tracking. The headset is custom-fitted with Tobii eye-tracking technology, allowing the system to identify the coordinates and objects that a user is looking at, at any given time during the simulation. In the environment, engineering students can see through the headset a virtual manufacturing environment composed of a series of workstations and are able to interact with workpieces in the virtual environment. For example, a student can pick up virtual plastic bricks and assemble them together using the wireless controller in hand. Second, engineering students are asked to design and assemble car toys that satisfy predefined customer requirements while minimizing the total cost of production. Third, data-driven models are developed to analyze eye-movement patterns of engineering students. For instance, problem-solving skills are measured by the extent to which the eye-movement patterns of engineering students are similar to the pattern of a subject matter expert (SME), an ideal person who sets the expert criterion for the car toy assembly process. Benchmark experiments are conducted with a comprehensive measure of performance metrics such as cycle time, the number of station switches, weight, price, and quality of car toys. Experimental results show that eye-tracking modeling is efficient and effective to measure problem-solving skills of engineering students. The proposed VR learning factory was integrated into undergraduate manufacturing courses to enhance student learning and problem-solving skills.

Zhu, R., & Aqlan, F., & Zhao, R., & Yang, H. (2021, July), Eye-Track Modeling of Problem-Solving in Virtual Manufacturing Environments Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. https://strategy.asee.org/37169

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015