AI-Enhanced DOBOT Magician for Classroom Education: Hand Gesture Control for Hazardous Material Handling Simulation
- Conference
- Location
-
Arlington, TX, Texas
- Publication Date
-
March 9, 2025
- Start Date
-
March 9, 2025
- End Date
-
March 11, 2025
- Page Count
-
11
- Permanent URL
-
https://peer.asee.org/55028
Paper Authors
Ahmed Hasan University of New Mexico
Experience in the interaction of inorganic contaminants with mineral surfaces, colloidal transport of radioactive and metal contaminants and their applications to remediation, physicochemical characterization of soil and mineral surfaces, development of g
Alex V Maynes
I am an Undergraduate Computer Science student at the University of New Mexico with a passion for artificial intelligence and robotics. My goal is to develop AI-driven systems that enhance human capability, improve safety, and push the boundaries of human-machine collaboration. With experience in software development, research, and my time in the U.S. Air Force, I strive to create technology that not only solves real-world problems but also helps people overcome personal limitations. I am driven by the belief that AI can be a powerful tool for both individual and societal advancement.
Abstract
Students in the ME-150 class in the Mechanical Engineering Department at the University of New Mexico get engaged in engineering research using experimental tools to learn about machine programming and control, Newton’s three laws of motion, advanced tribology, propulsion and engines, and bioengineering 1,2. Desktop robotic arms called DOBOT Magicians are used to conduct algorithmic programming to run manufacturing simulations. However, the DOBOT capabilities are limited to manufacturing simulations and do not extend to other applications, such as decommissioning of hazardous plants, remote handling of hazardous materials, or working in hazardous environments to humans. A research project was developed to explore a novel application in human-robot integration using computer vision tools, OpenCV and Google’s MediaPipe, in Python to control a DOBOT Magician robotic arm with hand gestures. Preliminary results demonstrate successful hand tracking, gesture interpretation, and corresponding robotic arm manipulation. The new capabilities allow the students in the ME-150 class to develop simulations to remotely control the DOBOTs to work in hazardous environments, perform precise tasks gripping, transporting and packaging hazardous materials, perform confined space inspections, and conduct specific tasks within hazardous environments.
Hasan, A., & Maynes, A. V. (2025, March), AI-Enhanced DOBOT Magician for Classroom Education: Hand Gesture Control for Hazardous Material Handling Simulation Paper presented at 2025 ASEE -GSW Annual Conference, Arlington, TX, Texas. https://peer.asee.org/55028
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2025 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015