Asee peer logo

Tactile Learning: Making a Computer Vision Course Accessible through Touched-Based Interfaces

Download Paper |

Conference

2024 ASEE Annual Conference & Exposition

Location

Portland, Oregon

Publication Date

June 23, 2024

Start Date

June 23, 2024

End Date

July 12, 2024

Conference Session

Transformative Learning in STEM: Accessibility, Social Impact, and Inclusivity in Higher Education

Tagged Divisions

Equity and Culture & Social Justice in Education Division (EQUITY)

Tagged Topic

Diversity

Permanent URL

https://peer.asee.org/48051

Request a correction

Paper Authors

biography

Seth Polsley University of Nebraska, Lincoln Orcid 16x16 orcid.org/0000-0002-8805-8375

visit author page

Dr. Seth Polsley is an Assistant Professor of Practice in the Jeffrey S. Raikes School of Computer Science and Management at the University of Nebraska–Lincoln, with his academic home in the School of Computing. His research focuses on the combination of intelligent systems design and human-computer interaction in order to support novel educational and universal computing experiences.

visit author page

biography

Amanda Kate Lacy

visit author page

Amanda Lacy is a PhD student at Texas A&M University in the department of Computer Science and Engineering. Her interests are broad, with an emphasis on applying computing to promote access to information and spaces, both virtual and physical. She hol

visit author page

biography

Samantha Ray Texas A&M University Orcid 16x16 orcid.org/0000-0003-3189-8899

visit author page

Samantha Ray is a Computer Engineering PhD student at Texas A&M University. Her research focuses on creating intelligent systems for tasks that require human-like levels of understanding. She has previously worked on human activity recognition (HAR) syste

visit author page

biography

Tracy Anne Hammond Texas A&M University Orcid 16x16 orcid.org/0000-0001-7272-0507

visit author page

Dr. Hammond is Director of the Texas A&M University Institute for Engineering Education & Innovation and also the chair of the Engineering Education Faculty. She is also Director of the Sketch Recognition Lab and Professor in the Department of Computer Science & Engineering. She is a member of the Center for Population and Aging, the Center for Remote Health Technologies & Systems as well as the Institute for Data Science. Hammond is a PI for over 13 million in funded research, from NSF, DARPA, Google, Microsoft, and others. Hammond holds a Ph.D. in Computer Science and FTO (Finance Technology Option) from the Massachusetts Institute of Technology, and four degrees from Columbia University: an M.S in Anthropology, an M.S. in Computer Science, a B.A. in Mathematics, and a B.S. in Applied Mathematics and Physics. Hammond advised 17 UG theses, 29 MS theses, and 10 Ph.D. dissertations. Hammond is the 2020 recipient of the TEES Faculty Fellows Award and the 2011 recipient of the Charles H. Barclay, Jr. '45 Faculty Fellow Award. Hammond has been featured on the Discovery Channel and other news sources. Hammond is dedicated to diversity and equity, which is reflected in her publications, research, teaching, service, and mentoring. More at http://srl.tamu.edu and http://ieei.tamu.edu.

visit author page

Download Paper |

Abstract

The term "visual learner" is a ubiquitous concept in education. It is often associated with experiential- or example-based teaching that helps the students understand a concept through its application. However, for those with visual impairments, visual learning may not be an option. The Royal National Institute for the Blind reports that there are significant barriers to learning technical content for blind individuals, including access to visual resources and, correspondingly, difficulty interpreting visual concepts. In this work, we discuss experiences and key takeaways from adapting aspects of a computer science graduate course to be more accessible for the blind. The course teaches advanced machine learning methods rooted in computer vision and gesture-based methods in order to build sketch recognition systems. Additionally, it emphasizes elements of human-computer interaction and interface design, many of which are visual concepts. In order to adapt the curriculum, we used a high-resolution tactile display capable of mirroring imagery from a video display into a depth map that could be felt. This enabled the dual presentation of visual content as tactile surface maps. Through this process, we learned several best practices in terms of how to create content that transfers well from one modality to another, and we also developed a number of guidelines for creation of teaching materials like notes and assignments in a way that is more screen-reader friendly. This paper shares key takeaways while also communicating student and teacher perspectives on developing, teaching, and using these materials with the goal of encouraging more technical courses that are traditionally visually-based to consider possible ways to becoming more accessible.

Polsley, S., & Lacy, A. K., & Ray, S., & Hammond, T. A. (2024, June), Tactile Learning: Making a Computer Vision Course Accessible through Touched-Based Interfaces Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/48051

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2024 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015