Asee peer logo

Deep Learning at a Distance: Remotely Working to Surveil Sharks

Download Paper |

Conference

2021 ASEE Virtual Annual Conference Content Access

Location

Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

Microsoft Teams, Deep Learning, and Classroom Flipping

Tagged Division

Ocean and Marine

Page Count

19

DOI

10.18260/1-2--36896

Permanent URL

https://peer.asee.org/36896

Download Count

640

Paper Authors

biography

Grace Nolan California Polytechnic State University, San Luis Obispo

visit author page

Grace is a third year Computer Science student and Undergraduate Researcher at Cal Poly SLO. Her experience and areas of interest are in artificial intelligence and UI/UX design.

visit author page

biography

Franz J. Kurfess California Polytechnic State University, San Luis Obispo Orcid 16x16 orcid.org/0000-0003-1418-7198

visit author page

Franz J. Kurfess is a professor in the Computer Science and Software Engineering Department, California Polytechnic State University, San Luis Obispo, where he teaches mostly courses in Artificial Intelligence, Human-Computer Interaction, and User-Centered Design. Before joining Cal Poly, he was with Concordia University in Montreal, Canada, the New Jersey Institute of Technology, the University of Ulm, Germany, the International Computer Science Institute in Berkeley, CA, and the Technical University in Munich, where he obtained his M.S. and Ph.D. in Computer Science.

His main areas of research are Artificial Intelligence and Human-Computer Interaction, with particular interest in the intersection of the two fields.

visit author page

author page

Kathirvel A. Gounder

author page

Damon Tan

author page

Casey Daly California Polytechnic State University, San Luis Obispo

biography

Caroline Skae California Polytechnic State University, San Luis Obispo

visit author page

Caroline graduated from California Polytechnic State University, San Luis Obispo in Summer 2020 and received a B.S. in General Engineering with an individual course study in marine conservation and technology. She is an avid diver and has a strong fascination with sharks. She is currently working in both plastic conversion technology and regenerative ocean farming on the Central Coast of California.

visit author page

Download Paper |

Abstract

In the months following the novel Coronavirus pandemic outbreak, the world has seen an immediate and unprecedented global shift towards remote learning and working. In the academic field specifically, it has fundamentally shifted how the process of learning happens. Throughout the summer of 2020, we had the opportunity to observe how doing research remotely would affect the complicated dynamics of working in a cross-disciplinary team. Our project centered around utilizing machine learning technologies to detect sharks in videos taken from drones, as well as a few possible applications of this technology. Traditionally, a project such as this would involve weekly in-person meetings, with in-person collaboration opportunities on such things as developing our Neural Network computational model, designing user interfaces, and discussing shark behaviors with domain experts such as marine biologists. However, due to the circumstances of the pandemic, we had to make do with weekly online Zoom meetings, as well as figuring out how to collaborate with each other to do the technical aspects of this project remotely. Our team of engineering students comes from a variety of backgrounds including computer science, software engineering, biomedical engineering, and marine biology. The project itself incorporates the use of drones to collect video footage, Machine Learning to process the images, and marine biology in order to analyze the behavior of sharks in their natural habitat in a noninvasive way. Collaboration with a team of marine biologists specializing in sharks at a different university was essential, but our inability to meet with them in person imposed a significant hurdle. Working remotely with a team of this size and range of skills was a learning process during which we overcame numerous logistical, technical, and personal obstacles. In the end, however, we succeeded in developing a system that is capable of locating objects of interest in the video footage and to assign those objects to categories such as shark, seal, tuna, boat, surfer, paddleboarder, swimmer, and others. This system is the basis for a front-end to be used by marine biologists in the behavior of sharks and other marine life: whereas in the past, marine biology students would spend endless hours watching drone video footage to identify snippets of interest, they can now focus on the behavior analysis of the animals. Although the immediate results of this project were obtained in the identification of sharks, work is underway to expand this to other domains. While object recognition has been studied and applied widely, our specific situation involving drones flying over water posed additional challenges such as the presence of glare, waves, and foam. Further challenges are the identification of relatively small objects at varying depths in the water, viewed from a moving object (the drone) at different heights, speeds, and angles. From a technical perspective, the use of a centralized database, advanced labeling software, sophisticated Machine Learning tools, and powerful cloud computing facilities allowed us to do meaningful work during this time while keeping ourselves organized and productive. Though we could not physically meet each other or any sharks, this summer project was an invaluable learning experience with innovation in the application of Artificial Intelligence to show for it.

Nolan, G., & Kurfess, F. J., & Gounder, K. A., & Tan, D., & Daly, C., & Skae, C. (2021, July), Deep Learning at a Distance: Remotely Working to Surveil Sharks Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36896

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015