July 26, 2021
July 26, 2021
July 19, 2022
Ocean and Marine
In the months following the novel Coronavirus pandemic outbreak, the world has seen an immediate and unprecedented global shift towards remote learning and working. In the academic field specifically, it has fundamentally shifted how the process of learning happens. Throughout the summer of 2020, we had the opportunity to observe how doing research remotely would affect the complicated dynamics of working in a cross-disciplinary team. Our project centered around utilizing machine learning technologies to detect sharks in videos taken from drones, as well as a few possible applications of this technology. Traditionally, a project such as this would involve weekly in-person meetings, with in-person collaboration opportunities on such things as developing our Neural Network computational model, designing user interfaces, and discussing shark behaviors with domain experts such as marine biologists. However, due to the circumstances of the pandemic, we had to make do with weekly online Zoom meetings, as well as figuring out how to collaborate with each other to do the technical aspects of this project remotely. Our team of engineering students comes from a variety of backgrounds including computer science, software engineering, biomedical engineering, and marine biology. The project itself incorporates the use of drones to collect video footage, Machine Learning to process the images, and marine biology in order to analyze the behavior of sharks in their natural habitat in a noninvasive way. Collaboration with a team of marine biologists specializing in sharks at a different university was essential, but our inability to meet with them in person imposed a significant hurdle. Working remotely with a team of this size and range of skills was a learning process during which we overcame numerous logistical, technical, and personal obstacles. In the end, however, we succeeded in developing a system that is capable of locating objects of interest in the video footage and to assign those objects to categories such as shark, seal, tuna, boat, surfer, paddleboarder, swimmer, and others. This system is the basis for a front-end to be used by marine biologists in the behavior of sharks and other marine life: whereas in the past, marine biology students would spend endless hours watching drone video footage to identify snippets of interest, they can now focus on the behavior analysis of the animals. Although the immediate results of this project were obtained in the identification of sharks, work is underway to expand this to other domains. While object recognition has been studied and applied widely, our specific situation involving drones flying over water posed additional challenges such as the presence of glare, waves, and foam. Further challenges are the identification of relatively small objects at varying depths in the water, viewed from a moving object (the drone) at different heights, speeds, and angles. From a technical perspective, the use of a centralized database, advanced labeling software, sophisticated Machine Learning tools, and powerful cloud computing facilities allowed us to do meaningful work during this time while keeping ourselves organized and productive. Though we could not physically meet each other or any sharks, this summer project was an invaluable learning experience with innovation in the application of Artificial Intelligence to show for it.
Nolan, G., & Kurfess, F. J., & Gounder, K. A., & Tan, D., & Daly, C., & Skae, C. (2021, July), Deep Learning at a Distance: Remotely Working to Surveil Sharks Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36896
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015