Interdisciplinary team of students is developing computer algorithms to review video footage
and alert lifeguards to the presence of sharks — to protect people and sharks
SAN LUIS OBISPO — A Cal Poly research project that combines drone surveillance and artificial intelligence could one day help lifeguards warn beachgoers about sharks in the water.
Shark Spotting with Drones, conducted with CSU Long Beach’s Shark Lab, was among this year’s Summer Undergraduate Research Program projects. The SURP program, in Cal Poly’s College of Engineering, pairs undergraduate students with faculty mentors and industry to conduct meaningful, real-world research.
The research aims to alert lifeguards and protect sharks by enhancing AI methods to identify sharks and other objects from aerial video.
“The goal is to automate ingestion of drone video data and flight metadata to generate spatially explicit information on water recreational activities — when, where and who is interacting with sharks along the coast,” said Chris Lowe, director of the Shark Lab, which is known for developing innovative technology, including drones and underwater cameras, to learn about sharks in their natural habitat.
“Since human and shark beach activities are correlated to environmental conditions, the ultimate goal will be to use a series of environmental variables, such as water temperature, solar index, wave height and direction to predict where people and shark encounters might be most likely to co-occur,” he said.
According to the nonprofit Shark Research Committee, which began in 1963 to document attacks and research species indigenous to the Pacific Coast of North America, shark attacks are rare. The committee reports that since 1952 there have been only 14 fatal attacks along the California coast, including two in San Luis Obispo County — one killing a Cal Poly student swimming off Morro Bay in 1957 and another that claimed the life of swimmer in Avila Beach in 2003.
Yet, experts say the population of great white sharks — responsible for 90 percent of fatal and non-fatal attacks in the Golden State — is increasing along the West Coast. And since they can grow to be a dinosaur-like 20 feet in length, that represents a health threat.
For this project, Cal Poly students helped the Shark Lab develop machine learning algorithms, called neural networks, to identify water users — such as surfers, swimmers and bodyboarders — from drone footage.
“This is no easy task because light, sea conditions, and even the position of the water user on a board will matter,” Lowe said.
Cal Poly’s interdisciplinary team of students had backgrounds in computer science and related fields, and an interest or background in marine biology.
Computer science senior Grace Nolan, an avid surfer, swimmer and scuba diver from Thousand Oaks, California, found the project was a perfect way to combine her interests in ocean biology and artificial intelligence.
“This project stood out because I have always spent time in and around the ocean and the notion of helping protect it and its inhabitants sounded very appealing to me,” Nolan said. “So, I jumped at the chance to look at the ocean in a whole different way — from the perspective of a drone. I believe that this technology will be instrumental to the way we look at the ocean, especially now that we can do so in a non-invasive way.”
During this virtual learning summer, Nolan, and her teammates analyzed Shark Lab video collected from Santa Barbara to San Diego. The other students included: Kathir Gounder, a computer science senior from Fremont, California; software engineering major Casey Daly from Santa Rosa, California; Damon Tan, a biomedical engineering sophomore from Rosemead, California; and general engineering student Caroline Skae of Las Vegas.
The work included creating an improved machine learning model with more than 1,300 images of sharks, seals, boats, surfers and swimmers. Nolan said it was a deep dive into the key components of modern computer science, like AI, machine learning and neural networks, which employ a series of algorithms that aim to recognize underlying relationships in a set of data through a process mimicking the way a human brain operates.
“The biggest challenge for me has been learning the ropes of artificial intelligence and how the system works, in terms of image recognition,” Nolan said. “I’ve found that being thrown into the deep end with such an abstract discipline as AI has been one of the best and most rewarding ways to learn. It’s really not as intimidating a topic as the reputation that precedes it.”
Nolan said neural nets and machine learning fall under the umbrella of AI and form the center of the challenge.
“We are focused on training and using a deep neural network to do the image recognition needed for the project,” she said. “The network describes relationships and patterns in order to recognize objects from pixels in an image. Machine learning is the ability of the program to learn from these relationships and identify similar objects when shown new images.”
The research, which is sponsored by Nic and Sara Johnson, builds on a preliminary study conducted by two of Professor Franz Kurfess’ computer science classes last fall. Kurfess said his SURP team hopes to improve the accuracy rate of the drones currently used by the Cal State Long Beach researchers.
“While our initial work confirmed that our approach is viable, and the results deliver accuracy rates of about 80 to 90 percent with limited data sets and training efforts, we believe that there is significant room for improvement through a more concentrated effort by the SURP team,” Kurfess said. “The team has made impressive progress, and importantly, the students seem to be having fun with the project.”
Ultimately, the project will result in something lifeguards can use to reduce the threat of shark and human encounters, said the Shark Lab’s Lowe.
“While we still have a long way to go, I think through the use of AI we’ll be able to provide the lifeguards forecasts of where and when might be most ‘sharky’ ” days so that water users can be forewarned,” he said. “Just like stingray and rip current warnings.”
Images collected from drones between Santa Barbara and San Diego show sharks and surfers.
In photo at the top, members of the Shark Spotting with Drones team worked virtually over the summer, applying knowledge of artificial intelligence and machine learning to data collected by drones.
Contact: Pat Pemberton
805-235-0555; [email protected]
September 10, 2020
# # #
