Drones for the Environment: Battling Trash in San Francisco Estuaries

The combination of AI-powered software and drones for the environment is incredibly effective.   In this first part of a two-part series, read about how researchers in San Francisco are using drone imagery to help clean up San Francisco’s estuaries.

Researchers use drones, machine-learning algorithms to battle trash

By Jim Magill

(Part one of a two-part series on the use of drone-captured images and machine-learning software in the cause of cleaning up the environment. Part two will examine a program of the Danish Climate Ministry that uses aerial drones to capture images of polluted waterways and unmanned sailing robots to remove trash from the water.)

Researchers in California are combining drone-captured imagery with the latest artificial intelligence (AI) technology to solve an age-old problem, finding plastics and other trash strewn along the banks of creeks and streams before it can be swept away and wind up adding to the growing pollution of a bay or an ocean.

With funding from the California Ocean Protection Council, the California Department of Public Health and the U.S. Environmental Protection Agency, the San Francisco Estuary Initiative (SFEI) is using a DJI Mavic 2 Pro drone to fly over designated areas along stream and creeks throughout the state to capture multiple images. Using machine-learning tools developed by software company Kinetica, these images can be analyzed in real time to identify pieces of trash much more quickly and efficiently than through the use of more conventional methods.

SFEI Program Director Tony Hale, initiated the drone flight project as a way to modernize the institute’s existing trash-detection program, which previously had relied largely on people wearing waders walking along the stream bank.

“We thought, ‘This is very old-school. Is there anything we can do to demonstrate what’s possible by leveraging new technologies?’” he said. “By taking a drone up we knew that we could cover more space, more quickly with fewer people, which would translate into savings of time and money.”

Another advantage to using a drone, rather than a team of people on foot, to survey an assessment area is it allows the researchers to visit the same site multiple times. “So instead of being able to monitor a given site once a year, or at most twice a year, you can go out lot more often to get a denser picture of what’s happening,” Hale said.

The drone flights typically take place over assessment areas that can range up to several hundred yards long and up to 100 yards wide. The drone pilot typically flies the UAV at an altitude of about 100 feet, high enough to remain above the tree line, but close enough to the ground to obtain sharp and easily analyzable photographic images.

Hale said at the program’s outset, SFEI initially flew a DJI Phantom, but then switched to the Mavic 2 Pro. “We found the Mavic 2 Pro to be much more reliable. It was smaller, more portable, easier to deal with, and a reliable flyer,” he said. “We ran into some issues with battery life dealing with the Phantom. The Mavic 2 Pro has been a real workhorse for us.”

The researchers had another reason for choosing the Mavic 2 Pro. By using the same relatively inexpensive drone, other agencies that don’t have big budgets for doing trash identification work could duplicate SFEI’s program with similar results. “We wanted to demonstrate the functionality of using the types of vehicles you can just buy right off the shelf for not a lot money,” Hale said.

For the same reason, the project developers opted not to add any additional sensors or modify the drone itself in any way. However, they did choose to employ Esri Site Scan, a premium software package to optimize the planning, data-collection and data-distribution functions. “It made things easier for us so we weren’t spending all of our time doing the flight planning and maintaining the data,” Hale said.

Bringing Kinetica into partnership with SFEI also helped move the project forward. “We were running into some challenges in refining our algorithm,” Hale said. “Partnering with Kinetica helped to facilitate our continued iteration of tools and the machine-learning algorithms so that we could more quickly arrive at conclusions about the right direction to go with refinements.”

Nick Alonso, director of global solutions engineering at Kinetica, said the company’s machine-learning algorithm helps speed the work flow of collecting, sorting and analyzing the thousands of images collected by the drone to locate and identify individual pieces of trash.

“We are enabling these users to hook into these streaming feeds directly, stream them directly to a target table, and the second they hit that table form this type of analysis, a seamless end-to-end machine-learning work flow,” he said. Absent the Kinetica software, the task of analyzing the image data, “could take anywhere from days, weeks or even months.”

Speedy data analysis is of the essence when it comes to identifying trash for later pickup, he said. A variety of factors could likely move the trash from where it was first spotted by the drone: environmental factors such as rain, wind and wildlife, and human intervention, including vehicle and foot traffic.

“Even in a matter of five or six hours, the likelihood of that piece of trash being in the same area is pretty low,” Alonso said.

SFEI’s work using drone-created images and AI software to identify relatively large pieces of trash, such as plastic bottles, has led to the launch of another project to identify much smaller pieces of waste from the air. With funding from the California Department of Public Health, the institute has created another algorithm to analyze drone-collected images to detect cigarette butts.

With its new algorithm and other software tools provided by its partners Kinetica and Oracle, and with a drone capturing images from an altitude of 60 feet, the researchers were able to successfully identify cigarette butts 90% of the time on hardscape surfaces like asphalt parking lots. The institute is currently working to refine the learning algorithm, to enable the detection and identification of cigarette butts in other likely locations such as in parks and along footpaths.