How AI is Helping Solve the Problem of Marine Debris
A Colby professor and her team design a sustainable marine debris cleanup framework
Maine’s offshore islands, famous for their granite cliffs, pounding surf, and dark spruce forests, are wild, remote, and beautiful places.
Something they’re not, however, is pristine.
All manner of trash and detritus are deposited onto their shores, in large part because of the waves, currents, and winds that swirl around the islands. On Allen Island, part of Colby’s Island Campus, the quiet coves and beaches often are littered with marine debris that routinely includes bottles, cans, single-use plastics, microplastics, derelict fishing gear, and styrofoam. Visitors once even found a full-size refrigerator that had washed up on the island.
This marine debris is a persistent issue that poses an ecological threat to birds, fish, and other animal life. But for Colby faculty and students, it also presents an important opportunity to find new ways for technology and artificial intelligence to help solve problems.
It was Amanda Stent, the inaugural director of the Davis Institute for Artificial Intelligence, who first told Tahiya Chowdhury, now an assistant professor of computer science, about the problem of marine debris. It got her thinking.
The traditional approach to removing trash from the shores of Allen Island takes a lot of time and a lot of human attention and energy. Chowdhury wondered if computer vision, a field of artificial intelligence that enables computers to interpret and make decisions based on visual data, could simplify the procedure by detecting and identifying trash from drone photos of the shoreline.
She bet that it could, and she was right.
Chowdhury and others pioneered an AI protocol to detect and identify trash that is comparable to state-of-the-art human-supervised methods. Last summer, she, Ray Wang ’26, Whitney King, the Dr. Frank and Theodora Miselis Professor of Chemistry, and Nicholas Record, senior research scientist at the Bigelow Laboratory for Ocean Sciences, published a peer-reviewed paper about the project. They have plans to broaden the study to include more locations and models.
“This is just the first step, and there are more things we can do to find ways to shed light on this broader problem—not just for Allen Island and Maine, but for other places as well,” Chowdhury said.
Using AI to make the world better
As AI technology advances and becomes more intertwined with many aspects of human life, people are continuing to evaluate both the opportunities and challenges that it can pose. For her part, Chowdhury, who came to Colby in 2022 as the first postdoctoral fellow at the Davis Institute for Artificial Intelligence, has long been keen on finding ways to use AI technology to make the world a better place.
“I was really interested in using computer vision for applications that have a social impact,” she said. “I wanted to work on things that are related to sustainability.”
Colby’s traditional approach to shoreline cleanup on Allen Island features two events each year, each requiring teams of volunteers. At the first event, people fan across the island to look for and record the location of trash. But they don’t pick it up at that time, in part because Maine regulations prohibit anyone except the owner from touching fishing gear, and a lot of what is found on shores is derelict, or “ghost,” fishing gear. At the second event, volunteers use the information from the first to find and remove as much of the debris as possible.
“The entire process right now is very manual, and it can be tedious, time-consuming, and very dependent on the weather,” Chowdhury said.
They began to imagine what an automated approach to the problem would look like, and they used drone images of Allen Island as a starting point. Humans could simply look at the photos and identify the debris they see, but that is so time-consuming it’s not practical, she said.
Instead, Chowdhury and her team plugged in different existing AI solutions and engineered an approach that would work. They decided that the computers needed to break it down to an automated two-step process, with the first step involving looking at the images and finding trash in them. The second step asks the AI model to identify the trash, initially by comparing the photos with a very large image dataset from the internet.
“We told the model that it didn’t have to classify all the different things. Just find the trash,” Chowdhury said.
Hiccups and innovations
Even so, it wasn’t always a simple task. Trash easily recognizable by human eyes at ground level needed to be correctly identified by an AI model using drone photos taken from a higher altitude and different angles, which led to complications. For instance, the AI model initially misidentified an orange lobster buoy as a carrot, and it couldn’t recognize a lobster trap at all.
“The type of trash that you would see in a place really depends on the economy of that area,” Chowdhury said. “In Maine, we are talking about lobster buoys or traps because that’s a key part of the economy here. In other parts of the world, they are also experiencing the same problem with marine debris, but the type of trash is very different.”
The key, she said, is to engage with local stakeholders to understand what type of trash is found, instead of relying on AI models that likely were trained on very different types of data. The Colby team needed to teach the AI model new labels to use. Instead of “lobster trap,” which it didn’t recognize, they substituted “cage,” which it did.
For Wang, working on the project and coauthoring the paper about it have been highlights of his time at the College.
“The coolest part for me was the interdisciplinary nature of this project,” said Wang, a computer science major concentrating on AI. “We got to work with a data set that addressed a real-world issue, marine debris and how it’s affecting our coastlines, and we integrated computer science and software engineering into that problem.”
It felt good, and important, to try to solve the problem, or at least to make it easier for people to clean up the island.
“I was able to see the actual impact that my project had in the real world, and a lot of times I think you don’t get to see that part,” Wang said.
Ultimately, the team succeeded at training the model to recognize and correctly identify trash, detecting objects with 69-percent accuracy, which compares favorably to human detection accuracy.
“Computers are much faster, and humans have so many more important things to do than just look at these images,” Chowdhury said. “I think human time will be better spent on what happens after trash is detected. How do we assign people to clean it up, and what do we see from the pattern? That’s where I think human expertise will be really smart.”
AI and human intelligence are best used together
Just adding AI to the mix has already been a help, according to King.
“There was an opportunity to tie in this problem we have with new AI capability,” he said. “We flew it with a drone, we took images, [the AI model] classified it and identified where the trash was, and we were able to send groups out and say, ‘OK. Pick up every bit of trash in this area.’”
In the future, Chowdhury and others would like to expand the project beyond Allen Island.
“Marine debris and trash are not unique to Maine. This is a worldwide problem,” she said. “For there to be a more robust approach, we need to see whether there are better models that have better detection and classification approaches.”
It also seems likely that the approach will need to incorporate citizen science, the practice of public participation and collaboration in scientific research to increase scientific knowledge. This could be as easy as using a phone app to log photos of marine debris.
“We are not removing the humans completely from the process,” Chowdhury said. “Marine debris is a global-scale problem, and you cannot rely only on the researchers or the policymakers.”
For her, a goal is to have human expertise and AI work together to solve this problem.
“A lot of the AI solutions that come out don’t make any real-world impact,” Chowdhury said. “People involved in community work can help by improving the model and figuring out what will actually work in the real world.”
