fabians.site
← all writing
article Jul 2 · 4 min

Protecting endangered sea turtles with machine learning

Automatically cataloging drone footage to help improve sea turtle poaching alert systems.

A computer vision detector only knows the footage it was trained on, and turtle footage looks nothing alike from one place to the next. The same species over a bright reef, a deep green swell, or murky coastal water barely reads as the same thing to a model, and what it learned in one water will probably fail in the next.

That is the problem behind my master's thesis. I worked on it with DELT, an NGO that builds sea turtle detectors from drone footage and hands them to conservation groups around the world. They shared their datasets and trusted me to run with the problem.

Two sea turtles over turquoise reef shallows, from a drone. Japan
A sea turtle at the surface of deep green water, from a drone. Costa Rica
A sea turtle in murky green coastal water, from a drone. Florida
Same species, three countries, three very different waters.

The problem

Most sea turtle species are endangered, and one of the most direct threats is egg poaching. On unprotected beaches in Costa Rica, poachers take more than half of some species' nests.

DELT fights this with computer vision. Conservation groups fly drones over nesting beaches, and DELT's models spot the turtles and flag which nests to protect before poachers reach them.

This is domain shift, and it is a headache: one model, footage from everywhere, worse in each new place. Specializing a detector per condition would help, but first you need to know which conditions exist, and sorting a huge catalog by hand does not scale.

Can we sort the footage into its capture conditions automatically, so each detector is tuned to the world it works in?

One detector Clear water Murky water Night vision Holds up Struggles Struggles
One detector, three conditions. Tap a condition to see how it holds up.
The same model handles every condition, and its accuracy does not hold up equally. Illustrative.

How I approached it

The tool for this is clustering: letting a computer sort things into groups by similarity, without anyone telling it the groups ahead of time.

You cannot watch thousands of clips by hand. So instead of labels, I describe each clip with a handful of numbers, like how big the turtle is in the frame and the color and clarity of the water. Clips with similar numbers land together, and each group is a capture regime: a recurring kind of footage.

Group 1Group 2Group 3
Clustering
Drone clip, group 1 regimeDrone clip, group 2 regimeDrone clip, group 3 regimeDrone clip, group 1 regimeDrone clip, group 2 regimeDrone clip, group 3 regimeDrone clip, group 1 regimeDrone clip, group 2 regimeDrone clip, group 3 regime
Real clips ride in, get sorted by the clustering step, and come out grouped by regime.

To be sure the groups were real and not one algorithm's quirk, I ran three different clustering methods, each with its own idea of what a group even is: K-means, Ward, and HDBSCAN.

Each tile is one drone clip. The three methods sort the same clips like this:

center
K-means You choose how many groups. Each clip joins whichever one it sits closest to.
cut into 3
Ward Every clip starts on its own, then the two most alike keep pairing up, until you split them into groups.
noise
HDBSCAN Wherever clips bunch up close together, that is a group. The loners out on their own get left out.

Run all three and they land on nearly the same grouping, 0.91 on a scale where 1 is identical and 0 is random (it's the Adjusted Rand Index, if you want to look it up). HDBSCAN is the real tell: I never told it how many groups to look for, and it still found the same three on its own, plus a handful of clips it refused to place.

It's reassuring that three methods with such different assumptions settle on the same groups. When they do, the structure is far more likely to be a property of the footage than the result of any single algorithm.

The hard parts

Two things fought me from day one.

One campaign was almost the entire catalog

The catalog is seven campaigns: two drone surveys from Japan, one each from Costa Rica, Florida and the Bahamas, a batch of clips pulled from public YouTube videos, and a small set from Conservation AI. And they are wildly uneven. Of the 539 clips, a single campaign is 471 of them, about 87%. The other six split the last 13% between them; the smallest brought a single clip.

87%

of all 539 clips are one campaign, Japan (471 of them). The other six, the teal corner, share the last 13%.

That is dangerous: clustering can "find" groups that are really just the dominant campaign in disguise. So I checked it the other way around: drop each campaign in turn, re-cluster, and see how much the grouping shifts. Losing the giant barely moves it, 0.84 on that same 1-is-identical scale. The surprise runs the other way: drop one of a couple of the small campaigns instead and the grouping scrambles, all the way down to 0.09.

So which is it, a fragile method or a quirk of the score? Mostly the score. Those small campaigns are not separate clusters that vanish when you drop them; they pin down the shape of the big cluster, and with the number of groups fixed at three, removing them makes the algorithm re-slice the whole catalog. A similarity measured over every clip then falls hard, even though the group I actually trust, the high-altitude one, comes back almost unchanged. The 0.09 is mostly the imbalance talking, not the cluster groups dissolving.

Most clips never recorded their altitude

The single most useful number, how high the drone was flying, was almost never recorded, so I estimated it. A sea turtle's shell is a roughly known size (about 0.65 m across), so how wide it appears in the frame reveals how far the camera was: fill the frame and the drone was low; a tiny speck and it was high. It is standard pinhole geometry: give it the turtle's real width plus the camera's focal length and sensor size (here, a 4.49 mm focal length and a 6.17 mm sensor), and it returns the altitude in metres.

what the camera sees, looking straight down

The turtle spans 12% of the frame's width, so the drone was about 3.9 m up.

altitude = real size × focal length ÷ (share of the frame × sensor size)

So, did it work?

Yes, the sorting part. The clips fall into three capture regimes: subaquatic close-ups, high-altitude passes, and coastal mid-altitude flights. The high-altitude group is the solid one, 28 clips from five different projects across four regions (Japan, Costa Rica, Florida, the Bahamas), grouped by their shared conditions and not their source.

That is the part DELT can use. One detector can't be sharp on clear water, murky high-altitude passes and night vision all at once. Split the footage by regime, tune a model to each, and it should detect more turtles, which means more nests reached before the poachers do.

I did not train those specialist models myself. Showing one actually beats the general detector means measuring the improvement on enough labeled clips per regime, and most of the groups are still too small for that number to mean much. So rather than chase a result that would not hold up, I focused on the part that scales: the clustering workflow itself.

It ships as turtle-cluster, a package DELT re-runs on its own catalog in about three minutes, so each regime can keep filling up until it is big enough to train a detector on.

The code is on GitHub. If you work on aerial wildlife detection and any of this is useful, tell me.