New AI Tracks Every Fish in a School — With Zero Human Labeling

Scientists who study fish have a boring, expensive problem. To understand how schools of fish behave, someone has to watch hours of video and manually label which fish is which, frame by frame. It's slow, it's error-prone, and the fish all look identical anyway. A team of researchers has now built an AI called TrackFish3D that does this automatically, using several cameras pointing at the same tank at once.

The clever part is that it never needs humans to label anything. Instead of trying to recognize each fish by how it looks, the system uses geometry — when two cameras see the same fish, its position in 3D space lines up. That natural agreement becomes the training signal. A second component remembers each fish's identity across frames, so when one swims behind another for a moment, the AI doesn't lose track of it. The model is trained once on unlabeled footage and then just works on new videos. On the researchers' own test it lifted tracking accuracy from 87.7% to 95.8%, and on a standard zebrafish dataset it scored 81.1% versus 77.4% for the best existing method. It even performed well on real bird videos, which it was never specifically trained for.

Why should you care? Fish tracking isn't just an academic hobby. Fish farms are a multi-billion-dollar global food source, and operators need to spot sick or stressed fish early. Marine scientists want to know how warming oceans change fish behavior — and counting animals by hand simply doesn't scale. Anywhere cameras already exist and someone needs to follow individual animals or objects, this same trick could cut the human labor dramatically.

The catch is the cameras. TrackFish3D needs multiple calibrated cameras fixed in known positions — a lab tank or a farm pen, not a snorkeling video from your phone. And even at 95.8%, it still makes mistakes in crowded, chaotic scenes, where fish overlap constantly. It's a powerful research tool, not a consumer app you'll download next week.