New AI Tracks Dozens of Moving Objects 4x Faster on a Laptop

Keeping track of many moving objects at once — pedestrians, cars, drones, athletes on a field — is a surprisingly hard problem for computers. Today's best AI trackers are built on transformers, the same engine behind ChatGPT. The catch: every time a new frame of video arrives, these trackers re-chew a whole window of old frames, like re-reading the entire chapter every time you turn a page. That wastes computing power and time.

A new paper from researchers Zhongdi Liu and Huangyu Dai proposes a leaner approach called Causal Neural Set Filtering. Instead of re-reading everything, the AI looks only at the current moment and carries its memory forward in a compact, organized summary — similar to how you remember where a car was a second ago without replaying the whole drive. Three clever mechanisms do the heavy lifting: a rule that stops the AI from tagging two objects as the same thing, a decades-old math trick (a Kalman filter) for guessing where each object will be next, and a simple 'born or gone' system that adds new objects when they appear and drops them when they leave.

The results are striking. Tested on simulated scenes, the new system scored 19.3% to 30.4% better on standard tracking accuracy measures than a leading competitor, while being 55.9% smaller and 3.76 times faster on a single ordinary processor thread. Translation: it doesn't need an expensive graphics card. Smaller and faster matters because it means this kind of tracking could run on cheap cameras, phones, or drones rather than in a data center.

There's an honest catch. All testing was done on computer-generated scenes, not real footage, so real-world performance — with rain, crowds, and messy lighting — is unproven. It's also a short academic paper, not a product, and the authors are comparing against one specific competitor. Interesting signal, not a sure thing.