TSA: New AI method keeps track of objects even when they disappear

Current video object-centric learning methods, like recurrent slot-attention, assume every object slot should be updated and decoded at every frame. This causes problems when an object is absent or fully occluded: the slot's representation drifts (update-induced state drift) and the inactive slot still influences reconstruction (decoder-induced reconstruction interference). Researchers from multiple institutions (Duc Nguyen, Sieu Tran, Hao Vo, Khoa Vo, Duy Minh Ho Nguyen, Nghi D. Q. Bui, Anh Nguyen, Long Mai, Ngan Le) propose Temporal Slot Activation (TSA) to address these issues.

TSA learns a per-slot, per-frame activation score alpha_k,t in (0,1) without any visibility supervision. This activation acts as a shared latent control variable for slot lifecycle. When a slot is inactive, TSA anchors its state to the previous frame via activation-gated updating, and suppresses decoder participation through an activation-dependent additive bias on attention logits before softmax normalization. Additionally, a Temporal Context Encoder conditions activation prediction on per-slot temporal memory, improving decisions under partial occlusion and gradual reappearance. TSA was evaluated on MOVi-C, MOVi-E, YT-VIS, and OVIS benchmarks using both standard (FG-ARI, mBO) and tracking-based (IDF1, HOTA) metrics, showing consistent improvements in object decomposition and temporal identity preservation, with particularly large gains on long, heavily occluded videos.