MIME model boosts two-person motion retrieval by 12.8%

Researchers have unveiled MIME (Multimodal Interactive Motion Encoder), a novel model that tackles the problem of aligning natural language with two-person interactive motion—a critical capability for animation, AR/VR, and embodied AI. Unlike existing models that treat each person's motion independently, MIME uses stream-based co-attention mechanisms combined with explicit interaction features and a curriculum-based contrastive training strategy. This allows it to capture both individual actor dynamics and the shared structure between two interacting people from text descriptions.

On the Inter-X benchmark, MIME consistently outperforms early and late fusion baselines across gallery sizes, achieving a 12.8% relative improvement in text-to-motion retrieval recall at rank 1 with a 2,000-sample gallery. When used as a frozen auxiliary prior on downstream tasks (TIMotion and InterMask), it improves semantic alignment metrics while maintaining comparable FID scores on the unseen InterHuman dataset. These results demonstrate that interaction-aware multimodal encoding can boost retrieval and transfer to improve motion generation. The paper is under review at WACV 2027.