AI Still Can't Understand Your Body from a First-Person Video
New test shows even the best AI fails at simple body-motion questions—here's why that matters.
A new benchmark called SOMA tests how well AI understands human body motion from first-person video, using 1,632 human-verified questions across body configuration, motion dynamics, and temporal reasoning. Across 37 MLLMs plus human evaluation, humans scored 68.5% from video alone, while the strongest model, Qwen3.6-27B, reached 50.0% zero-shot.
The researchers say MLLMs often misread the wearer’s body, misjudge timing, and show biases such as favoring the right hand or under-counting repetitions. To help narrow the gap, they introduce a training dataset of aligned video, body motion, and language supervision, plus a multimodal baseline, and say the benchmark, training data, and code will be released.
- Humans score 68.5% on body-motion questions from first-person video; the best AI gets only 50%.
- AI often makes mistakes like confusing left and right hands or miscounting repetitions.
- Researchers released a new training dataset and baseline model to help AI improve.
Why It Matters
AI fitness coaches, health monitors, and smart glasses could misread your movements, making them unreliable or unsafe.