AI Video Editors Are Being Fooled by Fake Lip Sync
Your favorite AI video tools might be tricking you with bad lip-syncing — here's how
A new study audited four automatic audio-visual synchronization metrics that are widely used to rank and train audio-visual generators. The results show no single winner: one metric was strongest at tracking timing offsets, another pair matched a human-aligned proxy better, and one stood out as the weakest standalone metric. The metrics also disagreed with each other, and combining them didn't beat the best individual metric. The authors recommend reporting AV-sync as a "Reliability Card" with metric-family breakdowns and confidence intervals instead of a single bare score.
- AI tools that check lip-sync in videos often disagree and sometimes give good scores to bad syncs
- Researchers found four popular metrics failed reliability tests, with scores varying widely
- Experts now recommend showing multiple scores instead of a single rating to avoid being misled
Why It Matters
Your favorite AI videos might have awkward lip movements — and the grading system is broken