Avatar V generates 1080p avatar videos from video references, beating top models
New framework conditions on full video token sequences to reproduce talking rhythms and micro-expressions.
Generating avatar videos that are not only visually similar but behaviorally recognizable—faithfully reproducing a person's talking rhythm, gestural tendencies, and expression dynamics—has remained an open challenge. Existing methods typically condition on single static images, which provide insufficient identity information and fail to capture dynamic motion traits. Avatar V, developed by a team of 23 researchers, addresses this by conditioning directly on the full token sequence of a reference video. This approach learns to reproduce both static identity attributes (facial geometry, skin texture) and dynamic behavioral patterns through attention over the reference context. Key technical innovations include Sparse Reference Attention, an asymmetric mechanism achieving linear-complexity conditioning on arbitrarily long references; a motion representation stream enabling closed-loop talking style transfer; and an identity-aware super-resolution refiner that inherits the full reference conditioning.
To support this framework at scale, the team built a data engine curating over 100 million training clips from 50 million raw videos. A five-stage training pipeline was deployed across thousands of GPUs, incorporating flow matching pre-training, personality fine-tuning, two-phase distillation (achieving more than 10x acceleration), and RLHF alignment. The result is a system that generates 1080p videos of unlimited duration, achieving state-of-the-art identity preservation, lip synchronization, and generation quality. On cross-scene benchmarks, Avatar V consistently outperformed leading systems including Seedance 2.0, Kling O3 Pro, Veo 3.1, and OmniHuman 1.5 in both automated metrics and human evaluation, setting a new standard for behaviorally accurate digital avatars.
- Sparse Reference Attention enables linear-complexity conditioning on arbitrarily long video references, capturing both static and dynamic identity traits.
- Trained on 100M+ clips from 50M raw videos using a five-stage pipeline with flow matching, personality fine-tuning, two-phase distillation (>10x acceleration), and RLHF.
- Outperforms Seedance 2.0, Kling O3 Pro, Veo 3.1, and OmniHuman 1.5 on identity preservation, lip sync, and generation quality in cross-scene benchmarks.
Why It Matters
Enables hyper-realistic, behaviorally accurate digital avatars for video production, virtual presence, and personalized media at scale.