AffectFlow-DINO boosts emotion AI with uncertainty-aware rectified flow
New model improves fear detection from 3.8% to 33.1% without retraining
AffectFlow-DINO, developed by researchers Bekhouche, Sellam, Dornaika, and Hadid, tackles the inherent ambiguity of in-the-wild facial behavior by replacing deterministic predictions with a conditional rectified-flow head. Built on a frozen DINOv3 ViT-S/16 vision transformer, the system jointly estimates continuous valence-arousal, classifies eight basic facial expressions, and detects twelve Action Units from static images. By learning a generative distribution rather than a single output, the model enables uncertainty-aware one-to-many predictions via Monte Carlo sampling. Extensive ablation studies show that the rectified flow decoder consistently outperforms deterministic baselines, with valence-arousal concordance correlation (CCC-V) improving by +0.058.
A key innovation is post-hoc threshold calibration, which dramatically recovers performance on severely imbalanced rare classes without retraining—fear detection accuracy jumps from 3.8% to 33.1%. Combined with backbone fine-tuning and flow retuning, the final model achieves a multi-task performance score (P_MTL) of 1.177, substantially beating the official ABAW challenge baseline of 0.45. The work demonstrates that generative uncertainty modeling, paired with smart calibration, can significantly improve affective computing systems, especially for rare and subtle emotional expressions.
- Uses conditional rectified flow to model prediction ambiguity, enabling uncertainty-aware one-to-many estimates via Monte Carlo sampling
- Improves valence-arousal correlation by +0.058 CCC-V and recovers fear class accuracy from 3.8% to 33.1% with post-hoc threshold calibration
- Achieves P_MTL=1.177, outperforming the 0.45 baseline by over 2.6x on the 11th ABAW challenge
Why It Matters
Uncertainty-aware emotion AI can drastically improve rare-expression detection, critical for mental health and human-computer interaction.