Towards High Fidelity Face Swapping: A Comprehensive Survey and New Benchmark

Face swapping has advanced rapidly thanks to GANs and diffusion models, but the field suffers from fragmented paradigms and inconsistent evaluation. In a new paper, Qi Li and seven co-authors from the Chinese Academy of Sciences and UC Merced deliver the first comprehensive survey solely focused on face swapping. They categorize methods into five paradigms—generative adversarial networks, variational autoencoders, diffusion models, transformer-based approaches, and hybrid systems—analyzing design principles, strengths, and limitations of each. The survey highlights how prior work often lumps face swapping into broader deepfake detection, missing the unique challenges of identity preservation, attribute control, and fidelity.

To address evaluation inconsistencies, the team introduces CASIA FaceSwapping, a purpose-built benchmark dataset with balanced race, gender, and age distributions and explicit variations in pose, expression, and lighting. They define standardized protocols—including fixed train/test splits, evaluation metrics (e.g., FID, ID retention, landmark consistency), and robustness tests against compression and blur. Experiments on representative methods yield new insights: diffusion-based approaches excel in fidelity but struggle with identity preservation, while GANs offer better attribute control. The benchmark and protocols are publicly available, providing a principled foundation for future development of robust, controllable face swapping techniques.