PnP: New noise-based defense for speaker verification attacks with 0.014 real-time factor

Modern automatic speaker verification (ASV) systems are increasingly vulnerable to adversarial perturbations—tiny, inaudible changes that can fool a system into accepting an imposter or rejecting a legitimate user. Traditional defenses rely on diffusion-based purification, which iteratively removes noise but suffers from high inference latency. In a new paper submitted to IEEE TASLP, researchers from multiple institutions (including Yibo Bai, Sizhou Chen, and others) flipped the paradigm: instead of removing noise, they propose the Positive-Incentive Noise Predictor (PnP), which learns to add input-adaptive noise (termed π-noise) that actively improves robustness. The key insight is that most of the robustness in diffusion models comes from the forward noising process, not the reverse denoising. PnP exploits this by mixing learned noise directly into the audio, achieving a real-time factor of just 0.014 (orders of magnitude faster than diffusion alternatives) while preserving accuracy on natural speech. In experiments across four advanced ASV backbones, PnP effectively defends against white-box, black-box, and even defender-aware adaptive attacks. The framework can also be cascaded with a diffusion denoiser to further improve perceptual quality. Code and purified audio examples are available on GitHub.