ZP-KWS: The AI Framework That Lets You Define Any Keyword—and Blocks Impostors Without Retraining

Researchers from National Taiwan Normal University and Academia Sinica have published ZP-KWS, a lightweight framework for personalized keyword spotting (UD-KWS) that adds speaker verification to user-defined wake-word detection. Existing UD-KWS systems learn speaker-invariant representations, meaning anyone uttering the correct keyword can trigger them. ZP-KWS addresses this dual zero-shot challenge—unseen keywords and unseen speakers—by pairing a phoneme-supervised audio encoder with a compact GE2E-pretrained speaker encoder (about 0.9M parameters). The total model runs within 1.55M parameters, making it suitable for edge deployment on devices like smart speakers or phones.

At inference, ZP-KWS uses multiplicative late fusion, giving each branch (keyword detection and speaker verification) independent veto power. This allows flexible operation modes: from conventional detection (any speaker) to strict speaker-gated activation (only the enrolled user) without retraining. On benchmarks including LibriPhrase, Google Speech Commands, and Qualcomm datasets, ZP-KWS achieves up to 60% reduction in target-only false rejection rate at a fixed 1% false acceptance rate relative to the strongest baseline, while maintaining competitive keyword detection accuracy. The paper has been accepted to Interspeech 2026.