New AI Tech Makes Voice Assistants Faster and Cheaper

Researchers introduced SPAR-K, a framework to speed up spoken language models that mix text and speech output. Instead of running every token through the full model depth, SPAR-K lets most speech positions exit at a fixed intermediate layer, while periodic full-depth refresh steps prevent quality drift. Tested on Step-Audio-2-mini and GLM-4-Voice across four datasets, it kept accuracy nearly intact—with a maximum drop of 0.82%—while reducing average speech decoding depth by up to 11% on one model and 5% on the other, all with negligible changes in quality metrics and no extra computation. The paper also found that confidence-based early exit strategies, popular in text LLMs, work poorly for speech tokens, showing that speech needs its own specialized early exit design.