Researchers propose CASA for scalable AI speaking assessment
New architecture cuts speech evaluation costs by 50% while boosting accuracy
A team led by Nhan Phan from Aalto University and collaborators has introduced CASA, a novel architecture for automatic speaking assessment (ASA) that achieves state-of-the-art performance with a simpler, more efficient design. The system combines Whisper-medium (an open-source speech encoder) with Qwen3.5-2B (a 2B-parameter large language model from Alibaba's Qwen team), creating a multimodal pipeline that evaluates both *how* speech is delivered (acoustic features) and *what* is said (content accuracy).
Published on arXiv (arXiv:2608.13101) and earmarked for ICASSP 2027, CASA demonstrates a root mean square error (RMSE) of 0.358 on the Speak & Improve Corpus 2025—a 15% improvement over prior state-of-the-art models. Notably, it achieves this with approximately half the estimated inference parameters, reducing computational costs while maintaining accuracy. The architecture’s modular design, which relies on three handcrafted fluency features, allows seamless adaptation to other ASA datasets without structural changes. Through ablation studies, the researchers validate the complementary roles of acoustic and content information, showing how the LLM’s reasoning capabilities enable training-free content validation—hinting at broader applications in language learning and assessment technologies.
- CASA combines Whisper-medium and Qwen3.5-2B to evaluate speaking performance with 0.358 RMSE, surpassing prior models by 15%
- Uses ~50% fewer inference parameters than competitors while achieving state-of-the-art results
- Modular design with handcrafted fluency features enables adaptation to new ASA datasets without structural changes
Why It Matters
CASA could revolutionize language learning tools by making AI speech assessment more accurate, affordable, and interpretable for educators and platforms.