CoAT framework lets audio models 'think' continuously before responding

Large Audio Language Models (LALMs) like Qwen2-Audio and Audio Flamingo have excelled at tasks from speech transcription to music analysis, but they suffer from a fundamental limitation: their hidden states are progressively optimized for text generation, causing them to discard rich acoustic information like phonetic nuance, prosody, sound events, affect, and pitch. This loss means that subtle audio cues are difficult to leverage in responses. The new Continuous Audio Thinking (CoAT) framework solves this by inserting a continuous latent workspace—a 'thinking block'—where the model can organize and preserve acoustic details before response generation. This workspace is grounded by distillation from specialized audio experts, providing a rich representation that the model can draw upon when producing text. Critically, CoAT processes the thinking block in a single prefill, avoiding any additional autoregressive decoding cost over the baseline.

CoAT was tested across three diverse LALM architectures: Qwen2-Audio, Qwen2.5-Omni-7B, and Audio Flamingo 3. On a broad benchmark suite covering audio reasoning, audio understanding, music classification, speech emotion recognition, and speech transcription, the framework consistently improved performance over baseline models. Further analysis confirmed that the auxiliary supervision from the thinking space propagates to the model's textual responses, indicating the acoustic information is genuinely utilized. By enabling audio AI to 'think' about sound before speaking, CoAT bridges a critical gap in multimodal understanding, allowing models to capture both the content and the nuance of audio input. This work represents a significant step toward more perceptive and context-aware audio AI systems.