MIT researchers propose DeTAiL for dementia classification from speech
Multimodal LLMs may hallucinate reasoning — DeTAiL uses internal representations instead.
A team from MIT and Harvard (Liming Wang, Neguine Rezaii, Bradford C. Dickerson, James Glass) published a paper examining whether multimodal large language models (MLLMs) benefit from explicit reasoning when classifying dementia from voice recordings. They found that naive approaches relying on text-based rationales can produce hallucinated and inconsistent diagnostic justifications, actually degrading performance below that of LLM-free baselines. This challenges the assumption that reasoning capabilities inherently improve automatic dementia classification (ADC) systems.
To address this, they propose DeTAiL (Dementia Thinker with Nonlinear Adaptor and Reinforcement Learning), an adaptor-based framework that taps into the internal representations of reasoning MLLMs rather than their text outputs. DeTAiL consistently outperforms both strong baselines and methods dependent on text-based rationales across two dementia datasets with different test formats and label granularities. The work highlights the need for careful integration of reasoning in medical AI and suggests that internal model states may be more reliable than generated rationales for clinical classification tasks.
- Naive text-based rationales from MLLMs cause hallucinations and lower dementia classification accuracy
- DeTAiL uses internal representations via a nonlinear adaptor + reinforcement learning, outperforming baselines
- Tested on two dementia datasets with distinct formats and label granularities; code/demo pending release
Why It Matters
Smarter AI diagnostics for dementia without relying on hallucinated rationales could improve clinical accuracy and trust.