New CASE method predicts when AI selection beats voting
CASE achieves up to 19-point accuracy gains over majority voting on tough AI questions
A team led by Zhixiang Wang and Ziliang Hong from the University of Central Florida has developed CASE (Correctness-Axis SElection), a novel method for improving the reliability of large language models (LLMs) by selecting correct answers directly from hidden states rather than relying on majority voting.
The core innovation is 'decodability', a leakage-free metric that predicts when hidden-state selection will outperform voting with high accuracy (Pearson r=0.75). Unlike majority voting, which fails on difficult questions due to correlated errors, CASE reads correctness signals from token-level hidden states using a trained linear gate. On held-out data, CASE delivers up to 19-point accuracy improvements on medium-difficulty questions and 16.8 points on hard ones across general and medical LLMs. Crucially, decodability depends on aligned knowledge recall rather than model scale, and its predictions transfer to unseen scientific domains within 3.8 points of accuracy.
- CASE uses a linear gate to select correct answers from hidden states, avoiding majority voting's correlated error problem
- Decodability metric predicts selection performance with r=0.75 correlation and AUC=0.60 decision threshold
- Method improves accuracy by up to 19 points on medium questions and 16.8 on hard ones, generalizing across domains
Why It Matters
Offers a measurable way to choose between voting and hidden-state selection for more reliable AI outputs in critical applications