New Method Lets Code LLMs Abstain from Hallucinated Outputs
A calibrated abstention rule helps LLMs avoid hallucinated code without external tests.
Large language models (LLMs) for code generation often produce plausible but functionally incorrect outputs — a problem known as hallucination. Researchers Yanke Zhou, Yuhao Tan, and colleagues have introduced a task abstention method that lets these models decide when not to generate code. The approach is grounded in multiple hypothesis testing and uses a calibrated abstention rule that evaluates generation consistency by actually running the code and checking execution outcomes. This avoids the need for oracle test cases or external databases, and handles syntactic variations in semantically equivalent code. The method comes with a rigorous, distribution-free theoretical guarantee on abstention decisions.
Evaluated on benchmark datasets using several open-source code LLMs, the method significantly improves the accuracy and efficiency of identifying tasks that lead to hallucinations, compared to existing techniques. By allowing models to abstain from uncertain tasks, this work provides a reliable mechanism for safer and more robust automated code generation, which is critical for production environments where faulty code can have serious consequences.
- Abstention decisions are calibrated using multiple hypothesis testing with theoretical guarantees.
- Consistency is measured by executing generated code, eliminating reliance on oracle test cases or external databases.
- Outperforms existing methods on benchmark datasets for open-source code LLMs in identifying hallucination-prone tasks.
Why It Matters
Makes automated code generation safer by allowing LLMs to know when they don't know, reducing deployment risks.