New audit finds LLM recommenders hallucinate 8.3%—and are miscalibrated on confidence

LLM-based recommenders are known to suggest items that don't exist in a target catalog, but prior audits only measure binary out-of-domain rates. A new paper from Srijith Ravikumar goes further by asking whether these models know when they're hallucinating. The study jointly audits hallucination rate (OOD@10) and verbalized-confidence calibration (ECE, Brier, reliability) for four zero-shot LLM recommenders from four vendors—Mistral Large, Llama-3.3-70B, GPT-OSS-120B, and Claude Sonnet 4.6—across MovieLens-25M, Amazon Reviews 2023 Toys, and Yelp Open Dataset, stratified by item popularity.

The results reveal a surprising inversion of typical LLM overconfidence. Hallucination is catalog-dependent—near zero on MovieLens but up to 8.3% on Amazon—yet all models are systematically under-confident in every one of the twelve test cells. Models verbalize a mean confidence of 67–86 on items they recommend with 92–100% accuracy, meaning they know correct answers better than their stated confidence suggests. Even at 0% hallucination on MovieLens, calibration error (ECE) reaches 0.223. The under-confidence appears to be an elicitation mismatch: "Just Ask" prompts solicit a generic recommendation-quality rating, not a catalog-membership probability. A conformal abstention threshold over verbalized confidence reduces hallucination by at most 0.7 percentage points across alpha values, at a coverage cost of 4–21 percentage points—so the threshold mostly removes correct items rather than filtering hallucinations.