Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models

A team of researchers has introduced a novel, training-free method for detecting hallucinations in large language models (LLMs) like GPT-4 and Claude. The core idea is that when an LLM is uncertain or fabricating information, the conditional distribution of possible outputs for a given prompt becomes more complex and varied. To quantify this without access to the model's internal probabilities, the method generates multiple sample responses and computes the optimal transport (Wasserstein) distances between the sets of token embeddings from these samples. This creates a cost matrix that measures how 'expensive' it is to transform one response into another, with higher average costs (AvgWD) and greater eigenvalue complexity (EigenWD) signaling potential hallucinations.

The framework, detailed in the arXiv paper "Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models," is designed to be lightweight and broadly applicable. A key innovation is its extension to black-box LLMs through a technique called teacher forcing, where a smaller, accessible 'teacher' model approximates the token generation process. In experiments, the AvgWD and EigenWD signals proved competitive with established uncertainty-based baselines and showed complementary strengths across different models and datasets. This positions distribution complexity as a powerful, standalone signal for assessing LLM truthfulness without the need for collecting labeled data or fine-tuning a separate detector model.