New Benchmark UnpredictaBench Tests LLMs' Ability to Sample True Distributions

A new research paper introduces UnpredictaBench, a benchmark designed to evaluate whether large language models (LLMs) can accurately sample from true underlying probability distributions, rather than simply producing varied outputs. The work, led by Amirhossein Abaskohi and collaborators from multiple universities, addresses a critical gap as LLMs are increasingly used as proxies for humans or other systems in economic simulations and complex modeling. Traditional metrics focusing on output diversity fail to capture whether sampled outputs match a target distribution's randomness — a requirement for realistic simulations.

UnpredictaBench includes 448 problems spanning canonical statistical distributions, distributions from stochastic programs, and natural-language scenarios describing random processes. The evaluation uses KS@N, a metric based on the Kolmogorov-Smirnov test, which quantifies how well model samples of size N approximate the ground-truth distribution. Across open and proprietary models, results show a wide capability spread: at KS@100, scores range from near 0 to over 20%, and no model surpasses 40%. Even adding reasoning only marginally improves scores. The benchmark highlights that simple distributional sampling remains an unsolved challenge, marking a necessary first step toward deploying LLMs as reliable simulators for complex systems.