Brett Reynolds' paper reveals why AI benchmark results don't always compose

A new academic paper by Brett Reynolds, published on arXiv (2607.26159), tackles a critical but overlooked issue in AI evaluation: even when each individual inference from a benchmark result seems valid, the overall chain of reasoning from benchmark to real-world deployment may still be unsound. Reynolds introduces the concept of "projectibility"—whether a bounded extension from observed test cases to unobserved real-world cases is warranted. The paper's core contribution is a non-composition principle: support for adjacent projections only warrants their composition if endpoints and assumptions align, and if dependence and uncertainty are carried through every step.

Using a legal-research case study, Reynolds shows how benchmark evidence and a deployment study can each be internally sound yet remain parallel—never actually connecting. A reanalysis and simulation further demonstrate that aggregate stability (e.g., average accuracy) can erase distinctions that later projections require, leading to false confidence. The proposed solution is a "projectibility audit" to systematically diagnose unsupported joins in benchmark-to-use arguments. For AI engineers, product managers, and evaluators, this work is a wake-up call: don't assume that a stack of valid benchmarks equals a valid deployment justification.