New algebraic identity unifies information theory foundations from Sanov to PAC-Bayes

Akshay Balsubramani's new paper, "Information from coincidences," introduces a single algebraic mixed coincidence identity that serves as a master equation for information theory. For any family of priors and real exponents, the log of the mixed count simultaneously becomes a Boltzmann coincidence weight, an exponential-family normalizer, a maximum-entropy value, and a KL-barycenter optimum. This identity unifies classical cornerstones: concentration of empirical distributions (Sanov-type decompositions, Gibbs conditioning), hypothesis-testing error exponents (Chernoff information, multi-way analogues), change-of-measure inequalities (Donsker-Varadhan, PAC-Bayes), and rare-pattern coincidence laws (Erdos-Renyi run-length, rate-distortion, birthday thresholds). It strictly generalizes Renyi entropy and divergence variational formulas to a W-prior simplex, handling unnormalized and continuum-indexed priors.

Among its practical consequences is an exact multi-prior PAC-Bayes penalty that subtracts an explicit "coincidence bonus" from the usual single-prior posterior penalty. The paper also derives the asymptotic MAP error exponent for W-ary hypothesis testing as an edge-restricted simplex optimum. Balsubramani demonstrates the calculus at scale on two large alphabets: for language-model next-token prediction, where the framework recovers contrastive decoding; and on human genomic regulatory sequences, where it separates correlated from diverse prior families along a sliding-window trace. At 78 pages with 16 figures, the work is submitted to NeurIPS 2026 and promises to simplify theoretical ML and statistics by providing a unified algebraic lens for diverse probabilistic bounds.