New theory unifies AI reward functions with information geometry

Proper scoring rules aren't just about evaluating forecasts — they're a doorway to generalized information theory. The logarithmic scoring rule prices information precisely as mutual information, but the same logic extends further: any convex function generates its own entropy, cross-entropy, and Bregman divergence. The quadratic score, for example, gives rise to the Gini impurity, and the gap between an entropy function and its tangent plane defines a generalized KL divergence. In this picture, entropy and cross-entropy are equally fundamental, and each information measure is just the vertical gap between a curve and its supporting tangent plane.