New RL method trains 7B model to match betting market NFL win probability calibration

A new paper from arXiv (Singh, Reddy, Chopra) tackles the challenge of training AI to make well-calibrated probabilistic forecasts—like NFL win probabilities—using reinforcement learning with verifiable rewards. Standard RL fails because rewarding a single stochastic outcome introduces noise that corrupts the model's chain-of-thought reasoning. The authors propose a label-free reward: a state-conditioned empirical win rate estimated from historical outcomes, which removes label noise. They also keep gradients off the reasoning path (via direct prediction or a gradient mask) to prevent corruption.

Trained solely with this reward—no human labels, no supervised fine-tuning—a 7B model achieves calibration matching the betting market's. Its Brier score equals that of a zero-shot frontier model and a tabular estimator, revealing the market's small edge comes from live in-game information beyond the shared inputs. The gradient mask preserves chain-of-thought reasoning that standard training destroys, enabling interpretable forecasts. This work suggests a path to calibrating AI forecasters across domains without expensive human annotations.