Degeneracy Distillery cuts simulation needs by 10x with Fisher information geometry

Degeneracies—parameters or labels that produce similar data—plague both machine learning and probabilistic inference, making model inversion and prediction brittle. A team led by T. Lucas Makinen introduces the Degeneracy Distillery, a method that (1) detects and (2) resolves degenerate parameter combinations automatically and symbolically, using only parameter-data or parameter-simulation pairs. The approach works by estimating and flattening the Fisher information matrix, exploring the information geometry of the likelihood to characterize degeneracies as an intrinsic property of the model—no observed data is required. Unlike posterior-based methods that flatten at a single point, the Distillery flattens globally, producing symbolic coordinate transformations that reveal which parameter combinations truly drive independent effects on the data.

In synthetic and real-world test cases, the Degeneracy Distillery required up to 10× fewer simulations for neural posterior estimation at matched validation calibration, while simultaneously offering physical interpretability about the system. The paper, currently on arXiv (cs.LG/2606.23838), includes 30 pages of theory and 10 figures, with supporting code released on GitHub. For professionals working on inverse problems, simulation-based inference, or scientific ML, this method promises massive efficiency gains—turning a traditional bottleneck (degeneracy) into a source of insight. The symbolic nature of the output also means domain scientists can see exactly which parameters interact, rather than treating the model as a black box.