PSEF: Transformer-based filter beats classical methods on complex data

Bayesian filtering tackles the challenge of inferring the evolving probability distribution of a system's state from noisy observations, critical for uncertainty quantification in fields like weather forecasting and robotics. Traditional methods struggle with nonlinear, non-Gaussian settings. The new Proper Scoring Ensemble Filter (PSEF) from Bach et al. solves this by training a permutation-invariant transformer to map a forecast ensemble and observations into an analysis ensemble. Training uses the energy score—a strictly proper scoring rule—that rewards accurate approximations of the full probability distribution, not just point estimates. The authors prove that under realizability, the population objective converges to the true Bayesian filter.

Numerical experiments show PSEF significantly outperforms classical ensemble Kalman filters (EnKF) and mean-squared-error-trained baselines on challenging distributions, including multi-modal posteriors. For near-Gaussian problems, a hybrid approach—learning a correction to EnKF—is most effective; for highly non-Gaussian cases, an end-to-end PSEF without inductive bias excels. The paper (87 pages, 17 figures) also establishes mean-field consistency between single-trajectory empirical objectives and the population objective. This work opens a new path for scalable, learned probabilistic filtering across scientific domains.