Closed-form predictive coding matches backprop using hierarchical Gaussian filters

A team of researchers (Baskakovs, Estebe, Enevoldsen, Nielbo, Mathys, Legrand) has published a paper on arXiv introducing a closed-form predictive coding algorithm that uses hierarchical Gaussian filters (HGFs) to address two long-standing limitations of predictive coding networks: slower training and degraded performance with increasing depth. They trace these problems to the simplification of fixing the precision matrix to the identity, which discards precision-weighted prediction errors required for fast, local, and Bayesian learning. By expressing predictive coding networks as deep HGFs, they restore precision-weighted message passing, yielding dynamic uncertainty estimates and Hebbian-compatible update rules at every layer.

This new approach allows networks to simultaneously learn activations, weights, and precisions under a single free-energy objective—no global error signal needed—and resolves inference without requiring iterations or automatic differentiation. On FashionMNIST, the closed-form method approaches backpropagation in epoch-level wall-clock cost while converging in fewer epochs, and it outperforms backprop on online learning, data efficiency, and concept-drift tasks. The work establishes a tractable foundation for deep predictive coding that retains biological plausibility and interpretability, bridging the gap with backprop-based deep learning.