Unsupervised Continual Learning for Amortized Bayesian Inference

A team of researchers including Aayush Mishra, Šimon Kucharský, and Paul-Christian Bürkner has introduced a groundbreaking framework for Unsupervised Continual Learning in Amortized Bayesian Inference (ABI). Their work addresses a critical limitation in current ABI systems, which typically suffer from performance degradation under model misspecification and struggle with sequentially arriving data. While self-consistency training on unlabeled empirical data can enhance network robustness, existing approaches are confined to static, single-task settings and fail to handle distribution shifts over time. The researchers' novel approach decouples simulation-based pre-training from unsupervised sequential fine-tuning on real-world data, creating a more adaptable and resilient system.

The technical innovation centers on two adaptation strategies designed to combat catastrophic forgetting: SC with episodic replay (using a memory buffer of past observations) and SC with elastic weight consolidation (regularizing updates to preserve task-critical parameters). Across three diverse case studies, these methods significantly mitigate forgetting and yield posterior estimates that outperform standard simulation-based training, achieving results closer to MCMC reference standards. This represents a substantial advancement toward trustworthy ABI that can maintain performance across evolving data streams and multiple tasks, potentially impacting fields from healthcare diagnostics to financial modeling where Bayesian methods are crucial but data distributions constantly shift.