LF-IBIS: Full Bayesian RL Without Explicit Likelihood Functions

Researchers Stefano Masini, Cecilia Viscardi, and Michela Baccini from the University of Florence (inferred) have introduced Likelihood-Free Iterated Batch Importance Sampling (LF-IBIS), a new algorithm for full Bayesian reinforcement learning (BRL). Traditional BRL methods rely on an explicit likelihood function to update beliefs about the environment — a requirement that often fails in real-world scenarios where dynamics are too complex to model. LF-IBIS overcomes this by combining Approximate Bayesian Computation (ABC) with Iterated Batch Importance Sampling (IBIS), enabling belief updates purely from observed interactions. This allows the agent to maintain a full posterior distribution over both environment parameters and optimal policies, providing a principled way to quantify policy uncertainty and balance exploration vs. exploitation.

The team validated LF-IBIS on a simulation study of response-adaptive randomization in clinical trials, where closed-form posteriors exist for comparison, confirming the method's accuracy. They further demonstrated its utility in settings with no closed-form posterior, showing how policies can be updated online based on the posterior of the optimal policy. The algorithm outputs approximate posterior distributions that capture uncertainty, making it particularly valuable for safety-critical applications in healthcare, robotics, and finance. LF-IBIS represents a significant step toward practical Bayesian RL in environments where likelihoods are inaccessible or intractable.