Causal emergence predicts final reward in RL agents, new study finds
Pigozzi & Levin show representations with high causal power forecast training outcomes.
In a new paper on arXiv, researchers Federico Pigozzi and Michael Levin introduce the Causally Emergent Alignment Hypothesis, demonstrating that causal emergence—the degree to which an agent exerts unique predictive power on its future—aligns with and predicts final reward in reinforcement learning (RL) agents. They analyzed neural-network agents across six environments of varying complexity, using different algorithms and architectures, and computed causal emergence in their latent-space representations via the recently proposed ΦID metric.
The results show that successful agents exhibit causal emergence early in training that consistently forecasts their eventual cumulative reward. Moreover, the dynamics of this emergence align with reward improvement over time. The authors argue this is a previously undisclosed axis of neural representation reorganization in RL, potentially enabling causal interventions to build better agents. The work also highlights a new parallel between biological and artificial learning systems.
- Tested RL agents across six environments, multiple algorithms and architectures
- Used ΦID metric to quantify causal emergence in latent representations
- Successful agents' causal emergence early in training predicted final reward consistently
Why It Matters
Could lead to RL agents that self-monitor causal power, enabling more robust and interpretable learning systems.