UC Berkeley study reveals how reward compression boosts RL learning efficiency

Researchers Gaia Molinaro and Anne Collins from UC Berkeley have published a study titled 'Reward function compression facilitates goal-dependent reinforcement learning,' proposing a cognitive mechanism that explains how humans efficiently learn to value novel outcomes. The theory posits that goal-dependent learning initially depends on capacity-limited working memory. With repeated experience, the brain creates a 'compressed' reward function—a simplified goal rule—that transfers to long-term memory. This automation frees up working memory resources, allowing faster and more efficient learning. The team validated this model across six behavioral experiments, showing that learning efficiency degrades as the goal space grows larger, but improves significantly when that goal space can be compressed into a simpler representation. They also found a positive correlation between reward processing speed and learning outcomes.

While the paper focuses on human cognition, the implications extend directly to artificial reinforcement learning (RL). Current AI RL agents often struggle with high-dimensional reward spaces and require extensive computation. The principle of reward compression could inspire new algorithmic designs where agents build simplified internal reward functions over time, reducing memory load and increasing sample efficiency. The study also offers actionable insights for behavioral interventions: helping people break down complex goals into compressible rules could improve task persistence and achievement. The authors note that the algorithmic details of how compression occurs remain to be established, but their computational models and experimental evidence provide a strong foundation for future work in both neuroscience and machine learning.