Schlegel et al. dissect RNN action encodings for better RL agents
New study reveals how action inputs boost recurrent networks in reinforcement learning.
Deep Dive
Published in TMLR 2023, Schlegel et al. discuss how action information can be incorporated into the state update function of recurrent neural networks (RNNs) used in reinforcement learning (RL). They discuss several choices in using action information and empirically evaluate the resulting architectures on a set of illustrative domains. The paper also discusses future work and challenges specific to the RL setting.
Key Points
- Evaluates 5 different action encoding strategies for RNNs in RL on partially observable domains.
- Published in TMLR 2023, a top-tier machine learning journal, by University of Alberta researchers.
- Finds that action encoding choice significantly impacts learning speed and final policy performance, with simpler encodings often winning.
Why It Matters
Offers concrete design rules for engineers building memory-based RL agents, a step toward reliable real-world deployment.