DATSQN: New ternary spiking neurons beat binary baseline in Atari DQN tasks
Asymmetric ternary neurons fix gradient bias and outperform binary spiking networks in seven Atari games.
A new ternary spiking neuron model is proposed to improve the representation capacity of binary spiking neurons in deep Q-learning. According to the article, a recently introduced ternary neuron model performs worse than binary models in deep Q-learning tasks, and the authors hypothesize that gradient estimation bias during training is the underlying cause. Their proposed ternary spiking neuron reduces this bias and serves as the fundamental computing unit in a deep spiking Q-network called the deep asymmetric ternary spiking Q-network (DATSQN). Tested on seven Atari games from the Gym environment, DATSQN mitigates the performance degradation of ternary neurons and improves the mean game score relative to the binary baseline under the evaluation settings used in the paper.
- DATSQN uses a new asymmetric ternary spiking neuron that reduces gradient estimation bias, a problem that plagues standard ternary neurons in DQN tasks.
- Standard ternary neurons underperform binary neurons in deep Q-learning, contradicting previous studies that claimed better representation capacity.
- The network was tested on seven Atari games from OpenAI Gym and improved the mean game score over binary spiking baselines.
Why It Matters
This technique could enable neuromorphic chips to run efficient reinforcement learning agents for robotics and edge AI.