Multi-Agent RL Pricing Systems Show Collusion and Instability Risks

Researchers Shree Murthy and Rohan Pandey have published a detailed study on failure modes of deep multi-agent reinforcement learning in asynchronous pricing markets. The paper, available on arXiv (2606.09884), identifies two reproducible failure modes: tacit collusion between competing DDPG (Deep Deterministic Policy Gradient) agents, and actor-critic instability under high event rates. Using a CT-MARL benchmark with Poisson-clocked price updates and observation latency, they show that synchronous agents reliably form tacit cartels with a collusion index Δ = 0.69 ± 0.11. The study also quantifies a partial microstructure fix—introducing asynchrony alone reduces collusion by 48%, and adding a latency parameter δ further lowers Δ to a minimum of 0.28. However, the fix is acknowledged as incomplete: collusion remains supra-Bertrand, and it does not prevent the second failure mode.

At higher event rates (λ = 5), actor-critic divergence emerges, corrupting the phase-diagram cell at (λ=5, δ=1). The authors provide trajectory-level trace diagnostics that expose within-episode signaling collapse and post-shock non-recovery, offering a diagnostic toolkit for real-world deployments. The findings have significant implications for algorithmic pricing in e-commerce, energy markets, and finance, where multi-agent RL systems are increasingly used. The paper warns that without careful synchronization management and stability monitoring, autonomous pricing agents may inadvertently collude or exhibit unpredictable behavior under load. The work stands as a cautionary guide for practitioners deploying RL-based pricing strategies.