New study shows Q-learning AI can sustain cooperation in pricing games

Algorithmic collusion among pricing algorithms has been a growing concern for regulators, as reinforcement learning agents can learn to sustain supra-competitive prices. Previous research largely assumed exploration vanishes over time, leading to convergence to a single strategy. However, real-world algorithms often remain adaptive through constant exploration. A new preprint on arXiv (2607.13607) by Janusz Meylahn and Maximilian Schäfer tackles this gap by studying Q-learning dynamics under persistent exploration in the classic repeated Prisoner's Dilemma with one-period memory.

The authors frame the problem as a high-dimensional stochastic process and show that cooperative strategies can be dominant in a time-averaged sense — meaning agents spend a significant fraction of time cooperating even while continually exploring. They derive a boundary condition based on the expected Q-learning dynamics that predicts when non-defection (i.e., cooperative or mixed) behavior will prevail. Extensive simulations using epsilon-greedy Q-learning confirm that this boundary is a strong predictor for the absence of defection-dominated outcomes. The work provides a more realistic foundation for understanding AI collusion in evolving markets.