Delayed Repression Destabilizes Multi-Agent Systems – Q-Learning Shows Partial Resilience

Igor Itkin's new paper, "Delayed Repression and Emergent Instability in Adaptive Multi-Agent Systems," investigates how regulatory institutions that observe, deliberate, and intervene with a characteristic delay can inadvertently destabilize otherwise stable multi-agent systems. The research is presented in two stages. First, a delayed replicator equation is analyzed, where autonomous agents benefit from radical behavior but face punishment based on a lagged institutional alarm signal. Itkin derives a closed-form critical delay threshold beyond which the unique interior equilibrium loses stability through a Hopf bifurcation, proving the bifurcation is supercritical (producing bounded oscillations) for a broad family of sigmoid response functions.

In the second stage, the study embeds N=240 agents on a network using tabular Q-learning and compares three decision architectures: non-reactive agents (fixed policy), reactive agents (threshold heuristic without memory), and Q-learning agents (adaptive with cumulative value estimates). The results contradict the naive expectation that learning amplifies instability. Non-reactive agents show 0% runaway across all delay values. Reactive agents collapse catastrophically, with 96% runaway at delay ≥ 8 steps. Q-learning agents achieve partial resilience, with only 66% runaway at delay = 20. The destabilizing ingredient is reactivity to delayed signals: agents that immediately exploit low-alarm windows create oscillatory feedback loops. Learning buffers this through implicit punishment memory encoded in Q-values.