Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Models

A team from DeepMind and the University of Amsterdam has published a groundbreaking paper introducing Code-Space Response Oracles (CSRO), a framework that fundamentally rethinks how multi-agent systems generate strategies. The research addresses a critical limitation in current multi-agent reinforcement learning (MARL) systems like Policy-Space Response Oracles (PSRO), which rely on 'black-box' neural networks that are impossible to interpret or debug. CSRO replaces these opaque RL agents with Large Language Models that generate policies as executable, human-readable code.

The framework explores three distinct approaches to leverage LLMs as policy generators: zero-shot prompting, iterative refinement, and a novel distributed system called AlphaEvolve. This evolutionary approach allows LLMs to collaboratively evolve and improve strategies over multiple generations. In testing, CSRO demonstrated competitive performance against traditional RL baselines while producing a diverse set of explainable policies. The system effectively shifts the paradigm from optimizing inscrutable neural network parameters to synthesizing transparent algorithmic behavior that humans can understand, audit, and trust.

This work represents a significant convergence of symbolic AI approaches with modern LLM capabilities, offering a path toward more trustworthy and controllable autonomous systems. By generating policies as code rather than neural network weights, CSRO enables developers to inspect, modify, and reason about agent behavior in complex multi-agent environments like financial markets, autonomous vehicle coordination, or strategic game playing. The framework leverages LLMs' pretrained knowledge to discover sophisticated, human-like strategies that might elude traditional RL approaches.