In2AI's training method beats GPT-5 with 8B parameter model

Training language model agents for multi-agent strategic interaction is notoriously difficult because the quality of any action depends on future events, illegal moves, or other agents' decisions. Standard reinforcement learning assumes per-step rewards, but that breaks down when outcomes are entangled across time and agents. Researchers from In2AI (Aliaksei Korshuk, Alexander Buyantuev, Ilya Makarov) solved this with a novel method: delayed per-step reward attribution with eligibility gating. They compute rewards only at episode end, propagate them back to originating steps using task-specific semantics, and exclude invalid steps from training.

Combined with asynchronous rollout generation via vLLM's continuous batching, curriculum-based opponent sampling, and multi-level stratified batch construction, the technique delivers stable, sample-efficient RL training in multi-agent environments. The team evaluated on MindGames Arena at NeurIPS 2025, where a single 8-billion-parameter open-source model matched or surpassed substantially larger proprietary systems, including GPT-5, in head-to-head play. It took first place in both the Open (unrestricted) and Efficient (≤8B parameters) tracks, proving that smarter algorithms can overcome size disadvantages.