EpicStar memory framework lets LLMs beat StarCraft II baselines with 10x fewer tokens

A new paper from Yi Wu and Zhimin Hu, published on arXiv and accepted at ICLR 2025, argues that LLMs are fundamentally poor strategists in long-horizon environments. Their analysis shows that finite attention causes LLM agents to lose strategic coherence over thousands of steps, leading to "strategic drift" where local decisions don't align with an overarching plan. To fix this, they propose EpicStar, a framework that treats memory as a policy rather than a simple context buffer.

EpicStar maintains two memory layers: a bank of successful past episodes (a heuristic for proven strategies) and a working memory that tracks immediate environmental changes. During inference, a dynamic gating mechanism decides whether to directly execute a retrieved action or trigger new reasoning by fusing retrieved episodes with current working memory. Evaluated in StarCraft II against diverse opponent styles, EpicStar achieved significantly higher win rates than baseline LLM agents, while consuming an order of magnitude fewer tokens. The improvement held across difficulty levels and strategies, suggesting that structured cross-episode memory, not raw scale, is the key to robust long-term strategic execution in autonomous LLM agents.