Evalatro benchmark lets LLMs play Balatro — none pass Ante 5
LLMs are failing at a roguelike card game — best model only reached Ante 5.
Developer Alexey (awfulalexey) built Evalatro, an open-source benchmark that forces LLMs to actually play the roguelike card game Balatro. Instead of processing screenshots, the model receives the full game state as structured text on each turn and must decide which cards to play — no tactical hints allowed. The setup uses the real Balatro game with Steamodded and balatrobot mods, running on fixed seeds for reproducibility. A public leaderboard at evalatro.dev tracks every run, and results are verified server-side to prevent cheating. The benchmark's goal is to clear Ante 12, a significantly harder challenge than the base game's Ante 8 win condition.
Initial results are underwhelming — the best model, mimo-v2.5-pro, barely crawled to Ante 5. DeepSeek-v4-pro couldn't even beat the boss on Ante 8 (though that result was lost on a leaderboard update). No model has come close to Ante 12. Alexey is seeking community feedback on the difficulty bar, additional metrics, and security holes. The project is open-source on GitHub and automatically installs on Windows/macOS while leaving the user's main save untouched. It's a novel way to test LLM reasoning and planning, and the public can watch the models' thought processes in the live viewer.
- Evalatro uses the real Balatro game via mods — models receive text state, not screenshots, and make autonomous play decisions.
- Fixed seeds ensure all models see identical card deals; scores are verified server-side to prevent cheating.
- Best model (mimo-v2.5-pro) reached Ante 5; goal is Ante 12 — far beyond current capabilities.
Why It Matters
This benchmark tests LLM reasoning and planning in a concrete, reproducible game environment — revealing current limitations.