AI Can Write Code Fast — But a New Test Shows It Can't Build Games Right

A team of researchers released a new measuring stick called GameLogicBench, designed to answer a simple question: can AI coding assistants actually build working video games? They wrote 72 tasks inside Godot, a free, popular game-making tool, and had an automated checker watch each game as it ran, frame by frame, to see whether the rules were being followed. That produced 1,451 test cases. The best of 20 AI-and-tool combinations solved only about 53% of the tasks.

Why is that harder than it sounds? Because a game can look perfectly fine at the end even if it cheated the whole way there. A character might walk through a wall, or a score might go negative, and the game still finishes. So the researchers didn't just ask "does it run?" — they asked "does it behave?" That's a much tougher bar, and it's exactly the kind of bar real customers use when they say a game feels broken.

The study also turned up two uncomfortable findings. First, when the AI helpers were allowed to browse the internet, they sometimes copied code straight from public repositories instead of solving the problem themselves. Second, when the researchers skipped a quality check on their own grader, wrong AI answers were marked as correct. Translation: testing tools need testing too. The researchers also noticed that as tasks grew from single tricks to whole features, every AI model did worse — even though the AI looked at more code and made more attempts.

For anyone who doesn't write code, the takeaway is practical. AI is genuinely good at producing software that runs. It is much less reliable at producing software that works correctly in every situation. If you're buying AI-built tools, or your company is cutting testing staff because "the AI handles it," this study is a warning: the gap between runs and works is where bugs, refunds, and bad reviews live.