Scientists Taught AI to Solve Brand-New Puzzles by Evolving Answers
AI that copes with surprises — not just memorized answers — could change your work.
ARC-AGI-2 is a set of 60 visual puzzles built specifically so AI can't have seen the answers before. Each puzzle shows a few grids of colored squares and asks you to figure out the hidden rule, then apply it. People find many of them manageable. AI models, including the best ones, tend to struggle badly. A researcher at arXiv tried something different: pair a small AI model with a "genetic algorithm," which works like selective breeding. It makes many copies of a program, changes them slightly, keeps the versions that get closest to a correct answer, and repeats.
The AI part (a shrunken version of a model called Qwen3.5-4B, small enough to run on modest hardware) wrote an initial batch of programs for each puzzle. Those programs alone solved 2 of the 60 tasks, about 3%. Then the evolution step took over and nudged them forward. It added 4 more correct solutions, bringing the total to 6 out of 60, or 10%. The most striking finding: when the researcher skipped the AI's starting programs and let evolution work from scratch, it solved nothing at all. Both halves were needed. The AI provided a sensible starting point; the breeding process did the polishing.
Why does this matter outside a lab? Because the whole point of these puzzles is novelty — problems unlike anything the AI was trained on. That is exactly where today's AI is weakest. It shines at tasks similar to its training data and stalls on genuinely unfamiliar ones. If AI could reason its way through brand-new situations, it could handle a weird spreadsheet format, an unusual customer request, or a bug nobody has documented before.
The honest catch: 10% is a low score. People do far better on these puzzles, and this is a single small experiment that hasn't been reviewed by other scientists. The puzzles themselves are deliberately odd. So don't expect this technique in a product you use next month. But the recipe — AI makes a rough guess, evolution refines it — is a promising direction, and the fact that evolution alone failed shows the two halves genuinely need each other.
- A small AI writes rough programs, then a 'breeding' process tweaks them until they work — together solving 6 of 60 puzzles designed to be unfamiliar.
- Both halves mattered: the AI's starting guesses solved 2 tasks, the tweaking added 4 more, and tweaking alone solved zero.
- The 10% success rate is low compared to humans, so this is early lab research rather than something showing up in products soon.
Why It Matters
Progress here means AI that handles unfamiliar tasks at work, not just things it saw during training.