SageMath-augmented LLM agents boost math reasoning by up to 27.8%

A new paper accepted to the 3rd AI for Math Workshop at ICML 2026 evaluates the power of integrating Computer Algebra Systems (CAS) with large language model agents. The authors—Pavel Snopov and German Magai—propose a ReAct-style agentic framework where LLMs reason iteratively with verifiable feedback from SageMath, supplemented by Context7 for up-to-date documentation. They test frontier models (including GPT-5.5 and Qwen 3.7-Max) on the RealMath benchmark, a set of research-level mathematical problems. The results are striking: average solve rate improvement of +9.7 percentage points across all models, with a range from +1.5 pp to +27.8 pp. GPT-5.5 achieved the highest overall solve rate at 75.2% while using the fewest tokens among tool-enabled configurations, and Qwen 3.7-Max saw the largest performance boost from SageMath access.

The study also introduces refinements to the RealMath benchmark itself, adding a multi-step post-processing procedure and a multi-stage validation pipeline to improve problem-set reliability. The findings demonstrate that CAS-augmented agents significantly narrow the gap between open-weight models (like Qwen) and closed models (like GPT-5.5), making advanced mathematical reasoning more accessible. By enabling LLMs to offload symbolic computations to SageMath, the agents can focus on high-level strategy and conjecture discovery. The authors believe this work is a step toward automated conjecture discovery, potentially transforming how mathematicians conduct computational exploration. The full code and data are available in the project repository.