arXiv study: AI agents solve coding 6x better yearly as GPT-5.6 Luna slashes costs

A new arXiv paper from Pranav Kumar Kaliaperumal (arXiv:2608.13675) provides a sweeping retrospective on language-model progress from BERT in October 2018 to the frontier agents of July 2026. The headline finding: the ability to resolve real coding issues has improved by nearly six times per year since late 2024. This dramatic acceleration coincides with a collapse in costs—OpenAI's budget model GPT-5.6 Luna delivers flagship-level performance for just $1 to $6 per million tokens, undercutting older, less capable models by a wide margin.

The paper also highlights a shift from monolithic models to task-targeted systems. Claude Opus 5 leads in frontend coding, Claude Fable 5 dominates repository-level coding, and GPT-5.6 Sol excels at terminal tasks. In a grade-school math benchmark using Qwen 2.5, basic sampling solved 58 of 100 problems, while advanced sampling reached 79. A confidence-ranking tool identified 47 correct answers within its top 50 selections, demonstrating practical utility for filtering and sorting large task batches. The full research materials are publicly available, allowing teams to reproduce the results and apply the cost-performance insights to their own AI infrastructure decisions.