Study finds Claude Sonnet 4.6 dominates GPT-5.4 in parallel code speedup
GPT-5.4 delivers zero speedup despite perfect correctness—Claude Sonnet 4.6 leads the pack.
A new arXiv paper (arXiv:2607.26083) presents a cross-language evaluation of three AI coding agents—Cursor's Composer 2.0, GPT 5.4, and Claude Sonnet 4.6—on parallel code generation. The researchers tested parallel implementations of sorting, graph traversal, and search algorithms across C++, Python, and Julia, starting from serial baselines. They measured both functional correctness and actual runtime speedup against custom serial baselines and third-party libraries.
The most striking finding: Claude Sonnet 4.6 delivered the strongest overall performance gains, while GPT 5.4, despite producing consistently correct parallel code, showed no measurable speedup in any scenario. Language choice significantly impacted results—C++ was most reliably parallelizable for graph algorithms, while Python and Julia achieved the largest speedups on search algorithms. No single language dominated across all categories, and Python and Julia even regressed on some graph algorithms. These results underscore the importance of including runtime performance efficiency as a core LLM metric, beyond just accuracy.
- Claude Sonnet 4.6 outperformed both GPT 5.4 and Cursor's Composer 2.0 in parallel code speedup across sorting, graph, and search algorithms.
- GPT 5.4 consistently produced correct parallel implementations but achieved zero measurable speedups, highlighting a gap between correctness and performance.
- Best language for speedup varied: C++ dominated graph algorithms; Python and Julia excelled in search—but no language won across all categories.
Why It Matters
Parallel programming is a key AI coding challenge—this study shows speed, not just correctness, must be a primary evaluation metric.