Study: LLMs beat HPC optimization frameworks with direct C code

A new arXiv preprint from Charles University investigates how LLMs can be used to optimize high-performance computing (HPC) code. The researchers compared two approaches: giving LLMs direct optimization goals and letting them generate C code, versus using traditional abstractions—specifically, building computation pipelines and optimization schedules via established frameworks like auto-tuning tools. They evaluated both strategies on the PolyBench benchmark suite, a standard set of numerical kernels used in HPC research.

The results are striking. LLMs prompted with explicit optimization objectives—such as 'minimize runtime while preserving correctness'—produced C code that outperformed code generated through framework-based pipeline construction and schedule optimization. The direct-generation approach also achieved higher validity rates, meaning the LLM's code was more likely to compile and produce correct results. The authors argue this challenges the assumption that traditional abstractions (grid search, constraint-based transformations) are necessary scaffolding for LLM-guided optimization. Instead, LLMs can leverage high-level semantic understanding to skip the rigid pipeline and directly emit optimized parallel code, potentially opening the door to more flexible, verifiable HPC optimization workflows.