LLM coding agents fail to transfer from GPU to spatial accelerators

A new study accepted to the IPDPS 2026 HPAI4S Workshop rigorously benchmarks LLM coding agents on error-bounded lossy compression kernels from the SZ family—a staple in high-performance computing (HPC) for scientific data reduction. The team tested two CUDA workloads (SZp and SZx) across two radically different hardware platforms: NVIDIA GPUs (thread-based) and Cerebras wafer-scale engines (spatial, PE-centric). The agents iteratively generated and optimized code, and the researchers tracked not just final throughput but also runtime behavior, failure modes, and sensitivity to prompt engineering.

The results reveal a clear cross-architecture divergence. On GPUs, more advanced LLMs achieved substantially higher throughput—but only when prompts were meticulously crafted; even small changes could tank performance. On Cerebras, the primary hurdle was simply generating executable programs under the unconventional spatial execution model, where thousands of processing elements run in lockstep. The agents proved far more effective on SZx (modular kernels with clean interfaces) than on SZp (tightly coupled bit-level pipelines), where structural dependencies stalled optimization. The authors conclude that HPC coding agent success on thread-based platforms doesn't automatically transfer to spatial accelerators, urging evaluation frameworks that capture both performance outcomes and architecture-specific robustness.