New 'Deopt-Reopt' method improves LLM-based C++ to CUDA porting – sometimes

Porting high-performance computing (HPC) code from CPU to GPU often stumbles because CPU-oriented optimizations confuse LLMs tasked with generating CUDA. Researchers propose a Deopt-Reopt workflow: first deoptimize (simplify) the input C++ to remove CPU-specific tweaks, then reoptimize for CUDA. They compared this to direct translation on 12 HPC kernels using two LLMs (gpt-oss-120b and qwen-3-235b) in single-shot and iterative settings. In single-shot mode, Deopt-Reopt was significantly faster (after BH-FDR correction) in 5 of 18 testable cases—most notably for conv2d—but direct translation was faster in 3 cases. An exploratory control that equalized LLM calls (Direct-3) left Deopt-Reopt ahead in only 4 of 19 testable cases, with Direct-3 winning 4.

In iterative settings, repeated generation and repair narrowed the performance gap, especially for the gpt model, though qwen-3-235b retained large Deopt-Reopt advantages on conv2d, ddgemm, and bgemm. Deopt-Reopt also affected feasibility: some kernels that rarely compiled under direct translation saw sharply higher success rates, while others dropped. Overall, the technique is effective but non-universal. The gains depend heavily on the specific kernel, the LLM model, the search budget for iterations, and the success rate of compilation. For engineers, this means a blanket deoptimization step isn't a magic bullet—context matters, and iterative refinement may be necessary.