Developer Tools

llama.cpp b10290 fixes mtmd audio crash with new ggml_build_forward_order

New llama.cpp release patches stale input bug in multimodal audio generation.

Deep Dive

llama.cpp, the popular C/C++ inference engine for LLMs, has shipped a new rolling release tagged b10290. The update centers on a subtle but critical fix in its underlying ggml tensor library: the addition of ggml_build_forward_order. Previously, ggml_build_forward_expand was used as an ordering hint to keep query, key, and value tensors together in the mtmd audio graph. However, that function also marks tensors and their ancestors for computation, which conflicts with ggml_build_forward_select's purpose of pruning unselected branches. As a result, when the GEN_WAV call ran, the unselected GEN_CODE branch was executed with an inp_code0 input that was never uploaded to the device, causing a get_rows bound assertion failure on CPU.

The new ggml_build_forward_order function solves this by inserting nodes into the graph without setting the compute flag, leaving the flag to be restored only when the branch is actually selected. The llama.cpp maintainers have switched the q/k/v hints in clip_graph::build_attn to use this new function, ensuring that audio graph construction no longer forces stale data into execution. This fix is especially relevant for developers running multimodal models on CPU, Vulkan, or other backends where memory uploads are explicit. The release also updates the AGENTS.md guidance for contributors and provides the usual matrix of prebuilt binaries across macOS, Linux, Windows, and Android. For AI engineers working with llama.cpp or building on ggml, this patch removes a tricky correctness issue that could surface as non-deterministic crashes during audio synthesis, making the library more reliable for production agents and edge device deployments.

Key Points
  • Added ggml_build_forward_order, a new function that inserts nodes without setting the compute flag
  • Fixes a CPU get_rows bound assertion in the mtmd audio graph where GEN_WAV executed the GEN_CODE branch with stale inp_code0
  • Updated clip_graph::build_attn to use the new ordering hint, preventing forced computation of unselected branches

Why It Matters

Reliable multimodal inference on CPU and edge devices requires avoiding stale-input crashes; this fix stabilizes audio generation in llama.cpp.

📬 Get the top 10 AI stories daily