b8190

The open-source project llama.cpp, maintained by ggml-org, has released a significant technical update (commit b8190) that patches a critical stability and performance flaw in its WebGPU backend. The core issue was that WebGPU, the modern web graphics API, imposes a hard limit of 65,535 workgroups per dispatch dimension. During large batch matrix multiplication (MUL_MAT) operations common in AI inference, this limit was being exceeded, causing operations to fail silently or, worse, lead to memory corruption. The fix restructures the compute shader dispatch logic to split work across two dimensions (X and Y), effectively bypassing the single-dimension bottleneck.

Technically, the update introduces a new `compute_2d_workgroups()` helper function and refactors three key shaders—`mul_mat_reg_tile.wgsl`, `mul_mat_subgroup_matrix.wgsl`, and `mul_mat.wgsl`—to reconstruct linear workgroup IDs from this 2D dispatch. Crucially, it also adds bounds checking to prevent 'over-dispatched' workgroups from accessing invalid memory, which was the root cause of the corruption. This enables stable and efficient inference with large batch sizes (tested from 128 to 1024), a common requirement for server-side processing or batched requests. The fix is a foundational improvement for running models like Llama 3 efficiently in browser-based and cross-platform applications using WebGPU.