llama.cpp b10321 fixes Metal NORM/RMS_NORM bug for non-multiple-32 layer sizes
Apple Silicon GPU users get a silent correctness fix for 33, 132, and 260 channel norms.
llama.cpp, the ubiquitous C++ library for running LLMs locally, just shipped release b10321. The headline fix targets its Apple Metal backend, correcting a subtle mathematical error in the NORM and RMS_NORM kernels. When a layer's row length (ne00_t) wasn't a multiple of the simdgroup size (32), the threadgroup could be sized incorrectly, leaving partial sums unread in shared memory. This produced slightly off mean and variance values for the entire row — especially noticeable for odd channel counts like 33, 132, or 260.
The fix rounds ne00_t up to a whole number of simdgroups rather than dropping the clamp, keeping threadgroups as small as possible without idle lanes. Before the patch, test-backend-ops on an M3 Pro passed only 25/50 NORM and 26/51 RMS_NORM cases; after, both are fully green (50/50 and 51/51), and the entire backend ops suite passes 13943/13943. While mainstream model hidden sizes (e.g., 4096, 8192) are multiples of 32 and unaffected, custom architectures with atypical norm dimensions will now produce numerically correct outputs on Apple Silicon.
- Fix targets ggml_metal_op_norm, affecting both GGML_OP_NORM and GGML_OP_RMS_NORM on Metal.
- Bug triggered when row length is not a multiple of 32, e.g., ne00_t = 33, 132, or 260.
- Post-fix: all 13943 backend-ops tests pass on M3 Pro, including full NORM and RMS_NORM suites.
Why It Matters
For local LLM developers on Mac, this ensures custom model architectures compute normalization correctly, avoiding silent output corruption.