b8089
The new commit splits matrix multiplication dispatches to handle batch dimensions exceeding hardware limits.
Deep Dive
The ggml-org team released commit b8089 for their popular llama.cpp project. This update fixes a Vulkan backend overflow by splitting the `mul_mat` (matrix multiplication) operation into multiple dispatches when batch dimensions exceed the maximum workgroup count. This prevents crashes and allows for stable execution of larger AI model batches on GPUs using the Vulkan API, which is common for running models like Llama 3 locally.
Why It Matters
Enables more stable and efficient local AI inference, especially for developers running large language models on consumer hardware.