b8546
Latest commit patches critical quantization and Metal GPU issues for the new vision model.
The open-source project llama.cpp, maintained by ggml-org, has released a new commit (b8546) that addresses critical compatibility issues for running the DeepSeek-OCR model. The patch specifically fixes a bug in the quantization process for the 'v.patch_embd' layer and resolves errors related to unsupported 'im2col' (image to column) operations when the model is executed using Apple's Metal Performance Shaders on Macs with M-series chips. This is a targeted fix for users attempting to leverage the latest multimodal models locally.
DeepSeek-OCR is a vision-language model capable of reading and interpreting text from images, making it useful for document analysis and data extraction. The bugs prevented the model from running correctly on a significant portion of the llama.cpp user base—Apple Silicon Mac owners. By patching these Metal-specific issues, the llama.cpp team ensures broader hardware accessibility for cutting-edge AI, reinforcing the project's role as a crucial bridge between complex models and consumer-grade hardware.
- Fixes quantization bug in 'v.patch_embd' layer for DeepSeek-OCR.
- Resolves unsupported 'im2col' operations on Apple Metal GPU backend.
- Enables stable local execution of the vision model on Macs with M1/M2/M3 chips.
Why It Matters
Removes a key barrier to running advanced multimodal AI locally on Apple hardware, democratizing access.