Cohere Command A+ GGUF quantized for local inference via llama.cpp
Effortlessly run Command A+ on your own machine with new GGUF support.
Deep Dive
Just in: llama.cpp added support for Command A Plus and North Mini Code this weekend. Unsloth has GGUFs for North Mini Code, but since no up‑to‑date GGUFs for Command A Plus were available, one developer converted and quantized it themselves.
Key Points
- Command A+ (104B parameters) now available as GGUF quantizations for llama.cpp, custom-crafted by a community developer after llama.cpp added support.
- The quantized files reduce memory needs, enabling local execution on high-end consumer hardware (e.g., 48GB+ RAM for 4-bit).
- Unsloth also released GGUFs for North Mini Code, providing two new model quantization options in the same weekend.
Why It Matters
Enables privacy-preserving, offline access to Cohere's frontier model on personal hardware, democratizing LLM experimentation.