Cohere's AI Models Now Work with Free Tool llama.cpp
Run powerful AI on your own laptop without paying cloud fees.
Deep Dive
Nick from Cohere says that after users asked for llama.cpp support, Cohere’s team recently merged llama.cpp support for the vision encoder of North Micro Vision Instruct. According to the post, that means all of Cohere’s latest model architectures are now compatible with llama.cpp; the LLM side uses the same architecture as North Mini Code and Command A+, which was already supported.
North Micro Vision is being used as the base for several new Cohere releases. Cohere also says Cohere Transcribe is now available in Unsloth and Unsloth Studio for fine-tuning, and describes it as an STT model.
Key Points
- Cohere's models now work with llama.cpp, a free tool for running AI locally.
- You can use Cohere's AI without cloud fees and keep your data private.
- The update includes vision models, so you can analyze images on your own device.
Why It Matters
Save money and protect privacy by running powerful AI on your own computer.