Ollama Adds Cloudflare's Clef AI That Can See Images

Ollama, a popular free tool for running AI models on your own computer, just added support for two new models from Cloudflare: Clef and Clef Flash. These are open-source, meaning anyone can use or modify them. What makes them special is that they can understand both text and images. So instead of just typing a question, you can show the AI a picture and ask about it. For example, you could upload a photo of a plant and ask what species it is, or show a chart and ask what it means.

The two models differ in size: Clef is 27 billion parameters, which is like having a bigger brain but needs more computing power. Clef Flash is 9 billion parameters, smaller and faster, but maybe less accurate on tough tasks. Both are multimodal, meaning they can handle images alongside text. This is a big step for open-source AI because until now, most image-capable models were only available through paid cloud services. Now you can run them locally, which means your data stays private.

To use them, you'll need Ollama installed on your computer. Then you can download the models and start experimenting. The update is version 0.35.1, and it's available now. Since these are open-source, developers can also build apps on top of them. This could lead to new tools for everything from education to accessibility, like helping visually impaired people understand images.

The catch? Running these models locally requires a decent computer, especially for the larger Clef model. You'll need a good graphics card and plenty of memory. Also, while they're powerful, they might not match the accuracy of top-tier cloud models like GPT-4. But for many everyday tasks, they're more than enough. And the privacy benefit is huge: your images and text never leave your device.