Free AI Tool Llama.cpp Gets Major Update: Run Powerful AI on Your Own Devices
You can now run advanced AI on your laptop or phone without internet—here's why that matters.
Deep Dive
The server entrypoint was refactored (#31113). The page lists an attestations link and build targets for macOS/iOS, Linux, Android, Windows, and openEuler, plus UI, across CPU, Vulkan, CUDA, ROCm, OpenVINO, SYCL, OpenCL Adreno, and Snapdragon Adreno GPU/Hexagon NPU configurations. macOS Apple Silicon arm64 with KleidiAI enabled is marked DISABLED, and openEuler is marked DISABLED.
Key Points
- Llama.cpp now works on more devices, including Windows, Mac, Linux, Android, and even some phones with special chips.
- You can run AI without internet, keeping your data private and saving money on cloud services.
- Setup requires technical skills, and local AI models are less powerful than cloud-based ones, but it's improving.
Why It Matters
This makes private, offline AI more accessible, potentially saving money and protecting your data from big tech.