Developer Tools

llama.cpp v9830 adds offline flag for model cache check

New --offline flag lets you verify cached models without touching the network...

Deep Dive

The ggml-org/llama.cpp project has released version b9830, a minor but impactful update that improves offline model management. The headline feature is an exposed --offline flag for the llama download command. Previously, the flag existed internally but was not accessible via the CLI. Now developers can run a script to check whether a specific model is already cached locally and ready for serving—without any network I/O. This is especially useful for production setups that need to guarantee offline operation or avoid accidental bandwidth costs.

Alongside the feature, the release fixes a latent use-after-free vulnerability in the URL-task's on_done callback. The bug occurred because a block-scoped variable (first_path) was captured by reference and accessed after the block had ended. This fix improves stability for any operation that triggers URL fetching. The release also includes build artifacts for a wide array of platforms: macOS (Apple Silicon, Intel, KleidiAI), Linux (x64, arm64, s390x with Vulkan, ROCm, OpenVINO, SYCL), Windows (x64, arm64, CUDA 12/13, Vulkan, OpenCL, HIP), Android (arm64), and openEuler. This broad support makes llama.cpp one of the most portable LLM inference engines available.

Key Points
  • New --offline flag for llama download to check cache without network
  • Fixes a use-after-free bug in the URL-task callback (first_path captured by ref)
  • Supports 20+ platform/backend combinations including CUDA 12/13, Vulkan, ROCm, OpenVINO, and KleidiAI

Why It Matters

Offline model checks reduce network overhead and improve reliability for production LLM serving pipelines.

📬 Get the top 10 AI stories daily