llama.cpp b9665 adds --offline flag for benchmarking
Benchmark LLMs without internet with new --offline flag
llama.cpp, the popular open-source C/C++ library for running large language models locally, has released version b9665. The key addition is the --offline flag for the benchmark tool, which enables users to conduct performance tests without an internet connection. This is particularly useful for developers working in air-gapped environments or who want to ensure consistent, reproducible benchmarks without external dependencies. The release was tagged by GitHub Actions and signed with a verified GPG key, ensuring authenticity.
The release includes pre-built binaries and XCFramework for a wide range of platforms: macOS (Apple Silicon and Intel), Linux (x86/ARM with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Android (ARM64), and Windows (x64/ARM64 CPU, CUDA 12 and 13, Vulkan, SYCL, HIP). The addition of the --offline flag simplifies benchmarking for edge deployments and local-only workflows. Developers can now run consistent evaluations without relying on network calls, making it easier to compare performance across different hardware configurations.
- New --offline flag added to llama.cpp benchmark tool for internet-free testing
- Release b9665 includes builds for macOS, Linux, Windows, Android, and multiple GPU backends
- Signed commit ensures authenticity; enables reproducible benchmarks in air-gapped environments
Why It Matters
Enables AI developers to benchmark LLM inference locally without internet, improving security and reproducibility.