b8840

The ggml-org development team has launched llama.cpp version b8840, a significant update to the widely-used open-source inference engine for running large language models locally. This release dramatically expands hardware compatibility, most notably adding Vulkan graphics API support for both Linux (Ubuntu x64/arm64) and Windows platforms. Vulkan support provides an alternative to CUDA for GPU acceleration, potentially enabling better performance on AMD graphics cards and integrated GPUs. The update also includes updated CUDA libraries (versions 12.4 and 13.1 DLLs for Windows), new builds for the openEuler Linux distribution targeting Huawei's Ascend AI processors (310p and 910b), and continued support for Apple Silicon, Intel, Android, and various other architectures.

This release represents a major step toward truly portable AI deployment. By exposing the media_tag property through the server's /props endpoint, developers gain better control over model serving configurations. The expanded platform support means organizations can now deploy the same llama.cpp codebase across everything from mobile devices (iOS/Android) to data center servers with specialized AI accelerators. The inclusion of multiple backends—including ROCm for AMD, OpenVINO for Intel, SYCL for heterogeneous computing, and HIP for AMD/NVIDIA compatibility—demonstrates the project's commitment to hardware-agnostic AI inference. For developers working with models like Meta's Llama 3 or Mistral's offerings, this update reduces the friction of moving between development and production environments across different hardware stacks.