llama.cpp b10043 adds CUDA Virtual Devices support
llama.cpp v10043 enables GPU virtualization for multi-GPU LLM inference
Deep Dive
llama.cpp released b10043 on July 16, adding CUDA Virtual Devices support with options to disable NCCL path when virtual devices are used, label virtual devices in description, and include GPUx2 server CI jobs plus code refactor. The project has 121k GitHub stars.
Key Points
- Adds CUDA Virtual Devices support for partitioning GPUs into multiple logical devices
- Includes NCCL path disable when virtual devices are used to avoid distributed conflicts
- Ships 27 platform builds including CUDA 12/13, Vulkan, ROCm 7.2, and Apple Silicon
Why It Matters
Enables better GPU resource sharing and isolation for production LLM deployments on multi-GPU servers.