llama.cpp server now hot swaps models in under 30 seconds

A recent Reddit post from user /u/Chuyito highlights a major improvement in the llama.cpp server: the ability to hot swap between large language models in under 30 seconds. Previously, switching models required a full server restart and could take minutes, especially when using PyTorch-based inference. Now, with a simple configuration preset file (e.g., qwen36-models.ini), users can define multiple models (like Qwen and Gemma) that the server can load on demand without downtime.

The implementation is clean and integrates smoothly with open-source UIs like openwebui and hermes. The poster shows a Docker command using `ghcr.io/ggml-org/llama.cpp:server-cuda13` with a `--models-preset` flag, plus a `--models-max 1` setting to limit concurrent models to one and save GPU memory. For bare-metal deployments, the same `./llama-server` binary with the `--models-preset` argument works. This feature is a game-changer for developers and researchers who need to rapidly test different models in a local or self-hosted environment, reducing friction and enabling faster iteration cycles.