llama.cpp b10144 fixes streaming bugs and route handling for slashed model names
Streaming breaks when model names contain slashes; this release patches it.
The latest llama.cpp release, b10144, addresses a critical bug where streaming endpoints failed when model names contained forward slashes—a common pattern in router-style model references like “ggml-org/...”. The root cause was that decoded path segments split before the conversation ID parameter was captured, causing stop and resume requests to never match the session. The fix moves the conversation ID to a query string parameter on both public routes and internal router-to-child hops, ensuring slashes survive encoding. Handlers remain unchanged since query and path params land in the same map. Regression tests now validate slashed model names.
Beyond the route fix, the release enhances reliability during model loading. Previously, a stop issued while the model was loading found nothing to cancel because the conversation was registered only after the autoload wait. Now conversations are registered before the wait, with a ticket that allows cancellation. If a page is reloaded while a model is loading, the pending request detaches from the client socket and reaches the child regardless, with the session buffer receiving the generation. The resume route returns a 503 status while the owner is loading, prompting the client to retry instead of dropping state. The WebUI persists the stream, polls on 503, and reattaches once the session exists. Additionally, after a page refresh the model load progress is now shown again, making the experience seamless.
- Fixed stream routes breaking when model names contain slashes (e.g., ggml-org/...) by moving conv_id to query string
- Conversations now survive client disconnect during model loading; pending requests detach and resume via 503 polling
- UI now displays model load progress after page refresh, preventing a blank state until first byte arrives
Why It Matters
Improves stability and user experience for llama.cpp server mode, especially with router-style model deployments.