b8267

The open-source llama.cpp project, maintained by ggml-org, has released a significant update with commit b8267. This patch fundamentally changes how the software's Metal backend handles GPU command buffer failures on Apple's macOS and iOS platforms. Previously, a failure—like an iOS app losing GPU access when sent to the background, or a macOS system disconnecting an external GPU (eGPU)—would trigger a GGML_ABORT call. This "fatal error" would forcibly terminate the entire host application, a disruptive experience for users and developers.

The new implementation replaces this abrupt termination with graceful error handling. When a Metal command buffer fails, the backend now sets an internal error flag and returns a GGML_STATUS_FAILED status code, aligning its behavior with the existing graph_compute function. This allows the calling application to be notified of the failure and respond appropriately without crashing. The backend enters a defined error state, and all subsequent inference attempts immediately return the failure status until the backend is properly recreated. The commit also ensures that any allocated Metal resources are correctly released on the error path, preventing memory leaks.

This is a crucial stability improvement for any application using llama.cpp for on-device AI inference on Apple devices. It enables developers to build more robust applications that can survive common system events without crashing, improving the user experience for mobile AI apps, creative tools, and other software leveraging local LLMs. The fix is part of the continuous maintenance of the widely adopted 97.6k-star project, which provides efficient inference for models like Llama 3.