Meta's Muse Glimmer powers on-device agentic AI via ExecuTorch, 128K context

Meta has introduced Muse Glimmer, a 30-billion-parameter open-weight model distilled from its larger Muse Spark model, designed specifically for on-device agentic AI workflows. To make this practical, ExecuTorch—Meta's deployment framework—adds end-to-end support for running Muse Glimmer on NVIDIA GPUs and Macs with Apple silicon. Unlike other local AI frameworks that rewrite models in non-Python languages, ExecuTorch lets engineers implement models and decoding strategies in PyTorch, then export them directly. The framework handles backend-specific lowering, including Triton on CUDA and MLX-native or custom Metal on Apple silicon, with ahead-of-time compilation optimizing the entire execution path, not just individual ops.

Muse Glimmer ships with several advanced capabilities: text and image inputs, direct GGUF export, native K-quant execution, and 128K+-token context length. It also supports DFlash, a parallel diffusion-based speculative decoding algorithm that reduces latency. Meta has published prebuilt PTE (serialized ExecuTorch) artifacts on Hugging Face for both CUDA and Metal backends, with options for text-only or text-plus-image and with or without DFlash. Developers can download these or build their own from GGUF checkpoints using ExecuTorch's export stack. The runtime includes a server for agentic use cases, enabling tools like Pi to call local models for actions such as reading files, running bash, and editing code—all on-device with low latency.