Meta's Muse Glimmer 30B runs 1M context with perfect retrieval on DGX Spark
A 30B model just hit 1M tokens on a dual-node cluster—every needle test passed.
Meta's Muse Glimmer 30B, released only a day prior, has been successfully pushed to a 1M-token context window on local hardware. The tester used a 2× NVIDIA DGX Spark cluster (GB10, 128 GB unified memory each, ~273 GB/s bandwidth) with llama.cpp master build and CUDA sm_121 + GGML_RPC support. Context was extended from the trained 131,072 tokens using YaRN rope scaling at 2×, 4×, and 8× factors, with the model's KV cache override set to 1M. The official GGUF file (K-Quant-Dynamic, ~18.3 GiB) was paired with the vision mmproj and DFlash block-diffusion drafter.
The results validate the model card's "131,072+" hint. Needle-in-haystack retrieval scored 3/3 at every rung: 97K tokens native, 188K (1.4×), 415K (2.9×), and 832K (6.35×, deepest needle ~749K). Speed on a single Spark reached ~10.5 tok/s baseline decode and 36–38 tok/s with DFlash (matching Meta's claimed 3.1× speedup), with prefill at ~700 tok/s short-context and ~390 tok/s deep into 832K prompts. Four concurrent streams yielded ~57 tok/s aggregate. RPC splitting across both nodes was ~30% slower at 25–28 tok/s, noted as fun but inefficient for a 20 GB model. Coding tests passed 7/7 and vision worked via the official mmproj.
- Context extended from 131K to 1M tokens using YaRN scaling on a 2× DGX Spark cluster with llama.cpp.
- Perfect 3/3 needle-in-haystack retrieval at 832K tokens; DFlash boosts decode to 36–38 tok/s from ~10.5 baseline.
- Passed 7/7 coding tests and vision tasks; total load with weights, drafter, vision, and 1M KV ≈ 60 GB on one node.
Why It Matters
Proves 1M-token context is practical on a single workstation, unlocking long-document agentic AI locally.