llama.cpp VRAM tricks: Free 1GB+ for larger context windows

For tech professionals running large language models locally on GPU, VRAM is the ultimate bottleneck. A recent Reddit thread on llama.cpp highlights practical strategies to reclaim memory and extend context windows. The user, running a 27B parameter Qwen model (Q4_0 quantization) on an RTX 3090 via eGPU with 150k token context, shares several parameters that actually free up space — and some that don't.

The biggest win: --no-mmproj-offload, which moves the vision projector (if your model supports vision) to CPU, freeing about 1GB of VRAM with a tiny performance hit. Next, KV cache quantization via --cache-type-k and --cache-type-v can cut memory allocation by 50–75% (e.g., using q4 ), and thanks to recent attention rotation improvements, quality drop is negligible — especially since users can then fit a larger base model. The user also notes that --flash-attn (now default) is essential for quantized KV caches. Conversely, parameters like --ctx-checkpoints, --parallel, and --fit-target did not help. For draft models, --spec-draft-n-max set to 2 balances speed and memory. These tips let users push context beyond 150k on consumer GPUs.