New AI Trick Makes Models Faster and Cheaper

AI models have traditionally relied on 'mixture of experts' (MoE) — a system where a router picks a few specialized blocks (like mini-AI brains) to handle tricky questions. This works well for reasoning, but it’s slow because the router waits until it sees the problem before deciding what to do.

Qwen’s new Qwen4Exp system flips this script. It still uses MoE for deep thinking, but adds a second tool: n-grams. These are like a lookup table for common phrases or facts. Instead of re-calculating “New York” every time, the AI just grabs it from its n-gram table — fast and lightweight. Qwen4Exp keeps 125 billion parameters in MoE, 51 billion in n-grams (stored on your SSD, not RAM), and only activates 6 billion per question. Think of it like having a librarian (MoE) and a card catalog (n-grams) — one does the hard work, the other hands you the book instantly.

The catch? N-grams are great for recall but useless for reasoning. Too much reliance on them turns AI into a glorified search engine, spitting out facts without understanding. Tests show keeping n-grams to about 20–25% of total capacity gives the best balance. And since n-grams live on your SSD (the slower but bigger storage on your computer), they don’t slow down RAM like MoE does.

For users, this means AI tools could become faster, cheaper to run, and more responsive — especially on regular laptops. It’s less about raw power, more about smart design: using tools you already have, just in a smarter way.