Geometry-Aware Decoding with Wasserstein-Regularized Truncation and Mass Penalties for Large Language Models
This new geometry-aware technique is beating all other decoding methods on major benchmarks.
Deep Dive
Researchers have introduced Top-W, a new geometry-aware decoding method for LLMs that uses Wasserstein distance over token embeddings to better balance creativity and coherence. It outperforms prior state-of-the-art decoding approaches by up to 33.7% on benchmarks like GSM8K, GPQA, AlpacaEval, and MT-Bench across three instruction-tuned models. The method improves both accuracy-focused tasks and boosts creativity in open-ended evaluations.
Why It Matters
This represents a significant, measurable leap in how we get LLMs to generate better, more reliable text.