R2LM achieves 12.9x faster parallel generation with asymmetric context

Discrete diffusion language models (dLLMs) can generate tokens in parallel, promising major speedups over autoregressive (AR) approaches. But they face a fundamental design dilemma: bidirectional attention delivers strong generation quality but blocks KV caching, crippling batch-serving throughput; causal attention enables cached inference but loses all right-side context, degrading quality. A new paper from researchers Yuhang Chen et al. introduces Bifocal dLLMs, a paradigm that resolves this via asymmetric bidirectional context. Their instantiation, R2LM (Right-to-Left Mamba), uses standard causal attention for precise left-context (fully KV-cache compatible) alongside a lightweight reverse Mamba SSM sidecar that feeds in compressed right-side context without breaking cacheability.

In experiments with continued pretraining of Qwen3-1.7B on 60B tokens, R2LM delivered 2.4x to 12.9x higher throughput than bidirectional dLLMs and 1.9x to 2.9x speedup over AR baselines under batch serving with KV caching. It exceeded the causal-only baseline on most benchmarks and surpassed the bidirectional dLLM on average. The work makes parallel diffusion models practical for real-time inference, particularly in latency-sensitive and high-throughput production environments.