Why Query Position in Diffusion LLMs Is a Hidden Threat — Auto-ICL Fix Revealed
Where you put your query in diffusion models can make or break results—researchers have a fix.
A new study from Zhengheng Li and colleagues uncovers a critical hidden variable in diffusion large language models (dLLMs): query placement. Unlike autoregressive (AR) models constrained by unidirectional attention, dLLMs use bidirectional attention, offering spatial flexibility for where to put the query in in-context learning (ICL). However, current practice blindly copies AR-style trailing-query templates, leading to severe positional bias. The researchers show that positional variance impacts generation quality as much as the semantic quality of the examples themselves. This stems from a "recency effect" in attention flow and task-dependent shifts in decoding trajectories.
To fix this without retraining, the team introduces Auto-ICL. Traditional confidence metrics (like single-step C_decoded) fail in dLLMs. Instead, they propose Average Confidence (C̄), which tracks the iterative decoding process. Auto-ICL uses this metric to dynamically route queries to optimal positions, training-free. Across diverse reasoning and perception benchmarks, it robustly approaches oracle performance. This paper establishes foundational spatial ICL baselines for dLLMs, a crucial step as diffusion models gain traction as alternatives to AR architectures.
- Query position in dLLMs has as much impact on generation quality as the semantic quality of ICL examples.
- Auto-ICL uses a novel metric, Average Confidence (C̄), to track decoding dynamics and adapt placement without training.
- The training-free method approaches oracle performance across heterogeneous reasoning and perception tasks.
Why It Matters
A practical, training-free optimization for diffusion LLMs that could significantly boost their reliability and usability in real-world applications.