DEPT tuning unifies query expansion and retrieval, beating baselines on BEIR

Large language models can both rewrite underspecified queries and encode passages as dense vectors, suggesting a single unified model for query expansion and retrieval. Until now, most systems relied on prompted expansions or separately trained modules, so the generated text was only indirectly aligned with the retrieval loss that actually judges it. Researchers from BUPT propose DEPT, which trains a single decoder-only LLM end to end: the same model produces the expansion, encodes the expanded query, and encodes candidate documents. This unified setting creates a moving-target problem—retrieval supervision should improve query-side expansion, but the same gradient also shifts the document embeddings that serve as retrieval targets.

DEPT solves this by constraining tuned document embeddings to stay close to their cached initial values, while allowing retrieval gradients to pass through straight-through decoding into the generator. The result is joint query–document movement reduced to query-side adaptation against approximately stable, whitened document embeddings. That stability supports index reuse—no need to re-embed the whole corpus after training—and enables online hard-negative mining during training. Experiments with Qwen3-4B-Instruct-2507 and LLaMA-3.2-3B-Instruct across five datasets from the BEIR benchmark show DEPT improves average retrieval quality over training-free, independently trained, and staged unified baselines. Ablations confirm the contributions of preservation, whitening, end-to-end expansion training, and online negatives. Code is publicly available.