DOPA framework boosts LLM in-context learning with OOD proxies

A new paper accepted at ACL 2026 tackles a critical weakness in in-context learning for large language models (LLMs): performance drops when the target domain is out-of-distribution (OOD) and its data is inaccessible. The authors—Hao Xu, Rite Bo, Fausto Giunchiglia, Yingji Li, and Rui Song—introduce DOPA, a demonstration retrieval framework that uses an OOD proxy to approximate the unknown target domain. DOPA guides the selection of informative and distributionally similar examples from a known source domain, addressing the challenge of evaluating target distribution without direct access. It further incorporates a Mahalanobis distance-based global diversity constraint to ensure selected demonstrations are varied, reducing overfitting to narrow patterns. Experimental results across multiple LLMs and tasks show that DOPA significantly boosts robustness in severe OOD scenarios.

DOPA's key innovation lies in using proxy-based evaluation to bypass the need for target domain data, making it practical for real-world deployments where target distributions are unpredictable. The Mahalanobis distance constraint measures the overall diversity of the retrieved set, preventing redundant examples. The framework was validated on classification and generation tasks, outperforming naive retrieval and other baselines. With LLMs becoming ubiquitous in dynamic environments, DOPA offers a lightweight yet effective way to maintain performance without costly fine-tuning. The code is publicly available, promising quick adoption by the research community.