New active learning algorithm cuts annotation costs for text-to-SQL systems

Few-shot example retrieval is the dominant method for grounding large language models (LLMs) in domain-specific text-to-SQL systems, but the quality of the annotated example bank directly governs accuracy. Expert annotation is prohibitively expensive, yet traditional active learning fails when applied to this problem due to three critical challenges: heteroscedasticity (annotation reliability varies per query), strict spatial diversity requirements across semantic topics (partition matroid constraints), and unknown true covariance structure of the embedding space (misspecification).

Arash Pourhabib formalizes the active selection of these examples as a constrained experimental design over the intrinsic, low-dimensional manifold of semantic query embeddings. He proposes a stratified greedy algorithm that maximizes a heteroscedastic mutual information objective, and proves that this objective remains submodular and approximately monotonic, yielding a theoretical constant-factor approximation guarantee. A spectral bound shows that the guarantee degrades gracefully even when the surrogate kernel differs from the true data-generating process. Empirical results demonstrate that the strategy significantly reduces labeling effort while maintaining high text-to-SQL retrieval accuracy, making it highly practical for resource-constrained deployments.