PORTS method boosts LLM tool selection with preference optimization
New training technique fine-tunes retrievers to pick the right tools for LLMs
Large language models (LLMs) increasingly rely on external tools to handle complex tasks, but managing large tool collections remains a challenge. Existing retrieval methods for pre-selecting tools are often misaligned with LLMs because they are trained separately. PORTS (Preference-Optimized Retrievers for Tool Selection) solves this by introducing an odds ratio preference optimization approach. It leverages a perplexity-inspired preference signal from a frozen LLM to fine-tune the retriever, directly optimizing the correlation between tool selection probabilities and downstream task performance. Additionally, it enforces a contrastive semantic loss on documentation strings to improve tool discrimination.
Extensive experiments across six datasets, two encoder models, and three LLMs demonstrate that PORTS significantly improves tool selection accuracy over baselines. The method is computationally efficient, requiring minimal extra training resources, and generalizes well to unseen queries and new tools. This practical alignment process makes PORTS valuable for real-world applications where tool sets evolve frequently, such as automated workflows, API orchestration, and agent systems. The work was presented at EMNLP 2025.
- Uses perplexity from a frozen LLM as a preference signal to guide retriever training.
- Optimizes the correlation between tool selection probability and downstream task performance.
- Validated on 6 datasets, 2 encoder models, and 3 LLMs with consistent accuracy improvements.
Why It Matters
Improves LLM tool integration for complex tasks with minimal computational cost and strong generalization.