ToolSense exposes AI's tool knowledge gap: retrieval ≠ understanding
New diagnostic reveals LLMs collapse by 64% on realistic queries despite benchmark success.
SAP's open-source ToolSense framework exposes a critical flaw in how large language models are evaluated for tool retrieval. Current benchmarks like ToolBench rely on verbose, fully-specified queries and constrained decoding—masking whether models genuinely understand tool functionality. ToolSense automatically generates three diagnostic benchmarks: a Realistic Retrieval Benchmark (RRB) with three ambiguity tiers, an MCQ probing benchmark, and a QA probing benchmark, all derived from any tool catalog.
When applied to ToolBench’s ~47k tools across five parametric model configurations, ToolSense reveals a stark knowledge-retrieval dissociation. On RRB queries, several configurations collapse by 50-64 percentage points compared to their fully-specified performance, falling below embedding-based baselines. Even models with strong retrieval scores score near-random on factual probes, indicating they memorize token paths without understanding tool semantics. The framework is available at github.com/SAP/toolsense.
- ToolSense introduces a Realistic Retrieval Benchmark (RRB) with three ambiguity tiers, causing some LLM configurations to drop 50-64 percentage points below fully-specified benchmarks.
- Despite strong retrieval on verbose ToolBench tests, several models scored near-random on factual MCQ probes, revealing a knowledge-retrieval dissociation.
- SAP open-sourced both the diagnostic framework and the diagnostic benchmarks for ToolBench's 47k tools at github.com/SAP/toolsense.
Why It Matters
For AI agents relying on tool catalogs, realistic evaluation prevents overconfident deployment and reveals hidden failure modes.