New AI training method cuts tool overuse by 34% in agentic models

Training agentic language models to decide when to call tools, consume tool responses, or answer directly is a critical challenge. A common strategy is multi-teacher on-policy distillation (OPD), where one teacher specializes in tool calls and another in direct responses. However, researchers from Jiabin Shen, Guang Chen, and Chengjun Mao discovered that this approach can induce a hidden behavior shift: the student model learns to over-call tools, even on examples where a direct answer would suffice. Aggregate loss metrics fail to capture this problem, because the tool-call teacher's tokens don't receive disproportionate exposure overall. Instead, the team identified a "behavior leverage imbalance" — certain structural tokens like <tool_call> and function names exert disproportionate control over the model's generation mode at the token level.

To address this, the authors introduce Soft Clamp, a per-token divergence calibration method that dynamically compresses extreme token-level Jensen-Shannon divergence while preserving nonzero gradients. On the APIGen-MT benchmark, Soft Clamp reduced over-calling from 13.7% (with vanilla generalized knowledge distillation) to 9.0% without sacrificing decision accuracy. In a BFCL multi-turn diagnostic, the method also decreased tool-call loops and repeated calls compared to other GKD variants. The findings underscore that multi-teacher OPD must monitor where teacher signals act, not just their aggregate magnitude. This work has practical implications for reducing unnecessary API calls and improving the reliability of agentic AI systems.