LSP token savings debunked: costs up to 118% more for coding agents

A new paper from researcher Pengcheng Xu (arXiv:2608.13568) rigorously tests a widespread assumption in AI coding tools: that semantic retrieval via the Language Server Protocol (LSP) is more token-efficient than simple lexical grep. The study introduces a single metric—tokens-to-success—and a five-arm ablation to isolate the LSP's contribution. Preliminary results on Python and TypeScript repos using Claude Opus 4.8, Sonnet 4.6, and Haiku 4.5 show the answer is conditional and usually negative.

On symbol-named localization, LSP adds 6% to 118% token overhead, and models default to grep (0-6% LSP use). On reference-completeness, LSP buys precision but not savings; only Haiku 4.5 benefits. Multi-file renames expose the strongest gap: grep solves them perfectly, while a location-only LSP fails 75% by missing call sites. Even a complete, index-warmed LSP with inline references recovers most of the gap but cannot close it, because renames must touch comments and strings that semantic references exclude. The takeaway: adaptive routing by task class, model capability, and lexical noise beats an LSP-always approach.