FinCacheServe cuts LLM calls 53% for mutable document RAG

Enterprise RAG systems frequently re-execute semantically identical queries against mutable documents (e.g., financial filings, legal evidence). Existing caching approaches either ignore document version changes or require costly recomputation. FinCacheServe, introduced in a new arXiv paper by Lingteng Zeng and Yifan Jin, treats each generated answer as a serving object indexed by enterprise intent and guarded by document versions, evidence fingerprints, tool fingerprints, model identity, and decoding configuration. This ensures dependency consistency—no stale answers are served even as underlying documents change.

Implemented on vLLM with Qwen2.5 models, FinCacheServe was evaluated on SEC-derived financial-document workloads. On a 2,230-request 7B parameter trace, it skipped 53.27% of LLM calls with zero dependency-stale outputs. Across three 32B operator-suite seeds, it skipped 53.31% of 544 requests—a significant improvement over versioned semantic caching (38.97%) and grounded-style reuse (22.43%). Capacity and SLO replays demonstrate oracle-bounded cache management, 100k-entry transactional metadata, and 44.30% lower estimated energy consumption per dependency-fresh 2-second SLO success compared to versioned semantic caching. The system is particularly valuable for regulated industries where document mutability and compliance demand both cost efficiency and answer freshness.