LipCache boosts edge image AI up to 1.65x with certified caching

Edge image classification services face a growing demand for low-latency, high-throughput inference. Existing semantic caching systems rely on empirical similarity thresholds that can silently misclassify inputs near decision boundaries. To solve this, Zhengzhe Xiang and colleagues propose LipCache, a certified semantic caching framework that works without modifying the deployed main model (MainNet). The system adds a lightweight network called GuardNet, which maps inputs into a low-dimensional feature space under a Lipschitz constraint. GuardNet then computes a per-sample certified reuse radius based on local classification margin and spectral norm of the classification head. At runtime, cached results are reused only if the query feature falls inside the certified reuse ball; otherwise, the query falls back to MainNet.

Across standard benchmarks—CIFAR, Tiny-ImageNet, and SVHN—LipCache achieves a measured speedup of up to 1.65x with limited end-to-end accuracy degradation, while all accepted cache hits satisfy GuardNet's certified-consistency condition. An enhanced GuardNet training recipe further improves cache hit rates in the Tiny-ImageNet multi-class extension while maintaining a 100% certified-consistency rate. Unlike threshold-based methods, LipCache converts cache hits into geometric certification decisions with explicit theoretical boundaries, eliminating silent misclassifications. This makes cache-assisted inference reliable for edge deployments where model fallback costs are high. The paper is available on arXiv (2608.13144) and positions LipCache as a practical path to dependable, low-cost edge vision.