New Budget-Adaptive Routing Skips Weak Edge Models to Cut Latency by 30%

Traditional edge-cloud inference pipelines run a lightweight weak model on every frame before a router decides to offload to a stronger cloud model. This weak-conditioned design wastes computation when offloading is likely. In a new paper for ACM SIGCOMM 2026, Geng, Mohan, and Ott argue that this is suboptimal under varying budgets. They introduce a weak-skipping estimator that operates directly on raw pixels, requiring only 0.153 GFLOPs—about 29 times less than a standard weak detector (4.49 GFLOPs). This estimator predicts whether to offload without ever running the weak model, saving both time and energy when offload is certain.

However, neither weak-skipping nor weak-conditioned placement dominates across all operating conditions. So the authors propose budget-adaptive routing: an offline-tuned threshold system that selects the best placement based on the current offload budget. On the PASCAL VOC benchmark, their method traces the upper accuracy envelope of both placements across the full operating curve. Results show per-frame latency reductions of up to 19.1ms (about 30% improvement at rho=0.9). Remarkably, at some operating points, budget-adaptive routing achieves up to 1.7 percentage points higher mean average precision (mAP) than the strong cloud model alone—while using far less compute. The work will appear at the NAIC '26 workshop in Denver.