New TSDS framework cuts edge LLM reasoning compute by up to 73%

LLM agents using the ReAct paradigm are powerful for complex multi-step tasks like multi-hop QA and robot control, but edge deployment demands tight reasoning budgets and reliability. The new TSDS framework solves this by integrating two key mechanisms: a 'convergence probe' that stops on-device reasoning once the intended action stabilizes, and a 'perplexity-based deferral rule' that escalates uncertain actions to a cloud model. Both are jointly calibrated using a multi-objective Learn-Then-Test (LTT) procedure, offering finite-sample guarantees on expected episode reward and cloud-call rate.

Evaluated on four benchmarks (GSM8K, HotpotQA, MBPP, and a household robot task), TSDS cuts per-episode thinking compute by 43-73% over deferral-only baselines while maintaining certified guarantees. This is a significant step toward practical edge AI—enabling resource-constrained devices to run LLM agents efficiently without sacrificing reliability, by offloading only when uncertainty is too high to act safely.