A Task Decomposition and Planning Framework for Efficient LLM Inference in AI-Enabled WiFi-Offload Networks
Decomposes tasks for collaborative edge execution, boosting reward by 80%.
Researchers Mingqi Han and Xinghua Sun from arXiv have introduced a task decomposition and planning framework for efficient LLM inference in AI-enabled WiFi-offload networks. The framework addresses the challenges of heterogeneous model capabilities, wireless contention, and uncertain task complexity by allowing each task to be executed locally, directly offloaded to an edge access point, or decomposed into multiple subtasks for collaborative execution across local and edge nodes. This approach enables more accurate estimation of execution quality and latency on heterogeneous nodes, leading to optimized subtask assignment, execution, and aggregation under communication, queuing, and computation constraints.
The proposed framework achieves a better latency-accuracy tradeoff than local-only and nearest-edge baselines, reducing average latency by 20% and improving overall reward by 80%. Additionally, the distilled lightweight planner approaches the performance of the large teacher model while remaining more suitable for practical edge deployment. This work is particularly relevant for resource-constrained wireless devices that require LLM services, as it enables efficient inference offloading in multi-user multi-edge WiFi networks.
- Framework uses LLM-based planner for task decomposition and subtask difficulty estimation.
- Achieves 20% lower latency and 80% higher reward compared to local-only and nearest-edge baselines.
- Distilled planner retains performance of large teacher model for practical edge deployment.
Why It Matters
Enables efficient LLM inference on resource-constrained devices, reducing latency and improving accuracy in WiFi-offload networks.