QLMIO framework cuts multimodal LLM latency by 58% in cloud-edge offloading

Deploying large multimodal models (LMMs) at the edge can reduce inference latency, but maintaining content quality while responding quickly is notoriously difficult. A new paper from researchers addresses this by introducing QLMIO (Quality-Latency Tradeoff-Aware MLLM Inference Offloading), a framework designed for cloud-edge continuum systems. The framework jointly optimizes generation quality and response latency, tackling the inherent unpredictability of MLLM inference. To validate their approach, the authors built a real-world cloud-edge testbed and collected MIOBench, a new benchmark dataset tailored specifically to the inference offloading problem.

The experimental results are striking: QLMIO reduces latency by up to 58.14% compared to baseline offloading methods, all while preserving the same task completion rate as executing every request solely on a powerful cloud server. This means edge devices can handle multimodal tasks (like analyzing images, audio, and text together) nearly as well as the cloud but much faster. The framework and dataset are open-sourced on GitHub, offering a practical foundation for future research in distributed LLM inference. The paper was submitted to IEEE Transactions on Mobile Computing in June 2026.