EdgeCoInfer cuts MLLM edge costs by 76% with 100% task completion

Running multiple Multimodal Large Language Models (MLLMs) concurrently on edge devices has been a major challenge due to tight resource constraints and high concurrency. Researchers from Singapore University of Technology and Design and Southwest University introduce EdgeCoInfer, a hierarchical collaborative inference framework that tackles this head-on. The key innovation is a granularity-adaptive deployment scheme that co-optimizes inter-model functional module sharing and intra-model fine-grained partitioning. To solve the underlying Mixed-Integer Non-Linear Programming (MINLP) problem, they developed a Hybrid Evolutionary Hierarchical Reinforcement Learning (HE-HRL) paradigm. This combines a Genetic Algorithm for discrete model placement decisions with a Soft Actor-Critic (SAC) agent for continuous resource allocation. A feasibility-guided constructive execution mechanism—featuring a constructive cut-step decoder, pre-act pruning, and two-phase curriculum learning—helps navigate sparse feasible regions.

Experimental results are striking: EdgeCoInfer achieves 100% task completion in high-concurrency scenarios while slashing system cost by 76% and memory usage by 71.88% compared to existing baselines. The framework is designed for mobile and edge environments where multiple MLLMs need to run simultaneously—think real-time AR assistants, autonomous drones, or on-device chatbots that combine vision and language. By making on-device multimodal AI both feasible and efficient, EdgeCoInfer paves the way for more capable edge AI applications without sacrificing performance or draining resources.