Safe Multi-Agent Deep Reinforcement Learning for Privacy-Aware Edge-Device Collaborative DNN Inference

A research team from unnamed institutions has published a paper proposing a novel framework for secure and efficient AI inference at the network edge. The work, "Safe Multi-Agent Deep Reinforcement Learning for Privacy-Aware Edge-Device Collaborative DNN Inference," addresses the critical challenge of running complex deep neural networks (DNNs) on resource-constrained devices like smartphones and IoT sensors. The core innovation is a collaborative system where an AI model's computational workload is intelligently split—some parts run locally on a user's device to protect sensitive data, while other, less sensitive computations are offloaded to more powerful edge servers. This adaptive model partitioning is managed not by a single controller, but by multiple AI agents working together, formulated as a Constrained Markov Decision Process (CMDP) to handle the trade-offs between speed, battery life, and privacy.

The team's solution is the Hierarchical Constrained Multi-Agent Proximal Policy Optimization with Lagrangian relaxation (HC-MAPPO-L) algorithm. This mouthful describes a sophisticated safe reinforcement learning framework that enhances the popular Multi-Agent PPO (MAPPO) method. It uses a three-layer hierarchical policy: one layer decides which AI models to deploy, another handles associating users with servers and splitting the models, and a third allocates computational resources like CPU and bandwidth, using an attention mechanism. The "Lagrangian relaxation" component is key—it acts as an adaptive penalty system to strictly enforce long-term performance constraints, ensuring inference tasks are completed within a guaranteed time window. Experimental results show HC-MAPPO-L consistently meets these stringent delay guarantees while achieving a superior balance, reducing overall system energy consumption and privacy leakage compared to existing baseline methods, making it a promising approach for the future of distributed, privacy-preserving AI.