New DRL Framework Cuts Edge-Cloud Latency by 20.8% with 50% Faster Convergence

A new research paper from Vo Phi Son, Van-Dinh Nguyen, Ngoc Hung Nguyen, Trinh Van Chien, and Symeon Chatzinotas introduces a two-timescale multi-layer deep reinforcement learning (DRL) framework with a latent action space (2T-MDRL-LA) to tackle the joint service placement, computational delegation, and power control (JSCP) problem in hierarchical edge-cloud computing (HECC) systems. The JSCP problem is a mixed-integer nonconvex NP-hard optimization due to strong coupling between discrete (service placement, delegation) and continuous (power control) variables. The authors exploit the inherent difference in decision dynamics by decomposing the problem into long-term system configuration and short-term resource allocation subproblems. The framework then uses a variational autoencoder to compress the high-dimensional combinatorial action space into a latent representation, making DRL training tractable.

Simulation results demonstrate that 2T-MDRL-LA effectively adapts to dynamic network conditions and achieves near-optimal performance compared to branch-and-bound solutions. Specifically, it reduces average end-to-end latency by up to 20.8% and improves resource utilization by 13% over a baseline without computational delegation. Additionally, the framework converges approximately 50% faster than conventional proximal policy optimization (PPO). These improvements are critical for real-time applications such as autonomous driving, smart cities, and industrial IoT, where latency and resource efficiency are paramount. The paper has been submitted for publication and is available on arXiv (2607.18288).