LLM-Driven Intent-Based Privacy-Aware Orchestration Across the Cloud-Edge Continuum

Researchers from Monash University and the University of Melbourne propose a dynamic pipeline reconfiguration system for LLM serving. It enables online adjustment of deployment configurations on heterogeneous GPU clusters (like NVIDIA A100 and L40s) to adapt to changing workloads. The method incurs less than 50ms of service downtime and introduces under 10% overhead on key performance metrics (TTFT/TPOT), allowing serverless platforms to optimize resource use for diverse AI inference jobs without significant interruptions.