TetraRL: Self-adaptive runtime balances real-time, reward, RAM & energy for on-device DRL
A single policy can switch optimization goals on the fly with near-zero overhead.
TetraRL, introduced by Zexin Li, Soheil Shirvani, and Cong Liu, tackles a growing pain in autonomous robotics: running deep reinforcement learning (DRL) directly on resource-constrained edge hardware. Unlike cloud-reliant systems, on-device DRL must simultaneously hit real-time deadlines, maximize task reward, keep RAM usage in check, and conserve battery. Previous approaches either optimized each objective in isolation or used heavyweight multi-objective methods that risked violating constraints. TetraRL unifies these four goals—real time, reward, RAM, and reserve (energy)—into one optimization problem. Its self‑adapting controller, powered by a preference-conditioned RL algorithm, dynamically navigates trade-offs based on user-defined runtime preferences. Hardware‑aware dynamic voltage and frequency scaling (DVFS) and a runtime Override Layer enforce hard constraints without breaking the model’s decision loop.
Evaluated on NVIDIA Jetson AGX Orin and Orin Nano across diverse DRL environments, TetraRL showed it could balance all four objectives effectively. Remarkably, a single trained policy supports continuously switchable optimization goals at runtime, meaning the same robot could shift from prioritizing speed to prioritizing energy savings without retraining. The framework introduces negligible overhead (less than 5% in most tests), making it practical for real‑time embedded systems. This work extends earlier RTSS'23 and RTSS'24 papers and is available on arXiv (2606.13891). For engineers building autonomous drones, rovers, or smart cameras that need to adapt on the fly, TetraRL offers a ready‑to‑use architectural blueprint.
- TetraRL unifies four objectives—real-time latency, task reward, RAM usage, and energy consumption—into a single optimization framework for on-device DRL.
- Uses a preference-conditioned RL controller with DVFS (dynamic voltage/frequency scaling) and an Override Layer to enforce constraints; tested on NVIDIA Jetson AGX Orin and Orin Nano.
- Single trained policy can switch between different optimization goals at runtime with negligible overhead (~5%), enabling adaptive behavior without retraining.
Why It Matters
Enables autonomous robots to dynamically trade off speed, accuracy, memory, and battery life—all on resource‑constrained hardware.