RAMS: New runtime controller speeds edge AI 5.6x without sacrificing VRU accuracy
RAMS switches YOLOv8 models on the fly, boosting latency by 5.6x while protecting vulnerable road users.
RAMS (Resource-Adaptive and Detection-Conditioned Model Switching) is a new runtime controller designed for embedded edge perception, particularly object detection on resource-constrained hardware. Developed by researchers Kushal Khemani, Evan Leri, George Xu, and Amit Hod, RAMS addresses the fundamental trade-off between inference latency and detection quality under dynamic resource pressure. It monitors device metrics like CPU/GPU load and memory usage, calibrates switching thresholds from idle behavior, and dynamically selects among three resident YOLOv8 tiers—NANO (320px), SMALL (416px), and MEDIUM (640px)—without incurring model-reload latency. The system defines five switching policies, including two detection-conditioned variants that prevent aggressive downgrades immediately after detecting vulnerable road users (VRUs), ensuring safety-critical objects are not missed during resource-constrained moments. Additionally, RAMS introduces the VRU-Weighted Accuracy Score (SWAS), a scalar metric for offline policy comparison without ground truth labels, along with an oracle-bounded variant to separate detector circularity from genuine tier-retention benefits.
Across diverse edge hardware—Raspberry Pi 5, x86 laptops, and Jetson Orin ONNX/TensorRT deployments—RAMS operates over a 37x latency range using identical controller equations. Under heavy load on Jetson Orin TensorRT, the safety2 policy achieves a mean latency of just 3.41 ms, which is 5.6x faster than fixed-MEDIUM inference, while retaining 74% of its proxy accuracy through near-NANO operation with selective SMALL and MEDIUM locks during VRU-positive windows. Detection-conditioned switching improves SWAS by 25.4% under oracle scoring and 47.3% under detector-derived scoring compared to threshold-only policies under heavy load. However, live KITTI evaluation reveals per-tier VRU recall of only 24.2%, 41.2%, and 59.0% for NANO, SMALL, and MEDIUM respectively, showing that reactive overrides are fundamentally constrained by baseline detector recall. This work has significant implications for real-time autonomous systems, robotics, and edge AI where balancing power, latency, and safety is critical.
- RAMS dynamically switches between YOLOv8 NANO (320px), SMALL (416px), and MEDIUM (640px) models without reload latency across Raspberry Pi 5, x86, and Jetson Orin hardware.
- Detection-conditioned switching prevents model downgrades immediately after detecting vulnerable road users (VRUs), improving safety-weighted accuracy by up to 47.3% over threshold-only policies.
- On Jetson Orin TensorRT under heavy load, RAMS achieves 5.6x faster inference (3.41 ms vs fixed-MEDIUM) while retaining 74% proxy accuracy.
Why It Matters
RAMS enables safer, faster edge AI for autonomous vehicles and robotics by dynamically balancing latency and detection quality under resource constraints.