Reason-to-Transmit: Deliberative Adaptive Communication for Cooperative Perception
New AI framework improves multi-agent perception by 58% AP, using transformers to decide what data to share.
Researchers Aayam Bansal and Ishaan Gangwani have published a paper introducing Reason-to-Transmit (R2T), a new framework designed to solve a critical bottleneck in autonomous vehicle networks: efficient communication. Current Vehicle-to-Everything (V2X) networks are bandwidth-constrained, forcing agents like self-driving cars to be selective about what sensor data they share. Existing methods use reactive rules, like sending data only from high-confidence areas, but they don't reason about why a specific piece of information would be valuable to a receiving vehicle.
R2T addresses this by equipping each agent with a lightweight transformer-based module. This module performs a 'deliberative' analysis, considering three key factors: the agent's own local scene context, what it estimates its neighbors are missing (the information gap), and the available bandwidth budget. It then makes intelligent, per-region decisions on what to transmit. The system was trained end-to-end with a bandwidth-aware objective and evaluated in a multi-agent bird's-eye-view perception environment.
The results are significant. While any communication improved perception performance by about 58% Average Precision (AP) over no communication at all, R2T's advantages became clear in challenging conditions. Under high occlusion—where one vehicle's view is severely blocked—R2T's reasoning capability provided clear gains over nine other baseline methods, nearly matching the performance of an 'oracle' with perfect information. Furthermore, the framework proved robust, with all methods degrading gracefully even under simulated 50% packet drops. This indicates that while the design of the data fusion system is paramount, adding deliberative communication provides crucial extra performance in the most difficult real-world scenarios.
- Uses a lightweight transformer module to reason about what data to share, considering scene context, neighbor info gaps, and bandwidth.
- Tested against 9 baselines, showing 58% AP improvement over no communication and excelling in high-occlusion scenarios.
- Demonstrates robustness to real-world network issues, maintaining performance with up to 50% simulated packet loss.
Why It Matters
Enables more efficient and reliable perception for autonomous vehicle fleets, a key step toward safer, scalable self-driving systems.