Interacting safely with cyclists using Hamilton-Jacobi reachability and reinforcement learning

Carnegie Mellon University researchers Aarati Andrea Noronha and Jean Oh have developed a groundbreaking AI framework that addresses one of autonomous driving's most challenging problems: safe interaction with cyclists. The paper, originally completed in 2020 as part of Noronha's graduate thesis, presents a hybrid approach that combines Hamilton-Jacobi reachability analysis with deep Q-learning to create autonomous vehicles that can navigate around cyclists with both safety guarantees and optimal efficiency.

The technical approach is sophisticated yet practical. The system computes a value function as the solution to a time-dependent Hamilton-Jacobi-Bellman inequality, providing a quantitative safety measure for every possible system state. This safety metric then becomes a structured reward signal within a reinforcement learning framework, allowing the AI to learn optimal navigation strategies while respecting hard safety constraints. Crucially, the framework models the cyclist's latent responses to the vehicle's actions, incorporating disturbance inputs that reflect human comfort levels and behavioral adaptation—a key innovation that moves beyond treating cyclists as simple obstacles.

In context, this research addresses a critical gap in autonomous vehicle safety. While most AV systems focus on pedestrian and vehicle interactions, cyclists present unique challenges due to their vulnerability, unpredictable movements, and shared road space. The CMU team's approach represents a significant advancement over existing methods by providing formal safety guarantees rather than just probabilistic safety. The framework was evaluated through comprehensive simulation and comparison with both human driving behavior and state-of-the-art methods, demonstrating superior performance in balancing safety and efficiency.

The implications are substantial for the future of autonomous transportation. As cities worldwide push for more cycling infrastructure and micromobility adoption grows, autonomous vehicles must be able to safely coexist with cyclists. This research provides a mathematically rigorous foundation for that coexistence, potentially accelerating AV deployment in urban environments while protecting vulnerable road users. The framework's ability to model human comfort and adaptation also suggests applications beyond cycling to other complex human-machine interactions in transportation systems.