ros2-fast-stereo brings real-time stereo depth to ROS2 with TensorRT-accelerated FastFoundationStereo
A new open-source ROS2 wrapper enables real-time stereo depth estimation using TensorRT-accelerated FastFoundationStereo model.
PraveenElango has released ros2-fast-stereo, an open-source ROS2 (Jazzy) workspace that provides a real-time stereo depth estimation pipeline using a TensorRT-accelerated port of NVIDIA's FastFoundationStereo model. The project addresses a gap in deploying foundational stereo depth models on arbitrary camera rigs by supporting seven projection models (Pinhole, Double Sphere, fisheye polynomial, Kannala-Brandt 8, equirectangular, Mei, and Omni) through a unified pixelToRay/rayToPixel interface. Crucially, it allows stereo pairs to consist of wide-FOV, non-parallel cameras whose optical axes point in different directions, as long as their fields of view overlap—the rectifying rotation is derived automatically from calibrated poses.
The pipeline keeps all processing on the GPU: GPU remap, normalization, TensorRT stereo-matching inference, and disparity-to-depth conversion, with data crossing host/device boundary only once per frame in and out. It also supports multiple overlapping stereo pairs simultaneously; a quad-stereo configuration (4 pairs, 8 cameras) runs through a single batch-4 TensorRT engine for omnidirectional depth coverage. Performance benchmarks (at 320x736 resolution) show 35ms inference for single stereo on an RTX 5000 Blackwell, 140ms for quad stereo, and 128ms on a Jetson AGX Orin. The repo includes a VS Code Dev Container with ROS2 Jazzy, CUDA 13.2, and TensorRT 10.16 for quick setup.
- Supports 7 camera models including fisheye and omnidirectional types, plus non-parallel stereo pairs.
- Entire pipeline runs on GPU (CUDA, NPP, TensorRT) with single data transfer per frame; 35ms inference on RTX 5000 Blackwell.
- Quad-stereo configuration (8 cameras) provides omnidirectional depth at 140ms per frame using batch-4 TensorRT engine.
Why It Matters
This project makes state-of-the-art stereo depth estimation practical for real-time robotics and autonomous systems with flexible camera setups.