Self-calibrating cross-camera homography for real-time ghost prediction in multi-camera person tracking[P]
This open-source system re-learns camera mappings in under 0.1ms to eliminate track loss.
Overwatch tackles the ghost prediction problem in multi-camera person tracking: when camera A loses a person but camera B still sees them, naive linear extrapolation fails because cameras have different coordinate systems. The system builds a 3x3 homography matrix H mapping camera B's pixel space to camera A's using foot-point correspondences from 64-dim HSV appearance descriptors (L2-normalized, EMA-smoothed at alpha=0.3). After 4+ pairs, cv2.findHomography() with RANSAC estimates H. The system auto-relearns every 5 new pairs and flushes H if reprojection error spikes (indicating camera movement).
Three fallback paths ensure robustness: Path A (H-PROJ, green) uses homography projection for highest accuracy; Path B (EXTRAP, red) provides pixel extrapolation with adaptive budget (min(250px, 80+40*t)); Path C (WORLD, orange) uses world-coordinate pinhole projection from a fused 3D Kalman state. Costs are minimal: homography re-estimation under 0.1ms, per-prediction projection under 0.001ms. Tracking uses Hungarian assignment with 0.6*IoU + 0.4*cosine appearance cost, falling back from DeepSORT to Hungarian to centroid. Sensor trust weighting and Kalman measurement noise scaling handle inconsistent observations. Limitations include ground-plane assumption breaking for steep camera angles and weak Re-ID with HSV histograms for similar clothing.
- Uses 64-dim HSV appearance descriptors with EMA smoothing (alpha=0.3) for correspondence matching.
- Three fallback paths: homography projection (<0.001ms), adaptive pixel extrapolation, and 3D Kalman world-coordinate projection.
- 57 unit tests covering Kalman, homography, and tracking, with CI via GitHub Actions.
Why It Matters
Enables robust, real-time multi-camera tracking without manual calibration—critical for surveillance, robotics, and autonomous systems.