New MADQI metric evaluates unsupervised maritime anomaly detection without labels

A team of researchers (Ismet Gocer, Zakirul Bhuiyan, Raza Hasan, Shakeel Ahmad) has published a paper on arXiv introducing MADQI—Maritime Anomaly Detection Quality Index. This is a composite evaluation metric designed specifically for unsupervised learning models used in detecting anomalous vessel behaviors from Automatic Identification System (AIS) data. Traditional unsupervised methods like Isolation Forest lack systematic evaluation measures because labeled ground truth is rarely available. MADQI fills that gap by combining four interconnected sub-metrics: Anomaly Rate Consistency (ARC), Physical Plausibility Score (PPS), Score Distribution Separation (SDS), and Extreme Case Evidence (ECE). These are normalized automatically using multi-chunk evaluation and adaptive scaling.

The framework processes AIS datasets using Haversine distance calculations to analyze spatial and behavioral characteristics such as speed inconsistencies, position jumps, time gaps, and abnormal turn angles. When tested on real maritime data, MADQI achieved an overall score of 80.37%, demonstrating strong performance. Notably, the ECE component scored 0.907, indicating excellent detection of extreme anomalies, while ARC achieved a perfect 1.000, showing consistent anomaly rate estimation. The results suggest that MADQI provides a reliable, label-free way to benchmark unsupervised anomaly detectors in maritime contexts, potentially improving safety and security in shipping lanes.