Beyond ROC-AUC: Study urges DET curves for biometric verification

A new arXiv paper by Ajan Ahmed and Masudul Imtiaz takes aim at a widely used metric in biometric verification: ROC-AUC. The authors argue that in real-world deployments, biometric systems operate under a strict false match budget, typically at very low false match rates (FMR) like 10^-3 or 10^-4. However, ROC-AUC calculates the average true match rate across the full FMR range from 0 to 1, placing equal weight on regions where the system is never used. This can mask poor performance at the critical low-FMR region and even reverse the apparent ordering of two systems.

To test this, the researchers evaluated seven pretrained matchers across four modalities—face (FaceNet, ArcFace), voice, iris, and fingerprint—using bootstrap confidence intervals and paired bootstrap tests. The results were striking: FaceNet showed a higher full ROC-AUC than ArcFace, but at FMR=10^-3, ArcFace had a significantly higher true match rate with non-overlapping confidence intervals. The same reversal pattern appeared across other modalities. The paper re-iterates the ISO/IEC 19795-1 standard: report error rates at stated operating points, use the detection error tradeoff (DET) curve, and provide uncertainty intervals. ROC-AUC and EER should remain only as supplementary context.