Signed Evidence Flow: New audit tool detects hidden conflict in ML predictions

In modern data analysis, two cases can have the same fitted confidence even when one is backed by strong, agreed evidence and the other by equally strong but contradictory signals. This gap hides real risk. Jeffery Opoku and David Banahene introduce Signed Evidence Flow (SEF), which combines a fitted prediction rule with signed feature attributions to quantify support, opposition, conflict, and perturbation stability. The paper proves that confidence determines conflict exactly when it also determines total evidence mass, and derives the conditional variance that makes conflict useful for loss prediction beyond confidence alone.

Across healthcare, Covertype, black-box, finance, and ten external datasets, SEF reveals that conflict sometimes separates risk among predictions that already appear confidently correct. In some tasks, high-conflict cases carry higher error risk; in others, low-conflict cases are riskier. To handle this, the authors introduce ScopeGate, a held-out permutation diagnostic that checks which direction applies before SEF is used for review triage. SEF is positioned as an audit tool rather than a universal risk score—it describes evidence structure, while an independent calibration sample determines whether that structure is useful in the target population.