A Computationally Efficient Learning of Artificial Intelligence System Reliability Considering Error Propagation

A team of researchers including Fenglian Pan and Yinwei Zhang has published a paper introducing a novel, computationally efficient framework for learning the reliability of complex, multi-stage Artificial Intelligence (AI) systems. The core challenge addressed is error propagation, where a mistake in an upstream component (like an object detection model) can cascade and corrupt downstream stages (like path planning), making overall system failure hard to predict. The researchers tackled three major hurdles: the scarcity of real-world failure data due to privacy, the statistical interdependence of errors across stages, and the computational complexity of analyzing high-frequency error events in systems processing vast data streams.

To solve the data problem, the team leveraged a physics-based autonomous vehicle simulation platform equipped with a 'justifiable error injector.' This tool generates high-quality, synthetic reliability data by systematically introducing faults. Building on this simulated data, they developed a new reliability modeling framework that explicitly characterizes how errors propagate. The model's parameters are estimated using a composite likelihood expectation-maximization (CLEM) algorithm, which is designed for computational efficiency and comes with theoretical performance guarantees. The paper demonstrates the framework's application to autonomous vehicle perception systems, showing it achieves strong predictive accuracy while being significantly faster than traditional methods, offering a practical tool for engineers to vet AI system safety before real-world deployment.