EaaS: Cloud-Native Microservices for AI Monitoring with Conformal Guarantees

In a new arXiv preprint, Lei Yang introduces EaaS (Evaluation-as-a-Service), a cloud-native reference architecture that transforms AI monitoring into a set of six stateless Kubernetes microservices. The system operationalizes key evaluation methods: conformal prediction using finite-sample-corrected Adaptive Prediction Sets, calibration assessment, drift detection via Random Fourier Feature-approximated Maximum Mean Discrepancy (RFF-MMD), fairness monitoring with bootstrap confidence intervals, a DAG-based pipeline orchestrator, and a results storage API. This microservice decomposition allows independent scaling and flexible orchestration for real-time and batch monitoring workloads.

Validation across four methodological pillars shows strong performance. Conformal prediction achieves empirical coverage within 1.4 percentage points of the nominal target across 50 random splits, with sub-2ms p99 latency at batch size 100. Drift detection via RFF-MMD reaches 100% detection power for mild and severe shifts, with Type I error between 5-8.5% and ~500ms latency suited for periodic checks. Fairness monitoring on the UCI Adult Income dataset reveals a demographic parity gap of 0.33 by race, with stable alerts across sequential batches. The author notes that no current open-source platform combines conformal-prediction-as-a-service, microservice decomposition, and DAG-based orchestration, making EaaS a novel contribution for production AI reliability.