A Benchmark for Strategic Auditee Gaming Under Continuous Compliance Monitoring
Five auditee strategies can evade continuous compliance monitoring—regulators beware.
Emerging regulations such as the EU AI Act and Digital Services Act mandate continuous post-deployment compliance audits, but this creates a new class of strategic gaming distinct from one-shot input/output manipulation. Burnat and Davidson formalize the problem as a T-round Stackelberg game where an auditor commits to a temporal policy and an adaptive auditee responds. They identify five auditee strategies: Delay (delaying outcome reports), Drift (varying reports within plausible noise envelopes), Cherry-pick (selecting among ambiguous metric definitions), Attrition (exploiting longitudinal sample dropout), and OffAuditDrift (audit-aware drifting that exploits the auditor's commitment).
The authors prove a key structural limitation (Observation 1): no noise-aware static auditor design can close both coverage gaps (missing certain violations) and granularity gaps (insufficient detail to detect drift). Two minimal extension policies—a sample-size-aware static rule (Periodic-with-floor) and a history-conditioned suspicion-escalation policy—each close only one axis, exactly as predicted. Remarkably, an audit-aware OffAuditDrift strategy defeats both. To support empirical work, they contribute a non-additive harm decomposition separating welfare loss (W) from coverage loss (C), showing how attrition shifts harm from regulator-accountable to invisible surfaces. They also release an extensible Python simulator calibrated to real DSA transparency data, enabling regulators to test policies against strategic auditees.
- Identifies five auditee gaming strategies (Delay, Drift, Cherry-pick, Attrition, OffAuditDrift) tailored to continuous compliance monitoring.
- Proves a structural limitation: static auditor policies cannot simultaneously close coverage and granularity gaps; adaptive policies are needed.
- Releases a reproducible simulator with harm decomposition (welfare loss W, coverage loss C) and policies calibrated to DSA Transparency Database statistics.
Why It Matters
With EU AI Act enforcement approaching, regulators must adopt adaptive audit policies to counter sophisticated gaming by AI companies.