Microsoft's ASSERT tests AI behavior from text rules

Microsoft unveiled ASSERT (Adaptive Spec-driven Scoring for Evaluation and Regression Testing), an open-source framework designed to automate the evaluation of AI systems against application-specific behaviors.

The tool addresses a critical gap in AI evaluation by converting plain-language descriptions of desired behavior—such as policies, constraints, or safety rules—into structured test cases. For example, a developer could input rules like 'limit confidential data to C-level executives' or 'avoid sending emails outside the company,' and ASSERT would generate test scenarios to verify compliance. The framework not only runs these tests but also records intermediate actions and tool calls, enabling developers to debug failures in production-like conditions. Sarah Bird, Microsoft’s Chief Product Officer of Responsible AI, emphasized that evaluation is foundational to trustworthy AI systems, stating that broader benchmarks often miss application-specific nuances that ASSERT captures. The release aligns with a broader industry trend toward rigorous, repeatable testing, as seen in tools like Stanford’s HELM and MLCommons’ AILuminate.