TestMap: Open-source infrastructure validates AI-generated unit tests

TestMap addresses a critical gap in AI-assisted software testing: determining whether foundation-model-generated unit tests are actually correct, useful, and maintainable. The tool automates the entire validation pipeline—from repository analysis and test mapping to baseline execution, coverage measurement, mutation testing, and test repair. For each generated candidate, TestMap logs not just final pass/fail but intermediate states like failed, repaired, and low-impact outcomes. This lifecycle visibility reveals model limitations, missing context, toolchain inefficiencies, and even potential faults in the system under test.

The infrastructure is designed for C#/.NET repositories and supports experiment tracking across different models, prompts, and generation strategies. TestMap is intended as a design case for making AI-generated tests observable and comparable. The paper also highlights key challenges: oracle and assertion quality, metric attribution, test maintainability, flakiness, execution cost, and developer acceptance. Accepted at AIWare 2026, the project is open source and aims to standardize how teams evaluate the trustworthiness of AI-generated test suites.