New AI Tool Cuts Software Testing Costs by Focusing on Risky Code
This could save companies millions in wasted testing and catch more bugs.
Software companies spend a huge amount of time and money testing every code change, even though most changes don't cause problems. A new approach called KG-AST uses AI to predict which changes are riskiest, so teams can focus their testing and AI review on those. In a study of 155 open-source projects, testing only the riskiest 20% of changes caught 49-64% of bugs, compared to just 20% when testing randomly. That's a big improvement—like finding a needle in a haystack by using a magnet instead of your hands.
The system works by looking at the history of code changes and the connections between different parts of the code. It learns from past projects to score new changes. Surprisingly, it doesn't need much training—about twelve projects are enough. And fancier features or cleaner data didn't help. The AI reviewer also benefits: when given a small map of the code around a change, it finds 32% of later-fixed bugs exactly, up from 26% without that context. Even when it flags something that wasn't a known bug, about half the time independent judges agreed it was a real issue.
For everyday users, this means software could become more reliable and cheaper to produce. Companies save on cloud computing costs for testing and developer time. But there are caveats: the study didn't measure actual CI minutes saved, and the labels for bugs are noisy. Also, the method assumes that a heavy test will catch the bug it covers, which isn't always true. Still, the results are promising and could lead to faster, more stable apps and websites.
The research also tested a static call-graph selector that runs only 8% of tests but includes the test written for the fix in 85% of cases. That's like having a smart assistant that knows exactly which tests to run. However, a fancy embedding technique called Poincare didn't add value here. Overall, this is a step toward smarter, more efficient software development that could benefit both companies and consumers.
- Testing only the riskiest 20% of code changes catches up to 64% of bugs, versus 20% with random testing.
- The AI reviewer finds 32% of later-fixed bugs when given a graph of the code around the change.
- About twelve training projects are enough for the model to work well across different programming languages.
Why It Matters
Faster, cheaper software development with fewer bugs means better apps and lower costs for everyone.