AI That Crashes Cars on Purpose Could Make Self-Driving Safer

Self-driving cars are trained on millions of miles of normal driving, but the situations that actually kill people are rare: a pedestrian stepping out from behind a parked truck, a cyclist swerving at dusk. Finding those rare cases by hand is slow and expensive. So a team of researchers at the University of Michigan built a system that hunts for them automatically, using two AI language models (the same kind of technology behind ChatGPT) working as a pair.

The design is simple to picture. One AI acts like a demanding driving instructor. The other acts like a student who writes out specific test scenarios — where cars are, how fast they move, when someone brakes — in a strict format a simulator can run. The instructor stays quiet while the tests keep finding crashes, and only steps in with new ideas when the crashes dry up. That back-and-forth keeps the search from getting stuck.

In tests using CARLA, a popular open-source driving simulator, their system produced a collision in 90.79% of the scenarios it generated, with an average of just 18.31 seconds before impact. One competing method found slightly more crashes overall, but its results swung wildly from run to run. The new system was also more consistent and produced more varied situations, which matters because a test suite full of near-identical crashes teaches engineers very little.

Perhaps most useful: about 60% of the generated crashes were judged "avoidable" — meaning a competent driver could have seen them coming. That's exactly what engineers want. A crash nobody could avoid is just bad luck; an avoidable one points at a fixable weakness in the car's perception or braking. The limitation is that simulators are not the real world. Weather, worn tires, and human unpredictability all behave differently outside the software, so passing these tests is evidence of safety, not proof of it.