AI Is Now Steering Cars and Planes — But Its Safety Checks Are Disappearing
The hidden code running cars and factories is changing — and losing its guardrails.
Almost everything around you that moves or heats or pumps — cars, planes, robots, factory lines, power plants — is run by a "controller." That's the software that decides how a machine reacts: speed up, slow down, shut off if it gets too hot. Engineers have traditionally built these controllers by hand, writing clear rules and, crucially, adding visible safety limits. They build them in a tool called Simulink (think of it as a visual flowchart builder for machine logic), where anyone can open the design and see exactly where the guardrails are.
This study looked at 62 real controllers from 10 industries, plus interviews with 13 working engineers. The headline finding: as AI takes over from hand-written rules, the explicit safety blocks mostly disappear. In the old designs you could point to a box that said "stop if pressure exceeds X." In the AI versions, that protection lives inside the AI's training — invisible, unlabeled, and impossible to trace. The researchers call this a "misalignment" between what engineers assume is happening and what's actually in the model.
There's a second surprise. AI controllers lean heavily on something called "discrete dynamics and user-defined abstraction" — essentially custom building blocks the engineers invented themselves. These barely appear in academic AI research at all, which means the AI being used in the real world looks almost nothing like the AI being written about. Theory and practice have drifted apart.
The practical worry is accountability. If a self-driving car or a medical pump misbehaves, regulators and investigators need to trace why. When safety lives in visible structure, that's possible. When it lives silently inside training data, it isn't. The study isn't anti-AI — it's a warning that we're adopting AI in safety-critical machines faster than we're learning to check it.
- Researchers examined 62 real machine controllers across 10 industries — from cars to factories — built with the engineering tool Simulink.
- When AI replaces hand-written rules, the visible safety limits mostly vanish, moving into invisible AI training instead.
- Most of a controller's bulk (68-72%) is just organizing files and subsystems, while the actual decision-making logic takes up very little space.
Why It Matters
The machines you trust daily are getting AI brains with fewer visible safety checks — and harder accountability when things fail.