Research & Papers

Robots Given Impossible Tasks Keep Trying Anyway, Study Finds

⚡Your future robot helper may not know when to stop — and that's a safety problem.

Deep Dive

Robot brains are getting smarter. The newest kind, called vision-language-action models (AI that can see a scene, understand a spoken command, and physically act on it), can already handle tasks like 'put the apple in the bowl.' But researchers wanted to know what happens when the request itself is wrong — the apple isn't there, or two instructions contradict each other. So they built a test set of 2,826 such broken commands and ran eight different robot AI systems through them.

The results were striking. Every system got worse at finishing the original goal, dropping by at least 17 percentage points, and one model called OpenVLA fell a whopping 56 percentage points. But the real discovery was subtler. Even when the robots clearly failed, they didn't stop and refuse. They kept moving toward the original target, kept the same early motion patterns, and barely slowed down. The researchers gave this pattern a name: 'Failed Persistence.' It showed up across the board.

Why does that matter? Because the way we normally grade robots only looks at whether the job got done. A robot that politely stops and a robot that stubbornly barrels forward look identical on a scorecard if both end in failure. The team also tried telling the models to double-check the request before acting. That didn't reliably fix things — the robots didn't change their behavior in a coordinated way.

So what does this mean for you? Robots are heading into warehouses, factories, hospitals, and eventually homes. A machine that can't recognize a bad or impossible request is a machine that keeps moving toward the wrong object — with real hands, real motors, and real force. The takeaway is that we need to test how robots behave during a task, not just whether they finish it.

Key Points
  • Eight AI robot systems were tested with 2,826 impossible or contradictory commands to see if they'd stop.
  • Most robots didn't stop — they kept moving toward the original goal, a pattern researchers call 'Failed Persistence.'
  • One model, OpenVLA, finished 56 percentage points fewer tasks, and simply telling robots to check the request first didn't reliably help.

Why It Matters

Robots that won't stop when a task makes no sense could damage property or hurt someone nearby.

📬 Get the top 10 AI stories daily