Polite Words Can Fool a Robot's Morality Filter, Test Finds

An independent researcher has been testing a "moral guard" for robots — software that reads a plain-text instruction and decides whether the machine should actually carry it out. The guard, called QERRA-v2 Classical, was tried out in a robot simulator on a PAL Robotics TIAGo robot. The finding: the guard catches obvious wrongdoing like fraud or open threats, but it gets fooled when bad intentions arrive in nice packaging.

In a 20-case test, the researcher took harmful orders and rephrased them with warm, caring words — "dear," "for your own future," "this is protective care." Each time, the system's suspicion score dropped by 13 to 16 percentage points. That small dip was enough to push serious violations below the alarm line, so the robot logged them as "safe" and would have carried them out. Remove the polite wrapping, and the robot refused again. In plain terms: flattery and institutional jargon act like camouflage for bad instructions, and a filter reading only words can be talked past.

The tests also exposed the opposite problem. One case described people settling a conflict maturely — someone "took full responsibility without making excuses" — and the guard wrongly raised a manipulation alarm. It saw conflict words and assumed ongoing coercion. A robot that cries wolf every time humans argue is useless.

A third case showed the hard limit: an impossible retail workload was rated "safe" because the cruelty lived in facts the text never mentioned, like store size, short staffing and hours left on the shift. The guard honestly refused to invent details it wasn't given. That is the boundary where reading words ends and real-world sensors — cameras, clocks, staffing data — must take over. The code and full test log are public on GitHub, and the researcher is asking for feedback. This is early, small-scale research in simulation, not a product anyone owns yet.