Robots Just Got Better at Guessing What You're About to Do

Most robot safety today is simple: don't bump into people. A new research system called HINT-Plan goes a step further. It uses Vision Language Models — AI that can look at a picture and understand what's in it — to watch a person through a camera and guess their goal. Are they walking to the fridge, the door, or the tool shelf? Published this month by a team of European researchers, this is an early-stage idea, not a product you can buy.

Once the robot has that guess, it turns it into a formal goal and solves a joint planning puzzle: what should I do, given what this person is probably about to do? The team also built a 'scene graph' — essentially a labelled map of objects and how they relate to each other — so the robot understands a cluttered room instead of just seeing random shapes. That matters because real kitchens, warehouses and hospital hallways are messy.

The researchers tested HINT-Plan in a photorealistic simulation, a computer-generated world that looks like real life. It planned successfully about 70% of the time when people were involved, beating older methods by as much as 35 percentage points, and it clashed with humans less often. In plain terms: the robot got in the way less and cooperated more, finishing tasks instead of freezing up.

So what? Today's delivery robots, warehouse machines and hospital helpers mostly dodge people at the last second. A robot that anticipates you could clear a doorway before you reach it, or hand you the right tool without being asked. The catch is big: this has only been tested in simulation, and it still fails roughly one time in three. A misread intention could send a robot straight into your path — and cameras watching your every move raise privacy questions too.