Vision-Based Human Awareness Estimation for Enhanced Safety and Efficiency of AMRs in Industrial Warehouses
New system uses a single RGB camera to detect if a worker is looking at an AMR, enabling smarter navigation.
A consortium led by Fraunhofer Austria and TU Wien has published a novel method for making warehouse robots smarter and safer around humans. The system, detailed in a paper for the 2025 IEEE ETFA conference, tackles a core inefficiency in current automation: robots typically treat all humans as generic obstacles, forcing unnecessary stops or detours even when a worker is fully aware of the robot and capable of sharing space. This conservative behavior slows down logistics operations.
The researchers' solution is a real-time, vision-based pipeline that uses just a single RGB camera mounted on an Autonomous Mobile Robot (AMR). It combines state-of-the-art 3D human pose estimation with head orientation analysis to create a "viewing cone" for each person. By calculating a human's position relative to the robot and where they are looking, the system can determine with high reliability whether the human is aware of the AMR's presence. The entire AI pipeline was rigorously tested and validated using synthetically generated data within the physics-accurate NVIDIA Isaac Sim robotics simulation environment.
This capability is a significant step beyond simple obstacle avoidance. By distinguishing between an unaware worker who needs a wide berth and an aware colleague who can coordinate movement, AMRs can make more nuanced, efficient decisions. They can maintain speed or choose closer paths when safe, directly boosting throughput. The technology promises to enhance both safety protocols and operational efficiency in industrial warehouses and factories where humans and robots increasingly collaborate.
- Uses a single RGB camera for real-time 3D human pose and head orientation estimation to build a 'viewing cone'.
- Validated in NVIDIA Isaac Sim with synthetic data, confirming reliable real-time detection of human position and attention.
- Enables AMRs to adapt motion based on human awareness, moving beyond generic obstacle avoidance to improve efficiency.
Why It Matters
It enables true human-robot collaboration in logistics, making warehouses safer while significantly boosting robot throughput and operational speed.