Robotic arm with spectral sensors picks strawberries, measures sweetness non-destructively

Yi Yang, Mark Cardamis, and Wen Hu (arXiv preprint 2026) present a robotic arm with an eye-in-hand spectral sensor that autonomously finds, approaches, and non-destructively measures strawberry sweetness. The system combines a YOLOv11s detector for real-time fruit localization, RGB-ToF depth alignment for accurate positioning, and a custom hand-eye calibration workflow to transform target coordinates into the robot's base frame. It then uses a waypoint-based search and incremental closed-loop approach to place the sensor at optimal working distance for spectral sensing.

Experimental results from 42 trials showed 88.1% end-to-end success, with detection at 95.24% and 100% approach success once a target was detected. The Park method for hand-eye calibration provided better cross-sample consistency than Andreff. Residual failures occurred mainly during valid-region extraction for sweetness estimation under difficult depth/reflectance conditions. The work demonstrates feasibility for scalable, non-destructive fruit quality assessment and provides a baseline for future Vision-Language-Action model integration.