Event-camera dataset boosts robotic table tennis with 36% better prediction

A team led by Raphaela Kreiser and Naoya Takahashi has introduced a game-changing dataset for robotic table tennis, published on arXiv as '1000 Rallies.' The dataset captures over 1,000 rallies from players of all skill levels using an event camera alongside 14 synchronized high-speed frame-based cameras at 200 FPS. Event cameras offer microsecond temporal resolution and eliminate motion blur, making them ideal for tracking fast-moving objects like table tennis balls. The researchers used this data to produce 1 kHz pseudo ground-truth labels for ball position, velocity, and spin.

Building on the dataset, they trained a convolutional neural network (CNN) that robustly estimates ball position and velocity from event data, even amidst background player motion. By feeding predicted velocity as an additional measurement into a Kalman filter, they reduced bounce-point prediction error by 36% compared to a position-only baseline. Finally, they closed the perception-action loop by integrating the event-based system with a Stäubli robotic arm, enabling the first real-time human-robot table tennis rallies driven entirely by event-based perception. This work paves the way for faster, more accurate robotic systems in dynamic environments.