Transformer-based predictor bridges sim-to-real gap for robotic table tennis
New SPAD strategy swaps simulation models with real-world predictors at deployment.
Robotic table tennis demands fast, accurate prediction of ball states for reliable control. Existing physics-based approaches require precise parameter identification and initial conditions, while learning-based methods struggle with long-range temporal dependencies and often rely on limited simulated data. Researchers propose a transformer-based framework that uses attention mechanisms to model temporal correlations directly from historical observations, eliminating the need for explicit flight or bounce models. They collected a large-scale real-world dataset from players of varying skill levels and diverse ball cannon configurations, enabling robust long-horizon forecasting.
Building on this predictor, the team introduces SPAD (Swap Predictor at Deployment), a plug-and-play sim-to-real transfer strategy. SPAD replaces the physics-based simulator used during training with the real-world-trained predictor at deployment, improving policy transferability without retraining. This simple substitution effectively narrows the sim-to-real gap while preserving the efficiency and scalability of simulation-based training. The approach demonstrates a practical pathway for deploying learned models in high-speed dynamic environments, with potential applications beyond table tennis.
- Uses transformer attention to model long-range temporal dependencies in ball trajectory, avoiding explicit physics models.
- Collected large-scale real-world dataset from players of varying skill levels and ball cannon configurations.
- SPAD strategy enables sim-to-real transfer by swapping the physics simulator with a real-world trained predictor at deployment, no retraining needed.
Why It Matters
This technique enables robust sim-to-real transfer for high-speed robotic control, reducing the need for costly real-world retraining.