Poor teleoperation experience and physics instability with UR robot in Isaacsim

A participant in NVIDIA's 'AI for Industry Challenge' has exposed significant physics simulation problems with Universal Robots (UR) arms in the latest Isaac Sim 5.1 environment. The developer, working on Ubuntu 24.04 with an RTX 5090 GPU, reported that UR robots become unstable during teleoperation—clipping through themselves, responding poorly to keyboard commands, and experiencing cable entanglement issues. In stark contrast, Franka Emika robots in the same Isaac Lab framework perform smoothly with precise SE(3) command following. This discrepancy raises questions about asset quality and physics modeling consistency in NVIDIA's flagship robotics simulation platform, which is critical for training real-world AI agents.

The technical issues documented include severe geometric collisions where robot parts intersect unrealistically, making the UR arm feel 'heavy' and unresponsive compared to the benchmark Franka performance. The problem appears specific to the UR robot implementation within the 'aic' repository for the competition, suggesting potential configuration or modeling errors rather than core engine flaws. For robotics researchers and companies using Isaac Sim for digital twin development or reinforcement learning, such instability could compromise training pipelines and transfer learning to physical systems. NVIDIA now faces pressure to address these simulation fidelity gaps as industrial AI adoption accelerates, with competitors like Unity and AWS RoboMaker watching closely.