SALT tokenizer lifts robot success to 71.9% by aligning actions with language

A new arXiv paper from Li Wenjie and colleagues at the intersection of robotics and AI tackles a fundamental flaw in vision-language-action models (VLAs): their action representations optimized purely for reconstruction lose the linguistic nuances of action verbs. Using BridgeV2, the team shows that action trajectories contain verb-grounding information beyond visual state changes, but discrete tokenization under L1/L2 losses systematically erodes it. The result is that robots can mimic the outcome of a task without understanding the way the action is performed.

To solve this, they introduce SALT (Semantically ALigned action Tokenizer), which extends a VQ-VAE-style tokenizer with a novel auxiliary objective: a frozen vision-language model must recover the original episode instruction from the quantized action latents. This forces the tokenizer to preserve semantic information tied to language while maintaining reconstruction fidelity. On SimplerEnv, policies trained with SALT achieve 71.9% average success, compared to 42.7% for a reconstruction-only VQ-VAE and 31.2% for FAST. SALT also develops verb-specialized codes, demonstrating that robot trajectories can serve as a source of language grounding — a promising step toward more robust, language-conditioned robot control.