CLAP turns VLMs into robot control models with language-action grounding

Vision-language-action models (VLAs) typically require heavy post-training on robot data, which can distort the original VLM backbone and obscure what semantic capabilities are actually being used. A new paper from researchers at OMRON SINIC X and Keio University introduces CLAP (Causal Language-Action Prediction), a minimal-modification approach that directly converts pretrained VLMs into effective VLAs. The key insight: rather than predicting actions as bare numeric tokens—which shifts the model away from its pretrained language distribution—CLAP prepends each action sequence with a natural-language action description. This causally conditions precise action-token prediction on a language-action plan without altering the backbone architecture.

With just single-epoch fine-tuning, a 2B-parameter CLAP model achieves 90.8% success on the LIBERO benchmark, a 14.9-point improvement over the VLA-0 baseline. It also demonstrates enhanced robustness on LIBERO-PRO under language, object, and spatial perturbations. The authors plan to release CLAP at 0.8B, 2B, and 4B as an open-weight, multi-scale family from a single VLM lineage, enabling controlled analysis of how VLM capabilities transfer to robot control. This work offers a transparent path for leveraging large pretrained models in robotics while preserving their original strengths.