New vision-language framework boosts robotic guidewire navigation accuracy
Multimodal LLM adapts rewards in real-time for safer endovascular procedures.
Robotic-assisted endovascular interventions — like clearing blocked arteries — require precise guidewire navigation within complex, patient-specific blood vessels. Current autonomous systems rely on static reward functions that fail to adapt as the procedure progresses, often getting stuck or making suboptimal moves. To solve this, a team of researchers (Wentong Tian, Jiyuan Zhao, et al.) presents a vision-language procedural reasoning (VL-PR) framework, accepted at IEEE/RSJ IROS 2026.
The core innovation: a multimodal large language model (MLLM) that interprets real-time X-ray or endoscopic images and infers the current procedural phase (e.g., entering a branch, crossing a lesion, reaching the target). Instead of generating low-level joystick commands, the MLLM outputs "procedural insights" that dynamically adjust the weights of a multi-objective reward function — balancing safety, speed, and stability per phase. Experiments on a physical robotic platform across diverse vascular anatomies showed enhanced task reliability and streamlined navigation efficiency compared to static-reward baselines, offering a scalable solution for multi-task robotic surgeries.
- VL-PR uses a multimodal LLM to infer procedural context from real-time visual observations, enabling context-aware reward adaptation.
- Tested on a physical robotic platform across diverse vascular scenarios, outperforming static-reward methods in reliability and efficiency.
- Accepted at IEEE/RSJ IROS 2026; framework allows a single policy to handle complex multi-phase navigation without retraining.
Why It Matters
This makes robotic surgery smarter — adapts to each phase without retraining, boosting safety and reducing procedure time.