Seeing Through Experts Eyes A Foundational Vision Language Model Trained on Radiologists Gaze and Reasoning

A research team led by Kinhei Lee has developed GazeX, a foundational vision-language model that fundamentally changes how AI interprets medical images by learning from radiologists' visual behavior. Unlike standard models that process semantic information, GazeX was trained on a curated dataset of over 30,000 gaze key frames from five radiologists, capturing their eye-tracking data, fixation patterns, and attention trajectories. This allows the model to emulate the structured, systematic protocols—such as the ABCDEF approach—that experts use to examine chest X-rays, ensuring all clinically relevant regions are assessed in a logical sequence. The goal is to bridge the critical gap between generic AI outputs and the nuanced, reliable reasoning required in clinical diagnostics.

GazeX was trained on a massive dataset of 231,835 radiographic studies, 780,014 question-answer pairs, and 1,162 image-sentence pairs with bounding boxes. The model demonstrates superior performance in radiology report generation, disease grounding, and visual question answering by producing outputs that are more accurate, interpretable, and consistent with expert judgment. Crucially, it generates verifiable evidence artifacts, including inspection trajectories and finding-linked localized regions, which enable efficient human verification. This approach of 'learning through expert eyes' provides a practical pathway toward more trustworthy, explainable, and diagnostically robust AI systems, not just for radiology but for other high-stakes visual domains.