Doctors' Eye Movements Now Help AI Spot Tumors in Medical Scans

AI that helps doctors read scans usually needs to be trained on thousands of images that specialists have carefully outlined or labeled. Creating that training material is slow and expensive — so most tools only exist for a few common conditions. A new method, called GazeRefine, could remove that bottleneck by letting a doctor's own eye movements teach the AI in real time.

Here's how it works: a doctor watches a colonoscopy video or a prostate MRI, and a camera tracks where their eyes linger. The system converts those short pauses into a ''hint'' about what matters, then automatically expands that hint to outline the whole suspicious area — a polyp, a tumor, or other abnormality. No training data, custom adapters, or fine-tuning is required, which means it can adapt to new imaging tasks on the spot.

It's like having a doctor point at the problem just by glancing at it, while the AI does the painstaking work of drawing the exact boundary. The system builds on a pre-existing computer vision model and is designed to keep the AI's attention anchored to the doctor's guidance, so it won't drift off and start segmenting something irrelevant.

In early tests, GazeRefine performed strongly on polyp detection in colonoscopy images, a key step for catching colorectal cancer early, and competitively on prostate MRI segmentation. There are still limitations — it hasn't been tested in every type of medical scan, and it may not outperform specialized models built for one task. But for real-world clinical settings, where new imaging types appear regularly and expert time is precious, this gaze-guided approach could make AI diagnosis far more flexible, affordable, and accessible to hospitals everywhere.