New Trick Makes AI Read X-Rays Better Without Extra Training

Doctors are starting to use AI that can look at X-rays and answer questions like "Is there a fracture?" or "Which organ is this?" Most medical AI tools require expensive, time-consuming extra training on massive sets of medical images. A new research paper argues that may not be necessary. Instead of retraining, the researchers use a technique called probing — essentially peeking inside a general-purpose AI that already knows both language and images, and reading the medical knowledge stored inside it.

The method, named MedProb, was tested on three well-known medical question-and-answer datasets, including pathology images and radiology scans. It answered multiple-choice questions more reliably than simply asking the AI in a prompt, and it also outperformed larger AI systems that had been specially trained for medicine. Even more surprising, smaller and cheaper AI models showed much more medical know-how than they get credit for. That suggests hospitals could achieve strong results with lightweight tools that run on ordinary computers, instead of paying for massive custom AI setups.

But the paper also includes a caution. AI models — including MedProb — show something called position bias: they tend to favor answers in certain positions, like a test-taker who picks "C" when unsure. In one case, this bias shifted results by up to 10 percentage points. In medical settings, such subtle quirks can be dangerous, because a wrong answer is not just a wrong answer — it affects real patients.

Despite that caveat, the findings are promising. They suggest that much of what AI needs to know about medicine is already inside general models, waiting to be unlocked. If that holds up, it could make AI diagnosis cheaper, faster, and easier to deploy in clinics that don't have deep pockets. And the bias issue, now identified, can be studied and addressed before these tools reach the doctor's office.