AI Tries to Predict Bladder Cancer Relapse — Accuracy Still Falls Short
368 patients, 159 AI attempts, and the best still missed roughly a third of the time.
Bladder cancer caught early but classified as high-risk is tricky for doctors. Some patients respond well to BCG (a decades-old immunotherapy that trains the immune system to attack cancer in the bladder), while others relapse or get worse. Today, doctors guess at that risk using general rules of thumb, and those guesses are often wrong. So researchers launched CHIMERA, a contest where AI teams from around the world tried to do better using three kinds of information at once: tissue images viewed under a microscope, genetic activity readouts, and standard patient records.
The contest handed out data from 368 patients and hid the rest for testing. In total, 159 models were submitted, and the 13 best were compared head-to-head. The winner correctly sorted patients into treatment-response categories about 73 percent of the time, and ranked who would relapse worse or sooner with about 68 percent accuracy — better than a coin flip, but well below what a doctor would need to change someone's care.
Here's the catch, and it's a big one. When the same AI was pointed at patients from a different hospital, its accuracy dropped noticeably — a common problem called "transportability." It also stumbled when some lab values were missing, which happens constantly in the real world. And one specific group of patients — those with a tumor feature called "T1 substage" — fooled nearly every model, regardless of how it was built. That's a humbling reminder that no two cancers behave alike.
So what does this mean for you? If you or a loved one has bladder cancer, this is not something to ask your doctor about tomorrow — it's research, not a clinic-ready tool. But the value is real: CHIMERA created a fair, shared scoreboard so future AI can be tested honestly across many hospitals. The path to AI that genuinely helps cancer patients runs through studies like this one — and it also means more reliable second opinions are probably years, not months, away.
- 159 research teams built AI that reads microscope slides and patient records to guess how bladder cancer will behave — the best got treatment response right about 73 percent of the time.
- The same AI got noticeably worse on patients from different hospitals, a real-world gap that matters because models must work everywhere, not just where they were trained.
- Patients with a tumor feature called 'T1 substage' stumped nearly every model, showing that some cancers are simply harder for machines — and doctors — to read.
Why It Matters
AI may one day give doctors a reliable second opinion on bladder cancer, but this study shows we're not there yet.