New multimodal AI fuses MRI and radiomic features for 96.13% brain tumor accuracy

Most deep learning models for brain tumor classification rely solely on MRI or CT images, ignoring the multimodal reasoning clinicians use—synthesizing symptoms, history, and quantitative imaging data. Researchers from the University of Hertfordshire and collaborators propose a two-branch neural network that addresses this gap. One branch uses a pre-trained CNN (e.g., ResNet) to encode MRI images, while a separate MLP processes 91 radiomic features covering intensity, texture, shape, and boundary descriptors. The two streams are fused using either simple concatenation, gated fusion, or bidirectional cross-modal attention.

Tested on a balanced dataset of 7,200 images across four classes (glioma, meningioma, pituitary, no-tumor), all multimodal configurations outperformed unimodal baselines. The best performance came from gated fusion, achieving 96.13% accuracy—a significant improvement over image-only or radiomics-only models. The method more closely replicates how clinicians integrate multiple data sources, potentially leading to more reliable automated diagnosis. The work is published on arXiv (2606.11107) as a step toward clinically realistic AI.