A Multimodal Deep Learning Framework for Edema Classification Using HCT and Clinical Data
A new framework fuses head CT scans with patient data for 87% accurate, interpretable brain swelling diagnosis.
A research team has introduced AttentionMixer, a novel deep learning framework designed to significantly improve the automated detection of brain edema, a life-threatening condition involving brain swelling. The system's key innovation is its structured fusion of two data types: 3D head CT (HCT) scans, processed by a self-supervised Vision Transformer Autoencoder (ViT-AE++), and routine clinical metadata like patient age and lab values. Instead of simply concatenating these sources, AttentionMixer uses a cross-attention mechanism where the imaging features act as queries to dynamically retrieve and integrate relevant context from the clinical data. This allows the model to adapt its analysis based on specific patient factors, making it both more accurate and interpretable.
The framework is completed by a lightweight MLP-Mixer that refines the fused representation for the final classification. Tested on an expert-annotated brain HCT cohort, AttentionMixer outperformed strong baselines, achieving an accuracy of 87.32%, a precision of 92.10%, and a critical area-under-the-curve (AUC) score of 94.14%. The model is also built for clinical practicality, featuring a learnable embedding to handle missing metadata commonly found in hospital records. Ablation studies confirmed the importance of both the cross-attention fusion and the MLP-Mixer, while an analysis of feature importance highlighted clinically meaningful variables that drive predictions, offering doctors a clear window into the AI's reasoning process.
- Achieves 94.14% AUC and 87.32% accuracy by fusing CT scans with clinical data via cross-attention.
- Uses a self-supervised Vision Transformer (ViT-AE++) to encode images, reducing need for large labeled datasets.
- Provides interpretable predictions by identifying key clinical variables and handles missing data for real-world use.
Why It Matters
This brings a more accurate, robust, and explainable AI diagnostic tool directly into critical neurology and emergency medicine workflows.