Dual-Modal Lung Cancer AI: Interpretable Radiology and Microscopy with Clinical Risk Integration
New AI fuses CT scans and tissue slides for 97% accurate, interpretable lung cancer classification.
A new research paper from Baramee Sukumal and Aueaphum Aueawatthanaphisut introduces a dual-modal AI framework designed to tackle a major challenge in lung cancer diagnosis. The system uniquely fuses two critical data types: computed tomography (CT) radiology scans and hematoxylin and eosin (H&E) stained histopathology slides. By employing convolutional neural networks to extract features from both imaging modalities and integrating clinical metadata, the AI aims to overcome the limitations of using CT scans alone, which can struggle to distinguish benign from malignant lesions. The model's predictions are combined using a weighted decision-level integration mechanism to classify five categories: adenocarcinoma, squamous cell carcinoma, large cell carcinoma, small cell lung cancer, and normal tissue.
A core innovation of this work is its commitment to transparency through explainable AI (XAI). The researchers applied and evaluated multiple XAI techniques—including Grad-CAM, Grad-CAM++, Integrated Gradients, and Saliency Maps—to generate visual heatmaps that highlight the regions of the images most influential to the AI's decision. This allows clinicians to see *why* the model made a specific diagnosis, aligning its reasoning with human expert annotations. In their experiments, Grad-CAM++ demonstrated the highest faithfulness and localization accuracy. The system achieved impressive performance metrics, including an area under the receiver operating characteristic curve (AUROC) above 0.97, an accuracy of 0.87, and a macro F1-score of 0.88.
The results indicate that this multimodal fusion approach can significantly enhance diagnostic performance while maintaining the model transparency required for clinical trust. The framework represents a meaningful step toward building future clinical decision support systems in precision oncology, where AI acts as a powerful, interpretable assistant to pathologists and radiologists, potentially leading to earlier and more accurate lung cancer diagnoses.
- Fuses CT scans and H&E pathology slides with clinical data for robust lung cancer subtype classification.
- Achieves high diagnostic performance with an AUROC >0.97 and 87% accuracy using explainable AI techniques.
- Grad-CAM++ provided the most faithful visual explanations, aligning AI reasoning with expert-annotated tumor regions.
Why It Matters
Provides clinicians with a highly accurate, transparent AI tool for earlier and more precise lung cancer diagnosis, a leading cause of cancer mortality.