MedConcept: Unsupervised Concept Discovery for Interpretability in Medical VLMs
New framework translates VLM activations into pseudo-reports, enabling physician-level inspection of AI reasoning.
A research team from the University of Utah and NVIDIA has developed MedConcept, a novel framework designed to solve the 'black box' problem in medical AI. While medical Vision-Language Models (VLMs) excel at tasks like tumor segmentation, their internal reasoning remains opaque, limiting clinical trust. MedConcept addresses this by discovering latent medical concepts—like 'consolidation' or 'pleural effusion'—directly from a model's pretrained representations in a fully unsupervised way. It identifies which sparse groups of neurons activate for specific concepts and grounds them in clinically verifiable textual semantics, effectively translating internal activations into human-readable, pseudo-report-style summaries.
Crucially, the team also introduced a first-of-its-kind quantitative evaluation protocol for concept-based interpretability. This protocol uses an independent, pretrained medical Large Language Model (LLM) as a frozen external evaluator. The LLM assesses how well the discovered concepts align with actual radiology reports, assigning three scores: Aligned, Unaligned, and Uncertain. This provides an objective, quantitative baseline for measuring interpretability, moving beyond qualitative visualizations. The framework's code and data will be released upon acceptance, offering a new tool for developers to build more transparent and trustworthy diagnostic AI systems.
- Unsupervised Discovery: Identifies latent medical concepts (e.g., specific pathologies) directly from pretrained VLM representations without manual labeling.
- Quantitative Verification: Introduces a scoring protocol (Aligned/Unaligned/Uncertain) using a frozen medical LLM to objectively evaluate concept alignment with radiology reports.
- Clinical Translation: Converts sparse neuron activations into pseudo-report-style summaries, enabling physician-level inspection of model reasoning for critical tasks.
Why It Matters
Provides a quantitative path to trustworthy AI diagnostics, potentially accelerating FDA approval and clinical adoption by making model reasoning inspectable.