Aloe-Vision: Open-source medical AI models challenge GPT-4V on exams
7B and 72B models trained on quality-filtered medical data, released fully open.
Aloe-Vision, introduced by researchers from multiple institutions, is a new family of open-source Large Vision-Language Models (LVLMs) specialized for healthcare. The models come in two sizes: 7B and 72B parameters, and are built on top of a carefully curated dataset called Aloe-Vision-Data. This dataset integrates medical and general domain sources, both multimodal and text-only, and is quality-filtered to avoid noisy or low-value examples. The researchers claim that high-quality training mixtures produce balanced models that achieve significant gains over baselines without sacrificing general capabilities, rivaling proprietary systems like GPT-4V in biomedical tasks.
To enable rigorous evaluation, the team created CareQA-Vision, a benchmark derived from Spain's national residency entrance exams for medical and nursing specialists (MIR and EIR). This benchmark offers novel vision questions with minimal risk of data contamination—a common issue in medical AI evaluations. Despite strong performance, the paper warns that current LVLMs, including Aloe-Vision, remain vulnerable to adversarial and misleading inputs, underscoring the need for robustness improvements before deployment in clinical settings. All models, data, and training recipes are released openly to promote reproducibility and further research.
- Aloe-Vision comes in two scales: 7B and 72B parameters, fully open-source with weights, data, and training recipes.
- CareQA-Vision benchmark uses fresh questions from Spanish MIR/EIR medical residency exams to avoid data contamination.
- Models match state-of-the-art alternatives but show vulnerabilities to adversarial inputs, limiting clinical readiness.
Why It Matters
Open-source medical AI that rivals proprietary models could accelerate safe, transparent clinical diagnostics.