LLaMA 3.1 extracts brain MRI data with 96% accuracy from Dutch reports

A team from Erasmus MC and collaborating institutions evaluated LLaMA 3.1's ability to extract structured information from Dutch neuroradiology reports. They used 947 brain MRI reports from a tertiary memory clinic (2016–2021), annotated by trained medical students across 30 variables—including visual rating scores, numerical counts, and free-text lesion locations. The study tested both zero-shot and few-shot prompting strategies, with and without English translation of the Dutch reports.

Results showed that LLaMA 3.1 achieved high zero-shot performance for visual rating scores: Medial Temporal Atrophy left 90%, right 96%; Global Cortical Atrophy 87%; and Fazekas scale 94%. Accuracy for detecting microbleed mentions was 93% and infarct mentions 82%. Numerical variables were more challenging: microbleed count accuracy was 80%, infarct count 66%. However, few-shot prompting using structural similarity-based example selection boosted microbleed count to 92% and infarct count to 81%. Free-text lesion location similarity reached 0.95. English translation yielded comparable results, suggesting the model understands medical context beyond language. The findings demonstrate that open-weight LLMs like LLaMA 3.1 can automate data extraction from radiology reports, enabling large-scale clinical research while preserving patient privacy.