Fidelity, Diversity, and Privacy: A Multi-Dimensional LLM Evaluation for Clinical Data Augmentation

A new arXiv paper (2604.27014) by Guillermo Iglesias and colleagues tests three large language models — DeepSeek-R1, OpenBioLLM-Llama3, and Qwen 3.5 — for generating synthetic clinical reports in mental health. The scarcity of annotated medical data (especially under strict privacy regulations) makes synthetic data augmentation attractive. The researchers conditioned text generation on specific ICD-10 diagnostic codes to produce realistic evaluation reports. To avoid mode collapse or memorization of private patient data, they designed a comprehensive evaluation framework across three dimensions: semantic fidelity (does the text clinically match the code?), lexical diversity (is there variety?), and privacy/plagiarism (does it leak real patient info?).

All three LLMs passed all tests, producing clinically coherent, lexically diverse, and privacy-safe synthetic reports. This significantly expands the available training data for clinical natural language processing tasks without compromising confidentiality. The study highlights that even open-weight models like OpenBioLLM can generate high-quality synthetic medical text when properly guided with condition codes. The 9-page paper includes one table and one figure detailing results. This work addresses a critical bottleneck in healthcare AI — data scarcity — while maintaining ethical safeguards.