PanDent benchmark exposes MLLM failures in dental X-ray diagnosis

PanDent, introduced by Xiaohan Li and colleagues, is a new benchmark designed to evaluate multimodal large language models (MLLMs) on dental panoramic radiography. The dataset includes 9,524 high-quality orthopantomograms (OPGs), each annotated by experienced dentists and validated by an oral and maxillofacial radiologist. These fine-grained, tooth-level annotations provide clinically reliable supervision, with radiology reports constructed using clinician-defined logic to ensure explicit correspondence between structured findings and free-text descriptions.

Experiments on proprietary, open-source, and medical-specific MLLMs reveal a critical gap: current models generate fluent radiology reports but lack clinical consistency, particularly in fine-grained localization and tooth-level diagnosis. Fine-tuning on PanDent substantially improves visual localization and diagnostic accuracy, proving the benchmark's value as a rigorous evaluation tool and a resource for developing clinically grounded dental AI.