Stanford/NUS researchers unveil FedPref for radiology AI
FedPref improves radiology report extraction by 9.1 points without sharing sensitive data
A team of researchers from Stanford University and the National University of Singapore (NUS) has developed **FedPref**, a federated preference learning framework designed to enhance AI-driven radiology report extraction while maintaining strict data privacy. The system leverages frozen public language models (specifically **Qwen3-8B**) to propose alternative JSON-structured extractions from radiology reports, which local annotators then rank. Institutions collaboratively train compact adapters without sharing raw reports or annotations, enabling cross-site collaboration on highly sensitive medical data.
In evaluations across six simulated hospitals with uneven data distribution and disease prevalence, FedPref achieved a **2.49-point improvement in client-mean F1 score** and a **9.10-point boost in worst-site F1 score** compared to isolated training. When compared to centralized training on pooled preference pairs, FedPref delivered results within **2.66 points** of the optimal performance (71.67 F1 for pooled training vs. 68.68 F1 for FedPref on a 400-report gold-standard test set). The framework’s heterogeneous teacher pool further ensures robustness by mitigating performance collapse in single-model sampling scenarios, making it particularly effective for smaller hospitals with limited local data.
- FedPref enables **collaborative AI training across hospitals without sharing raw radiology reports**, using a federated preference learning approach with frozen Qwen3-8B models.
- Improves **worst-site F1 by 9.10 points** and **client-mean F1 by 2.49 points** compared to isolated training, with results within 2.66 points of pooled-data training.
- Validated on a **manually curated 400-report gold test set**, achieving **68.68 F1** while preserving strict data privacy.
Why It Matters
Solves a critical privacy-preserving AI challenge in healthcare, enabling equitable model improvements for smaller hospitals without compromising patient data security.