New LLM System for Public Health Blocks Medical Advice, Ensures Safety

Researchers Ben Torkian and Jun Zhou have published a paper detailing a safety-constrained large language model (LLM) system designed specifically for public health information access, focusing on maternal and child health (MCH) resource navigation. The system addresses the critical challenge of deploying LLMs in healthcare contexts where uncontrolled generation could lead to harmful advice. Their solution employs a multi-layered architecture that integrates domain-restricted retrieval augmented generation (RAG), strict boundary enforcement to completely prevent the LLM from providing medical advice, anonymous multiuser session management, and comprehensive audit logging for monitoring and compliance. A key design choice is a controlled data pipeline that grounds all responses exclusively in curated public health resources, deliberately avoiding reliance on the model's pretrained medical knowledge to ensure information is both accurate and safe.

Implemented in a real-world public health setting, the system underwent scenario-based validation across in-scope, out-of-scope, and emergency queries. Results demonstrated consistent enforcement of safety constraints, reliable resource grounding, and stable system performance, with an average response time of 5.3 seconds. The paper discusses design trade-offs in balancing safety, usability, and system flexibility, offering practical guidance for deploying LLM-based systems in healthcare and other domains where strict information boundaries and accountability are required. This work represents a significant step toward practical, safe AI applications in public health.