PatientsWithPersonality simulator nearly matches human actors for clinical AI testing
New LLM-based framework uses HEXACO personality model for controllable, realistic patient diversity.
Simulating realistic patient interactions is critical for testing clinical applications of large language models (LLMs) at scale, but existing approaches often overshare information and lack behavioral diversity. Researchers from Google, Apple, and academic institutions introduce PatientsWithPersonality (PWP), a framework that generates diverse virtual patient responses through explicit personality parametrization over a latent patient state. Grounded in the HEXACO model—a six-dimensional personality space quantifying cooperativeness, honesty, emotionality, and more—PWP gives fine-grained control over conversational style and selective disclosure within a unified system.
In a clinician evaluation, PWP was rated nearly as realistic as interactions with recorded human actors, while being flagged as "too informative" far less often than prior simulators. The framework's personas span a substantially wider behavioral footprint than baselines, and their configured traits are recoverable by both clinicians and an autorater. By enabling realistic, steerable patient simulations, PWP offers a scalable path to more accurate and informative benchmarks for LLMs in healthcare without costly human studies.
- Uses HEXACO personality model with six dimensions (honesty, emotionality, extraversion, agreeableness, conscientiousness, openness) to parametrize patient behavior.
- Clinician evaluation rated PWP nearly as realistic as human actors, with far fewer instances of being "too informative" compared to prior simulators.
- Enables fine-grained control over conversational style, cooperativeness, and information disclosure for realistic LLM benchmarking.
Why It Matters
Enables scalable, realistic clinical AI testing without expensive human actors, accelerating safe LLM deployment in healthcare.