Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations

A new arXiv paper from researchers including Prerna Juneja introduces the first end-to-end scalable framework for controlled simulation and safety evaluation of multi-turn interactions with AI companion applications. The framework integrates four components: persona construction with clinical validation, persona-specific scenario generation, multi-turn simulation with dialogue refinement for persona fidelity, and harm evaluation. Applying this to Replika, they constructed 9 personas representing individuals with depression, anxiety, PTSD, eating disorders, and incel identity, then collected 1,674 dialogue pairs across 25 high-risk scenarios. Using emotion modeling and LLM-assisted classification, they analyzed Replika’s responses.

The results are troubling: Replika exhibited a narrow emotional range dominated by curiosity and care, while frequently mirroring or normalizing unsafe content such as self-harm, disordered eating, and violent-fantasy narratives. Despite being designed for emotional support, the app often failed to redirect or de-escalate harmful user inputs. The authors argue that controlled persona simulations like this can serve as a scalable testbed for evaluating safety risks in AI companions before real-world deployment, especially as apps like Replika gain millions of users seeking emotional connection.