AFA: Identity-Aware Memory for Preventing Persona Confusion in Multi-User Dialogue

Researchers led by Mohammad Al-Ratrout from the University of Delaware have introduced AFA (Adaptive Friend Agent), a modular framework designed to solve persona confusion in multi-user voice assistants. Persona confusion occurs when a single-user dialogue system conflates histories from different users in shared environments—one resident's preferences leak into another's responses, eroding utility and trust. AFA combines voice-based speaker identification with per-user memory stores to enable identity-aware, personalized dialogue across multiple users.

To train and evaluate AFA, the team constructed PAT (Personalized Agent chaT), a synthetic dataset of 58,289 persona-grounded dialogue turns spanning 133 user profiles across 12 real-world scenarios. They evaluated AFA on five LLM back-ends, with a LLaMA-2-70B model fine-tuned on PAT achieving the highest overall performance in standard response-quality benchmarks. To directly measure persona confusion prevention, they introduced an interleaved multi-user evaluation protocol with a novel metric, Persona Attribution Accuracy (PAA). Their results showed that identity-aware routing improved PAA from 35.7% to 61.3%, and human evaluation confirmed significantly higher perceived personalization in routing-enabled responses. This work establishes identity-aware user routing as the critical component for preventing persona confusion in multi-user conversational systems.