A Survey on LLM-based Conversational User Simulation
New taxonomy covers user granularity and simulation objectives for high-fidelity synthetic dialogue...
A large team of 30 researchers from institutions including Adobe Research, Amazon, and multiple universities has released a comprehensive survey on LLM-based conversational user simulation, now available on arXiv (arXiv:2604.24977). The paper, submitted in August 2025 and approved under survey MOD-81000, reviews how large language models enable high-fidelity generation of synthetic user conversations. The authors introduce a novel taxonomy that categorizes approaches by user granularity (e.g., individual vs. population-level simulation) and simulation objectives (e.g., task completion, persona consistency, or dialogue fluency).
The survey systematically analyzes core techniques, including prompt engineering, fine-tuning strategies, and evaluation methodologies like automatic metrics and human judgment. It covers applications in human-computer interaction (cs.HC) and computation and language (cs.CL), highlighting how these simulations support chatbot training, virtual assistant testing, and user behavior modeling. The paper identifies open challenges such as handling long-tail user behaviors, ensuring diversity in generated dialogues, and developing standardized benchmarks. By organizing existing work under a unified framework, the survey aims to keep the research community informed and facilitate future advancements in conversational AI.
- 30 researchers from Adobe, Amazon, and universities collaborated on the survey
- Novel taxonomy covers user granularity (individual vs. population) and simulation objectives
- Identifies open challenges like long-tail behaviors and lack of standardized benchmarks
Why It Matters
Standardizes the chaotic field of LLM user simulation, accelerating chatbot and virtual assistant development.