New Framework Auto-Optimizes Prompts for LLM-Based User Simulators in CRS

Conversational recommender systems (CRS) are vital for next-gen intelligent recommendations, enabling real-time preference elicitation and adaptive suggestions. However, evaluating CRS via human studies is expensive and time-consuming, while interaction data is often unavailable due to privacy concerns. LLM-based user simulators offer a solution by generating synthetic interactions, but existing approaches suffer from systematic positive bias, data leakage, and limited behavioral diversity, relying on brittle manual prompt engineering that demands extensive domain expertise.

To overcome these hurdles, the authors propose a multi-objective framework that automatically optimizes prompts for LLM-based user simulators. This framework simultaneously mitigates bias, data leakage, and diversity issues. Experimental results demonstrate improved behavioral alignment with human interaction patterns compared to baseline methods across diverse prompt settings. The work, to be published in IEEE ICDEW 2026, provides a scalable, automated approach to CRS evaluation and training data generation, reducing the need for costly human studies and manual tuning.