APG4RecSim: LLM Framework Boosts Recommendation Simulations by 7%

Modern recommender systems increasingly rely on LLM-based agent simulations for real-time evaluation, but existing approaches focus on memory and action modules while neglecting profile generation—a critical component for realistic agent behavior. Researchers from the University of Melbourne and Monash University introduce APG4RecSim (Automated Profile Generation for Recommendation Simulation), a framework that constructs coherent, bias-resilient user profiles with minimal human oversight. The system addresses the scarcity of simulation-specific datasets by automating profile creation, enabling scalable and generalizable evaluation across different recommendation contexts.

Extensive experiments on three benchmark datasets show APG4RecSim achieves best overall performance on discrimination, ranking, and rating tasks. It improves ranking quality by up to 7% in nDCG@10 and reduces rating distribution divergence by 8% in Jensen-Shannon Divergence compared to existing profile-generation baselines. Notably, the generated profiles are resilient to popularity and position biases and maintain stable performance across different LLMs and datasets. The work, accepted at SIGIR 2026, marks a significant step toward more reliable and scalable recommender system evaluation.