New benchmark reveals LLMs hit a personalization plateau in sales

A new paper from researchers at (presumably) industry and academia introduces SDR-Bench, a benchmark for evaluating the personalization capabilities of large language models in a two-party sales context. The benchmark adapts the Bayesian Persuasion framework to generative agents and comprises 6,279 real customer success stories spanning 22 industries and roughly 200 enterprises. The test simulates temporally constrained outreach to prevent data leakage. Across frontier LLMs and deep-research agents, the team observed a consistent "personalization plateau" — no model could statistically separate successful from unsuccessful outreach when tested on a Fortune 100 tech cohort. This suggests that current LLMs fail at true two-party personalization, where the sender and receiver have independent objectives.

To validate the framework, the researchers conducted a field deployment with 12 professional sales representatives. They found that 48% of model-generated content was rated immediately useful, and senior-expert agreement reached a Pearson correlation of 0.82 — indicating strong human alignment on quality. Despite this, the inability of models to predict actual outcomes highlights a critical gap. The authors have released both SDR-Arena (an interactive evaluation platform) and SDR-Bench publicly to support reproducible research. This work underscores that while LLMs can produce plausible personalized messages, they still lack the strategic insight to drive desired actions in real-world negotiations.