Microsoft’s SocialRL trains 4B models to out-negotiate GPT-5
A 4B-parameter model trained with SocialRL beats GPT-5 in negotiation games by 122% with theory-of-mind scaffolds...
Deep Dive
The authors present SocialRL, a general recipe that trains social reasoning directly, applied to a 4B model across six domains. With in-domain training, the 4B matches or exceeds the GPT-5 family per domain, closing 73–122% of the baseline-to-frontier gap on the negotiation games. Guided by cross-domain transfer structure, two strategies—cascade RL and multi-teacher on-policy distillation—consolidate per-domain specialists into a single unified 4B that reaches 0.627 average utility across all
Key Points
- SocialRL, a 4B-parameter model trained by Microsoft, matches or exceeds GPT-5 family performance across six negotiation domains.
- In-domain training closes 73-122% of the gap to frontier baselines, with buyers anchoring 78% below target vs. 3% for untrained models.
- A unified 4B model achieves 0.627 average utility, outperforming GPT-4.1, GPT-5.1, and GPT-5.2, using cascade RL and ToM-augmented distillation.
Why It Matters
Enables AI agents to negotiate more effectively, reducing privacy leaks and improving outcomes in real-world delegated tasks.