Instructing LLMs to Negotiate using Reinforcement Learning with Verifiable Rewards

A research team from MIT and Caltech has published a paper demonstrating how Reinforcement Learning from Verifiable Rewards (RLVR) can teach large language models to become expert negotiators. Their framework trains a mid-sized (30B parameter) buyer agent against a regulated LLM seller across a wide distribution of real-world products. The key innovation is grounding the reward signal directly in verifiable outcomes: maximizing economic surplus (the value gained from a deal) and strictly adhering to private budget constraints. This approach avoids vague or ungrounded objectives, forcing the AI to develop concrete, effective strategies.

During training, the agent exhibited a novel four-phase strategic evolution. It progressed from naive bargaining, to using aggressive starting prices, through a phase of negotiation deadlock, and ultimately developed sophisticated persuasive skills. The final trained 30B agent achieved a remarkable result: it significantly outperformed frontier models like GPT-4 and Claude 3 Opus—which are over ten times its size—in extracting surplus from negotiations. Furthermore, the agent demonstrated robust generalization, effectively negotiating with stronger counterparties it never encountered during training and remaining resilient against hostile, adversarial seller personas programmed to be difficult.