TraderBench: How Robust Are AI Agents in Adversarial Capital Markets?

A team of researchers including Xiaochuang Yuan and Hui Xu has introduced TraderBench, a novel benchmark designed to rigorously evaluate the robustness of AI agents in financial markets. The framework addresses key limitations of existing evaluation methods by combining expert-verified static tasks—like knowledge retrieval and analytical reasoning—with dynamic, adversarial trading simulations. These simulations are scored purely on realized performance metrics such as Sharpe ratio, returns, and drawdown, eliminating the variance introduced by LLM-based judges. TraderBench features two specialized tracks: a crypto trading environment with four progressive market-manipulation transforms, and an options derivatives scoring system that assesses P&L accuracy, Greeks, and risk management. Crucially, the trading scenarios can be refreshed with new market data to prevent benchmark contamination, ensuring evaluations remain relevant.

The researchers evaluated 13 AI models, ranging from 8B parameter open-source models to frontier models, across approximately 50 tasks. The results were revealing: 8 out of 13 models scored around 33 points in the crypto track with less than a 1-point variation across different adversarial conditions, indicating they employ fixed, non-adaptive strategies. Furthermore, while extended thinking (like chain-of-thought prompting) significantly boosted performance on retrieval tasks by 26 points, it had virtually no impact on actual trading outcomes, yielding only a +0.3 point change in crypto and a -0.1 point change in options. These findings underscore a critical gap in current AI agent development: a lack of genuine adaptation to dynamic, adversarial market conditions. The benchmark's performance-grounded approach highlights the need for evaluation frameworks that move beyond static knowledge tests to measure real-world financial decision-making under pressure.