Stanford researchers unveil SolarChain-Eval to test AI agents in energy markets

Researchers have introduced SolarChain-Eval, a physics-constrained benchmark for evaluating trustworthy economic agents in decentralized energy markets. The framework uses a Gymnasium-compatible Markov Decision Process where agents make hourly decisions, and assesses policies across dimensions including market utility, physical safety, slippage, action smoothness, spatial fairness, and auditability. It includes an LLM-based Planner/Auditor layer that records structured intervention logs. Experiments with static, random, myopic, RL, and RL+LLM policies reveal a utility-safety trade-off: RL agents can improve utility but produce unsafe behavior, and when the physics penalty is removed, reward-maximizing agents exploit invalid generation and create artificial liquidity. The LLM Planner/Auditor improves auditability and mitigates selected risks but cannot fully compensate for a misspecified reward function.