DRL framework boosts battery profits 7.56% with adaptive FCR bids

A new paper from researchers Celle Hendrickx, Fabio Pavirani, and Chris Develder, presented at ACM Sustainability Week 2026, addresses a key limitation in battery energy storage systems (BESS) participating in European ancillary service markets. Traditionally, batteries bid a uniform Frequency Containment Reserve (FCR) capacity that remains constant for the entire control period. This static approach fails to balance the trade-off between reserving energy for FCR delivery and using it for imbalance arbitrage (trading on price differences in real-time balancing markets).

To overcome this, the authors propose a two-stage framework. Stage one derives a time-varying FCR bid sequence using data-driven Monte Carlo optimization. Stage two employs a Deep Reinforcement Learning (DRL) agent that leverages residual flexibility for real-time imbalance trading while proactively managing the battery's State of Energy (SoE) to ensure FCR compliance. By incorporating daily cycle budgets and time-varying reserve commitments, the framework achieves a 7.56% profit increase compared to uniform baselines. These results demonstrate that non-uniform bidding can better align reserve obligations with rapidly changing imbalance opportunities, unlocking additional value for battery operators and supporting grid stability as renewable energy penetration grows.