New DP framework for decentralized POMDPs with delayed sharing

A new paper from Charalambous, Guvercin, and Djouadi tackles a fundamental problem in decentralized control: how to optimally coordinate multiple agents who share information with a delay. Building on Witsenhausen's 1971 formulation of decentralized partially observable Markov decision processes (POMDPs) with T-step delayed sharing, the authors develop structural properties of optimal strategies using a decentralized sequential team equilibrium concept. This generalizes person-by-person optimality from static team theory to dynamic settings.

Their key contribution is a set of dynamic programming (DP) equations that reveal a compression property: each agent's delayed sharing information pattern can be reduced into three components—a private posterior distribution conditioned on its own history, a centralized posterior shared by all agents, and the agent's private information state. The DP framework satisfies Markov recursions and a separation principle, allowing optimization over action spaces instead of strategy spaces. This substantially extends Witsenhausen's Assertion 8 and opens the door to scalable decentralized control algorithms.