Heteroscedastic GP-UCB cracks dynamic incentive design with unknown agent preferences

The principal-agent problem is a cornerstone of economics: a principal (e.g., a regulator or company) designs incentive contracts to steer an agent (e.g., a worker or EV owner) whose actions are unobservable and whose preferences are unknown. In dynamic settings, this becomes a sequential decision problem. Existing theory assumed deterministic agent utility, which makes the principal's expected utility discontinuous and forces computationally intractable discretizations of the contract space—a fundamental bottleneck.

To fix this, the TU Delft team introduces a stochastic component into the agent's utility model, capturing physical and behavioral variation. They formally prove this restores continuity of the principal's expected utility. That continuity unlocks a clean formulation: the interaction becomes a structured multi-armed bandit with heteroscedastic (non-uniform) noise. Their algorithm, Heteroscedastic GP-UCB, uses a Neural Network (Arcsin) kernel that matches the utility landscape's non-stationary, sigmoidal geometry. For an m-dimensional compact contract space, they establish a high-probability cumulative regret bound of O(√T (log T)^{m+1}).

They validate the framework on a real-world Vehicle-to-Grid (V2G) incentive design problem, proving its equivalence to a dynamic principal-agent setup and demonstrating superior economic performance for grid aggregators. This pushes incentive design from theory to deployable algorithms for smart grid, labor markets, and autonomous systems.