New MCTS variance estimator slashes bias in decision-making
Recursive estimator corrects hidden bias that standard methods miss—improving inventory and donor matching.
Researchers reveal that conventional sample-variance estimators in finite-horizon MDP tree search are biased and the bias does not vanish in the limit. They derive a recursive, consistent variance estimator that decomposes variance into three components: immediate reward, state transitions, and future value function uncertainty. An efficient node-local implementation is provided. Integrated into Monte Carlo Tree Search (MCTS), the new estimator improves algorithm performance in numerical examples from inventory control and kidney paired donation matching relative to the i.i.d.-based sample variance baseline.
- Standard i.i.d.-based sample variance estimator in MCTS is biased and bias persists as sample size grows
- New recursive estimator decomposes variance into reward, transition, and future-value components
- Integrated into MCTS, it improves decision quality in inventory control and kidney donor matching experiments
Why It Matters
Correcting this hidden bias could make AI planning systems more reliable in high-stakes logistics, healthcare, and finance.