Counterfactual Mugging as Limit of Psy-kosh's Problem: A Decision Theory Insight
New analysis shows classic decision theory puzzle reduces to a simpler non-anthropic problem with 10 copies and a shared bank account.
The classic counterfactual mugging problem, introduced to motivate updateless decision theory (UDT), involves Omega flipping a coin: on Heads it demands $1 from you; on Tails it offers you $5 only if it predicts you would have paid on Heads. This creates a counterfactual dependence that standard decision theories struggle with. The author reframes the scenario by replacing the counterfactual with a simulation: on Tails, Omega simulates a version of you from the Heads world and lets that simulated you choose. This simulation version may not care about the outside version, but a rational agent would self-modify to ensure caring.
The author then shows how this problem is a limiting case of Psy-kosh's non-anthropic problem, which involves 10 identical copies of you with a shared bank account. Omega flips a coin: if Heads, 9 copies receive a GreenMarble; if Tails, only 1 receives a GreenMarble. Those with a GreenMarble are offered a single payment (e.g., $1) to the shared account. The choice made by the GreenMarble recipients determines the outcome for all. In the limit where the probability of Heads approaches 100% (i.e., 9 out of 10 copies becoming all 10), this reduces exactly to counterfactual mugging.
This equivalence strengthens the case for UDT: rational agents should commit to a policy that maximizes expected utility across all possible observer-moments, not just the current one. The post also connects to Conitzer's 2017 Dutch book for EDT sleeping beauties and highlights how anthropic reasoning can be modeled without explicit counterfactual simulation.
- Counterfactual mugging is rephrased as a simulation-based problem where your decision in the Heads world affects the Tails outcome.
- Psy-kosh's non-anthropic problem uses 10 identical copies with a shared bank account; the limit of 90% Heads -> 100% yields counterfactual mugging.
- The equivalence supports updateless decision theory (UDT) as the rational approach for agents facing anthropic uncertainty.
Why It Matters
Sharpens foundations for decision theory under uncertainty, with direct implications for AI alignment and self-modifying agents.