AI Safety

Stuart Armstrong's Pragmatic FDT sidesteps equivalence pitfalls with isomorphism

New pragmatic variant of FDT uses isomorphism to avoid defining 'same algorithm'.

Deep Dive

Stuart Armstrong's new post tackles two core issues in decision theory. First, he introduces p-FDT (pragmatic functional decision theory) to sidestep the long-standing problem of defining when two algorithms are equivalent. Instead of requiring an abstract definition, p-FDT searches for isomorphisms—bijections between input and output sets that make two functions identical up to relabeling. For example, two calculators that differ only by a minus sign on outputs are isomorphic, so an agent can treat them as running the same algorithm without needing a metaphysical fact of the matter. The agent computes a causal baseline (CDT action), then evaluates each candidate isomorphism by assuming that choosing the agent's decision map also sets the isomorphic process in the world. It weights the isomorphism's truth probability and only adopts it if it beats the baseline in expected utility.

Second, Armstrong argues that when predictors make counterfactual predictions—such as predicting what an agent would do before it decides—the decision problem shades into game theory. This explains why EDT/TDT/UDT/FDT can appear irrational in some branches: it's the old blackmailer paradox in new clothing. By reframing these as game-theoretic interactions, the apparent irrationality dissolves. The post is a response to Bentham's Bulldog's critique that FDT lacks academic support, and Armstrong positions himself as a published academic decision theorist who believes defined variants of FDT can succeed where the original falters.

Key Points
  • p-FDT defines algorithmic equivalence via isomorphisms between functions, avoiding the need for abstract 'same algorithm' definitions.
  • The agent computes a causal baseline (CDT) and only adopts an isomorphism if it improves expected utility, weighted by the isomorphism's probability.
  • Armstrong argues that predictors making counterfactual predictions transform decision theory into game theory, explaining away irrational outcomes like paying blackmailers.

Why It Matters

Provides a practical, defined variant of FDT that could make it more palatable to academic decision theorists and clarify AI alignment debates.

📬 Get the top 10 AI stories daily