Updateless Decision Theory

• Updateless Decision Theory (UDT) is an idealized decision-making framework integrating policy selection and logical counterfactuals to overcome flaws in EDT and CDT.
• UDT evaluates an entire mapping from observations to actions, enabling improved performance in Newcomblike dilemmas and Evidential Blackmail scenarios.
• UDT employs logical counterfactuals to model dependencies among agents’ decisions, promoting cooperative outcomes in dilemmas like the Prisoner’s Dilemma.

Updateless Decision Theory (UDT) is an approach to idealized decision-making that seeks to address fundamental failures of classical decision theories—specifically, the incapacity of both Evidential Decision Theory (EDT) and Causal Decision Theory (CDT) to describe a decision procedure suitable for alignment of artificial systems with human interests. UDT integrates two key innovations: policy selection, which emphasizes choosing an entire mapping from observations to actions rather than conditioning on specific observations, and logical counterfactuals, which allow counterfactual reasoning about logically correlated agents or predictors even in the absence of causal connection (Soares et al., 2015).

1. Shortcomings of Classical Decision Theories

Classical EDT evaluates actions via conditioning, computing the expected utility of action $a$ as $EU_{EDT}(a) = E[ U | A = a ]$. In deterministic environments, events $A = a$ frequently have probability zero, leaving $EU_{EDT}(a)$ ill-defined. Even with injected stochasticity, EDT suffers from the "managing the news" problem; it recommends actions that skew evidence without causal efficacy, as in Evidential Blackmail, where paying to a blackmailer is recommended solely because it is evidence for a benign universe.

CDT formalizes decision-making using Pearl's do-calculus: the agent's action node $A$ is surgically intervened upon ($do(A=a)$), downstream effects are propagated, and utilities are computed as $EU_{CDT}(a) = E[ U | do(A = a) ]$. CDT thus avoids the evidential feedback loop but fails in cases involving logical dependence, notably Newcomblike problems. For example, in a one-shot Prisoner's Dilemma against a perfect copy, CDT defects, treating the copy's choice as fixed, missing utility available from exploiting logical equivalence ("my choice if and only if the copy's choice") (Soares et al., 2015).

2. Principle of Policy Selection

UDT replaces action-evaluation under uncertainty with policy selection. Rather than choosing an individual action in response to a specific observation, an agent selects an entire function—called a policy—that specifies an action for each possible observation. This transition from action-selection to policy-selection overcomes the pathologies imposed by conditioning or causal surgery, notably in counterfactual games where correlation between agents’ behaviors is logical rather than physical.

From this vantage, the decision process is reframed: the agent evaluates all possible policies, computes the utility expected from each when consistently applied across all relevant instances (including across copies or predictors), and chooses the policy that maximizes expected utility. This approach outperforms single-action evaluation in environments where other agents or system components are logically connected to the agent's policy.

3. Logical Counterfactuals

Logical counterfactuals extend counterfactual reasoning to handle scenarios where the outcomes depend on mathematical or programmatic correlations between agents or predictors. Classical do-calculus and standard conditionals are inadequate when the decision of one agent is entangled with that of others via logical dependence rather than causality.

In the one-shot Prisoner's Dilemma with a perfect copy, recognizing that "my choice ⇔ the copy's choice" allows an agent to select policies that achieve mutual cooperation ($2$ rather than $1$ per agent), maximizing global utility. These logical counterfactuals formally model "what would happen if I used a different policy," even when this entails counterparts adopting analogous policies.

4. Blackmail and Newcomblike Problems

UDT provides solutions to canonical challenges such as the Evidential Blackmail and Newcomblike scenarios that stymie both EDT and CDT. In the blackmail setup, both EDT and CDT produce counterintuitive recommendations to pay, either to align evidence or to minimize direct loss once blackmailed. UDT, by considering the selection of a global policy that consistently disincentivizes blackmail, avoids both evidential exploitation and credulous responses, aspiring to