Existential Prioritization (EP)

• Existential Prioritization (EP) is a quantitative framework that balances AI-specific and non-AI existential threats through rigorous hazard-rate comparisons.
• It uses mathematical models to assess the delay-benefit tradeoff, helping determine whether to postpone or accelerate ASI deployment.
• The framework informs policy by emphasizing increased AI alignment research and coordinated control over emergent high-risk technologies.

Existential Prioritization (EP) is a quantitative framework for analyzing and reducing existential risk from emerging technologies, especially artificial superintelligence (ASI). EP formalizes the real-time tradeoffs societies face between allocating marginal years to AI safety preparation and the accumulation of catastrophic risk from all other sources, such as engineered pandemics or molecular nanotechnology. Central to EP is a rigorous hazard-rate comparison, enabling policymakers to prioritize delaying, accelerating, or optimally timing ASI deployment according to the relative contributions of AI-specific and other existential risks (McAleese, 2022).

1. Mathematical Formalization of Existential Risk

Let $t$ denote the year at which the first ASI comes online. The total existential risk up to time $t$ decomposes into two terms:

• $R_{\mathrm{AI}}(t)$: The “step risk” of extinction from the first ASI if it appears in year $t$.
• $R_{\mathrm{other}}(t)$: The cumulative probability of extinction from all non-AI causes accrued up to year $t$ (e.g., natural disasters, anthropogenic “state” risks, or other novel catastrophic technologies).

The total risk, $R_{\mathrm{total}}(t)$, is given by: $$R_{\mathrm{total}}(t) = R_{\mathrm{other}}(t) + R_{\mathrm{AI}}(t)$$ $R_{\mathrm{other}}(t)$ is monotonically increasing with $t$, as more time leads to greater exposure to non-AI existential threats. $t$0 is typically decreasing due to advancements in alignment research, which reduce the probability that the first ASI will cause extinction.

2. Delay-Benefit Inequality and Optimization Criterion

EP provides an explicit criterion for whether delaying ASI is beneficial. Considering a small delay $t$1 in ASI deployment: $t$2 With $t$3 (accumulation of other risks) and $t$4 (AI risk mitigated by extra time for alignment), the condition for net existential risk reduction under delay is: $t$5 Introducing the annualized benefit $t$6 and annualized harm $t$7, the delay is net beneficial precisely when: $t$8 If the reverse holds, acceleration rather than delay reduces existential risk. The optimal strategy is thus dictated by the instantaneous hazard rates of AI and non-AI extinction mechanisms.

3. Functional Forms and Parameters for Risk Components

McAleese provides concrete stylized models for the two major risk curves:

• $t$9: Approximated as a linear or mildly convex function, declining from a high baseline (e.g., $R_{\mathrm{AI}}(t)$0 for 50% chance of extinction if ASI arrives in 2022) with slope $R_{\mathrm{AI}}(t)$1/yr:

$R_{\mathrm{AI}}(t)$2

where $R_{\mathrm{AI}}(t)$3 is the floor if alignment is nearly perfected.

• $R_{\mathrm{AI}}(t)$4: Composed of a slow, linear “state risk” at rate $R_{\mathrm{AI}}(t)$5/yr, plus discontinuous jumps as new non-AI catastrophic technologies mature:

$R_{\mathrm{AI}}(t)$6

Here, $R_{\mathrm{AI}}(t)$7 is the Heaviside step, $R_{\mathrm{AI}}(t)$8 the timeline for each i-th novel threat (e.g., synthetic biology, nanotech), and $R_{\mathrm{AI}}(t)$9 the associated extinction probability. $t$0 everywhere.

Critical factors altering these curves:

• Hardware overhang shifts $t$1 left and accelerates arrival of $t$2 jumps.
• Cognitive enhancement steepens the decline of $t$3 but may also advance capability timelines.
• War and weak coordination flatten $t$4 (slower safety progress) and increase $t$5.

4. Tradeoffs and Regime Analysis

At any candidate $t$6, the marginal tradeoff is:

• (A) Marginal reduction in AI risk per year of delay: $t$7.
• (B) Marginal increase in other risks per year: $t$8.

When $t$9 (e.g., if $R_{\mathrm{other}}(t)$0/yr and $R_{\mathrm{other}}(t)$1/yr), delay is strongly net beneficial. This applies if non-AI risks are dominated by slow-accumulating background hazards. If step-risks from other technologies emerge, raising $R_{\mathrm{other}}(t)$2 to parity or beyond, further delay becomes harmful and accelerating ASI can minimize $R_{\mathrm{other}}(t)$3. For intermediate values where $R_{\mathrm{other}}(t)$4, there exists a finite optimal wait time $R_{\mathrm{other}}(t)$5 at which: $R_{\mathrm{other}}(t)$6 Variations around $R_{\mathrm{other}}(t)$7 do not improve total existential safety to first order.

5. Policy Implications and Strategic Recommendations

Quantitative existential prioritization informs several robustly positive interventions:

• Expand and fund AI alignment research: Increasing the size and output of alignment efforts steepens $R_{\mathrm{other}}(t)$8 and raises $R_{\mathrm{other}}(t)$9.
• Broaden existential risk analysis: Improving estimates for $t$0, arrival times $t$1, and step risk probabilities $t$2 enables rational delay or acceleration choices.
• Implement differential technological development: Deliberately slowing high-risk non-AI technologies (e.g., biosecurity, nanotech regulation) while accelerating risk mitigation flattens $t$3 and sustains the regime where delay remains net positive.
• Enhance global governance and coordination: Strengthening cooperation lowers $t$4 and moderates shocks that could steepen hazard rates.
• Contain hardware overhang: Export controls or vetted distribution of advanced hardware prevent undesirable leftward shifts in $t$5, preserving leverage for optimal timing decisions.

Conditional strategies such as AI arms races to preempt other risks are not generally advised, as they may backfire without clear evidence that non-AI existential hazards have overtaken the marginal benefits of AI alignment progress.

6. Summary and Prospective Directions

Existential Prioritization reframes the management of global catastrophic risk as the continuous estimation and balancing of marginal hazard rates from AI and non-AI sources. The core principle is to maximize the marginal reduction in extinction risk from further alignment work ($t$6) while minimizing the marginal accumulation of other threats ($t$7). Preferred approaches focus on expanding alignment research, moderating the introduction and proliferation of dangerous novel technologies, and enhancing global coordination, thereby ensuring that delaying ASI—when possible—remains a net positive for civilization’s survival (McAleese, 2022).