Catastrophic Risk Characterization

• Catastrophic risk characterization is a multidisciplinary framework that defines, assesses, and mitigates extreme, low-probability events using robust statistical and causal methods.
• It integrates qualitative narrative scenario generation with quantitative underwriting by decomposing risk factors and applying extreme value theory to model tail risks.
• The approach emphasizes argument-level uncertainty, systematic scenario expansion, and pathway analysis to derive actionable, auditably traceable profiles for high-stakes applications.

Catastrophic risk characterization concerns the rigorous, multidisciplinary methodology for defining, assessing, quantifying, and mitigating risks whose outcomes, while extremely rare, have severe to existential consequences. The field requires robust techniques from probability theory, decision analysis, causality, extreme value theory (EVT), and qualitative scenario generation. This synthesis integrates foundational frameworks, the decomposition of argument sources, advances in qualitative-quantitative workflow, modern risk factor taxonomies, technical modeling tools, and operational protocols for high-stakes contexts (0810.5515, Carpenter et al., 26 Nov 2025, Barrett et al., 2022, Bales et al., 2024).

1. Conceptual Framework: Probability, Argument Failure, and Uncertainty Decomposition

The total probability of a catastrophe (denoted $X$) is fundamentally decomposed as

$P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$

where $A$ is the event that the expert argument for system safety is sound, and $\neg A$ that it is flawed (0810.5515). Critically, $P(X|A)$ is the probability of catastrophe given the argument holds; $P(X|\neg A)$ captures the risk under argument failure.

A granular approach splits $A$ into the conjunction of:

• $T$: adequacy of the underlying scientific theories;
• $M$: adequacy of the model constructed on those theories;
• $C$: correctness of calculations (analytical, numerical, or computational).

Thus,

$P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$0

Assuming approximate independence, the probability of argument failure is often estimated as

$P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$1

This framework distinguishes argument failure—which can dominate in low-probability/high-stakes domains—from standard model/parameter uncertainty that presumes the theory and calculational substrates are reliable (0810.5515).

2. Qualitative and Quantitative Event-Space Generation: "Dark Speculation" and Lévy-Based Risk Aggregation

A persistent obstacle is the "Lucretius problem": the difficulty of reasoning about event types not observed in historical data. The "dark speculation" methodology systematically expands the catastrophic event space by generating Loss-Inducing Catastrophic AI Narratives (LICAINs) using narrative scenario planning (Carpenter et al., 26 Nov 2025). Each narrative is formalized as a directed acyclic graph of happenings with contextual attributes, mapped into stochastic event specifications with trigger conditions, causal chains, and a loss function template (e.g., Pareto- or Weibull-distributed jump magnitudes).

Quantitative underwriting follows: for each hypothetical scenario, actuarial and domain experts elicit

• jump intensity $P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$2 (frequency),
• damage scale $P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$3,
• tail index $P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$4.

The total risk process is then modeled as a Lévy process

$P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$5

where $P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$6 and $P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$7 are jump sizes sampled from a mixture of distributions across both historical and speculative scenarios (Carpenter et al., 26 Nov 2025). Summary statistics include the probability of exceeding severe loss thresholds, expected shortfall, and tail index computation to assess "fat-tailedness".

This methodology mandates the independence of scenario-generation teams and underwriters, and the parallel, iterative examination of risk categories.

3. Risk Factor Taxonomies and Dimensional Decomposition

Modern AI-specific risk analysis classifies catastrophic risks by origin and propagation properties. Barrett et al. systematize catastrophic risk factors, including:

• Correlated robustness failures,
• Accumulated and compounding systemic risks,
• Misuse of multi-purpose or general-purpose AI,
• Mis-specified objectives,
• Recursive self-improvement,
• Adversarial attacks and misuse cases (Barrett et al., 2022).

Chin's dimensional characterization encodes each scenario $P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$8 as a vector in a seven-dimensional space: $P(X) = P(X|A)\,P(A) + P(X|\neg A)\,P(\neg A)$9 with well-defined ordinal or scalar values for intent (accidental/malicious), competency (failure-driven/capability-driven), entity (human/AI/hybrid), agent polarity, pathway linearity, reach (internalized/externalized), and harm order (first/second/etc.) (Chin, 8 Aug 2025). This decomposition supports scenario clustering, ensures systematic coverage, and enables generalizable mitigation rules.

4. Technical Risk Modeling: EVT, Cascading, Overfitting, and Stochastic Process Approaches

Extreme Value Theory (EVT) underpins the quantification of tail-dependent catastrophic risks in many domains:

• Marked Poisson point processes (MPP) and Peaks-over-Threshold (POT) methodology classify accidents by magnitude, fit generalized Pareto (GPD) distributions, and allow simulation-based pricing or capital assessment for reinsurance contexts (Leppisaari, 2013).
• Block maxima EVT and Generalized Extreme Value (GEV) distributions yield interpretable return levels for rare financial events (Cotter, 2011).
• Reinforcement learning can minimize catastrophic risk using policy gradients derived from EVT-fitted tail models, targeting CVaR or similar measures at extreme quantile thresholds (Davar et al., 2024).

Cascading Alternating Renewal Process (CARP) models

$A$0

explicitly represent internal and external risk-activation intensities, supporting rigorous estimation and limiting risk forecast uncertainty by providing asymptotic error bounds as data accumulates (Lin et al., 2017).

In high-dimensional machine learning, phase transitions demarcate "benign," "tempered," and "catastrophic" risk regimes, governed by spiked covariance structure, spike strength, and target alignment. The excess risk $A$1 can diverge for intermediate spike strengths and aligned signals, representing an analytically tractable signature of catastrophic overfitting (Li et al., 1 Oct 2025).

5. Pathway Modeling and Causal Risk Quantification

Graph-based models formalize risk propagation from initial hazards to terminal catastrophic harm. Given a risk pathway encoded as a DAG $A$2, the total risk is the sum over all consequences, weighted by the joint path probabilities: $A$3 where $A$4 is the chain of conditional probabilities along the path from hazard $A$5 to consequence $A$6 and $A$7 is the loss function (Chin, 8 Aug 2025). Mitigative and control interventions act by attenuating edge probabilities, enabling explicit calculation of risk reduction.

6. Methodologies for Identification, Assessment, and Management

Standardized frameworks for high-consequence AI risk management apply structured methodologies across the lifecycle: Map (scenario elicitation), Measure (impact/probability estimation), Manage (control implementation, prioritization), and Govern (oversight, incident reporting). Templates for reporting explicitly track scenario descriptions, impact ratings, catastrophic-risk-factor tags, probability estimates, risk ownership, and residual controls (Barrett et al., 2022).

Effective practice includes:

• Peer review and independent re-derivation,
• Use of multiple, independent lines of argument,
• Red vs Blue/team adversarial critique,
• Explicit quantification of argument reliability components (0810.5515).

Risk assessment techniques from other high-stakes domains (scenario analysis, fishbone/Ishikawa diagrams, advanced causal mapping, Delphi expert elicitation, system-theoretic process analysis, risk matrices) are adapted for AGI and AI system characterization (Koessler et al., 2023).

7. Functional Implications and Synthesis

The dominant lesson is that, for all ultra-low-probability, high-stakes risks, the probability that standard arguments for safety are themselves flawed ($A$8) can and often does surpass the modeled event conditional probability ($A$9). Accurate characterization mandates both rigorous decomposition of argument reliability and structured scenario/narrative expansion, followed by robust quantitative aggregation using EVT, stochastic process, or multidimensional causal tools (0810.5515, Carpenter et al., 26 Nov 2025, Chin, 8 Aug 2025).

Robust catastrophic risk quantification thereby requires the explicit modeling of

• Argument-level uncertainty,
• Comprehensive event-space through both observed and systematically constructed black-swan scenarios,
• Extreme-value-tail metrics,
• Interconnectedness and cascade amplification,
• Scenario-specific control placement and pathway attenuation.

This multi-layered characterization yields actionable and auditably traceable risk profiles, supporting more defensible policy formation and technical risk limits in high-consequence domains.