Causal Ladder: Trustworthy AGI

• Causal Ladder of Trustworthy AGI is a three-layer framework aligning AI's observational, interventional, and counterfactual reasoning with Pearl’s causal hierarchy.
• It integrates causal estimation methods, such as IRM and DML, to address spurious correlations and ensure consistent performance under distribution shifts.
• The framework emphasizes enhancing AGI safety by enforcing intervention-based reasoning and reflective counterfactual analysis, crucial for robust and aligned intelligence.

The Causal Ladder of Trustworthy AGI is a framework that organizes progress toward trustworthy artificial general intelligence by aligning AI capability with Pearl’s Ladder of Causation: association, intervention, and counterfactuals. In this view, systems restricted to observational prediction, $P(Y\mid X)$, remain highly capable correlation machines, but they are brittle under distribution shift, vulnerable to spurious features, and poorly equipped for alignment, safety, and calibrated uncertainty. Trustworthy AGI instead requires ascent to interventional reasoning, $P(Y\mid \mathrm{do}(X))$, and counterfactual reasoning, $P(Y_x\mid X=x',Y=y')$, so that models can answer not only “what is?” but also “what happens if we act?” and “what would have happened if we had acted differently?” (Yang et al., 2024, Fokoué, 22 May 2026). Recent survey, benchmark, and architecture papers extend this perspective across robustness, fairness, privacy, explainability, causal evaluation, and AGI governance (Liu et al., 2023, Ganguly et al., 2023, Chang, 13 Jan 2026, Geng et al., 9 Feb 2026).

1. Conceptual origin and scope

The term was introduced explicitly as part of the AI-$45^{\circ}$ Law roadmap, which treats AI capability and AI safety/trustworthiness as co-evolving dimensions. The ideal trajectory lies along a 45° line, conceptually $S \approx C$, with a Yellow Line as an early-warning threshold and a Red Line marking regions in which capability outstrips safety badly enough to create existential or catastrophic risks. Within that roadmap, the Causal Ladder of Trustworthy AGI is a three-layer structure: the Approximate Alignment Layer, the Intervenable Layer, and the Reflectable Layer (Yang et al., 2024).

This formulation is continuous with earlier causal work on trustworthy machine learning. Surveys of trustworthy ML recast robustness, fairness, adversarial robustness, interpretability, and large pretrained model alignment through Pearl’s hierarchy, arguing that many apparently separate methods are implicitly trying to control which features, mechanisms, or pathways a model relies on (Liu et al., 2023, Ganguly et al., 2023). In parallel, causal influence diagrams were proposed as a common language for AGI safety frameworks, making optimization targets, information links, and wireheading or feedback-manipulation incentives visually explicit (Everitt et al., 2019).

Pearl’s ladder	AGI layer	Core question
Association	Approximate Alignment Layer	“What is it?”
Intervention	Intervenable Layer	“What will happen if we intervene on $X$?”
Counterfactual	Reflectable Layer	“What would have happened if we had acted differently?”

The broader significance of the ladder is that it turns a philosophical distinction into a systems taxonomy. At the bottom are models aligned approximately through observational data and post hoc preference shaping. Above them are systems whose internal reasoning or behavior can be inspected and altered under interventions. At the top are systems capable of reflective, counterfactual self-correction and long-horizon evaluation of alternative policies (Yang et al., 2024).

2. Pearl’s hierarchy as a hierarchy of trust

The basic hierarchy is defined by three kinds of query. Level 1 – Association concerns $P(Y\mid X=x)$, or “seeing,” and corresponds to standard predictive modeling from observational data. Level 2 – Intervention concerns $P(Y\mid \mathrm{do}(X=x))$, or “doing,” and asks what happens when $X$ is set by intervention. Level 3 – Counterfactuals concerns $P(Y_x\mid X=x', Y=y')$, or “imagining,” and asks what would have happened under a different action given what actually occurred (Fokoué, 22 May 2026).

The central claim of the ladder literature is that Level 1 data cannot, in general, answer Level 2 and Level 3 queries without additional structural assumptions. The distinction between $P(Y\mid \mathrm{do}(X))$0 and $P(Y\mid \mathrm{do}(X))$1 is therefore not a matter of model scale, training set size, or capacity; it is a structural limitation. This is why the claim “prediction is about $P(Y\mid \mathrm{do}(X))$2 while intelligence demands $P(Y\mid \mathrm{do}(X))$3” functions as both a causal thesis and a trust thesis (Fokoué, 22 May 2026).

The strongest formal statement of this view is the Statistical Necessity Theorem for Causal Generalization. Let $P(Y\mid \mathrm{do}(X))$4 be a collection of environments $P(Y\mid \mathrm{do}(X))$5, each with distribution $P(Y\mid \mathrm{do}(X))$6 over $P(Y\mid \mathrm{do}(X))$7, and let $P(Y\mid \mathrm{do}(X))$8 denote risk in environment $P(Y\mid \mathrm{do}(X))$9. Causal OOD Generalization is defined by the existence of $P(Y_x\mid X=x',Y=y')$0 such that, for all $P(Y_x\mid X=x',Y=y')$1,

$P(Y_x\mid X=x',Y=y')$2

where $P(Y_x\mid X=x',Y=y')$3 is the Bayes risk under the causal mechanism from $P(Y_x\mid X=x',Y=y')$4 to $P(Y_x\mid X=x',Y=y')$5. Under an SCM $P(Y_x\mid X=x',Y=y')$6, features are decomposed into causally invariant features $P(Y_x\mid X=x',Y=y')$7 and spurious environment-specific features $P(Y_x\mid X=x',Y=y')$8. The theorem states that predictors using $P(Y_x\mid X=x',Y=y')$9 fail in the worst environment, while the predictor

$45^{\circ}$0

achieves Bayes-optimal risk uniformly across environments, because $45^{\circ}$1 for all $45^{\circ}$2 under the SCM. ERM on a single environment converges to minimizers of $45^{\circ}$3, not $45^{\circ}$4, and therefore exploits spurious correlations when they are predictive in the training environment (Fokoué, 22 May 2026).

This establishes the ladder as more than a descriptive metaphor. It is a claim that robust, aligned, and safe behavior under environment change requires modeling invariant mechanisms rather than optimizing only observational conditionals.

3. Unified causal-statistical machinery

One line of work formalizes the necessary machinery as a family of Causal Statistical Estimators (CSEs). A CSE is a triple $45^{\circ}$5, where $45^{\circ}$6 is an identification assumption, $45^{\circ}$7 is an adjustment procedure, and $45^{\circ}$8 is an estimator of a causal estimand such as the Average Treatment Effect

$45^{\circ}$9

Under this template, Pearl’s do-calculus, the Potential Outcomes framework, Double Machine Learning, and Invariant Risk Minimization are not competing worldviews but distinct instantiations of the same problem: identifying and estimating interventional quantities from observational data under different assumptions (Fokoué, 22 May 2026).

The most classical identification device is the backdoor formula

$S \approx C$0

used when a valid confounder set $S \approx C$1 blocks all backdoor paths from $S \approx C$2 to $S \approx C$3. The Potential Outcomes formulation expresses the same causal target with $S \approx C$4 and $S \approx C$5 under assumptions such as ignorability and overlap. Survey work emphasizes the equivalence between SCM and PO perspectives through relations such as $S \approx C$6 in an SCM with $S \approx C$7 (Fokoué, 22 May 2026, Ganguly et al., 2023).

For high-dimensional settings, Double Machine Learning (DML) is cast in a partially linear causal model,

$S \approx C$8

with orthogonality conditions $S \approx C$9 and $X$0. DML forms residuals and yields an estimator $X$1 that is root-$X$2 consistent if

$X$3

with asymptotic normality at the semiparametric efficiency bound. The significance for the causal ladder is that flexible ML models can be embedded within principled causal estimation rather than used only for prediction (Fokoué, 22 May 2026).

Invariant Risk Minimization (IRM) provides a representation-learning analogue. It seeks a representation $X$4 and classifier $X$5 such that the same classifier is optimal across environments: $X$6 Under the SCM assumptions used in the necessity theorem and sufficient environment diversity, IRM recovers a representation sufficient for the causal features $X$7 (Fokoué, 22 May 2026).

A complementary position paper generalizes these ideas by treating fairness, robustness, privacy, explainability, and accuracy as invariance requirements under different admissible changes to the data-generating process. Observational accuracy is

$X$8

whereas interventional accuracy is

$X$9

Trade-offs arise because trust objectives demand stability under different interventions: interventions on sensitive attributes for fairness, neighboring datasets for privacy, environments for robustness, and relevant versus irrelevant perturbations for explainability. Causality supplies selective invariance through path-specific and intervention-specific constraints, rather than blanket invariance that destroys utility (Binkyte et al., 4 May 2026).

4. Failure modes, benchmarks, and process verification

The causal ladder is also a diagnosis of contemporary AI failures. One treatment identifies three high-profile failures as manifestations of causal blindness. First, hallucination in LLMs arises because next-token training minimizes

$P(Y\mid X=x)$0

which is a pure $P(Y\mid X=x)$1 objective. Second, reward hacking in RLHF occurs when learned reward models exploit surface correlates of human ratings. Third, ERM-based models collapse under distribution shift because they rely on environment-specific features. The proposed remedies are correspondingly causal: backdoor-adjusted causal reward modeling for hallucination, instrumental variables or DML for reward hacking, and IRM for OOD failure (Fokoué, 22 May 2026).

The same paper gives concrete illustrations. In a synthetic reward-hacking experiment, a naive OLS reward model assigns a spuriously positive weight $P(Y\mid X=x)$2 to length $P(Y\mid X=x)$3, whereas the DML-based causal reward model recovers $P(Y\mid X=x)$4. In ColoredMNIST-style experiments, as spurious correlation strength $P(Y\mid X=x)$5 grows, ERM’s training accuracy rises toward 99% while OOD accuracy falls to $P(Y\mid X=x)$6, while a predictor using only true causal features remains stable (Fokoué, 22 May 2026).

Two diagnostic benchmarks operationalize these issues across Pearl’s ladder. T3 (Testing Trustworthy Thinking) contains 454 expert-curated vignettes across 10 domains and decomposes performance into Utility, Safety, and Wise Refusal. Utility is $P(Y\mid X=x)$7, Safety is $P(Y\mid X=x)$8, Wise Refusal Rate is $P(Y\mid X=x)$9, and False Confidence Rate is $P(Y\mid \mathrm{do}(X=x))$0. T3 reports an L1 Skepticism Trap in which safety-tuned models reject valid causal claims: GPT-4-Turbo scores Utility 100% and Safety 100%, whereas Claude Haiku 3.5 scores Utility 40% and Safety 96%. At L2, performance becomes a question of flip dynamics under pressure: GPT-4-Turbo begins at 98.0% accuracy but falls to 61.5% after self-doubt prompting, while Claude Sonnet 4.5 falls from 96.7% to 27.3%. At L3, T3 reports a Scaling Paradox: base GPT-4-Turbo achieves L3 Safety $P(Y\mid \mathrm{do}(X=x))$1, whereas base GPT-5.2 falls to $P(Y\mid \mathrm{do}(X=x))$2, largely because GPT-5.2 defaults to CONDITIONAL on 92% of many determinate items (Chang, 13 Jan 2026).

T3 also evaluates Recursive Causal Audit (RCA), a process wrapper that requires a direct answer, structured variables and causal sketch, explicit missing-information policy, and a judge check for schema compliance, internal consistency, trace-output consistency, and hint non-dominance. Under RCA, GPT-5.2 moves into the high-Safety region on L3 and CONDITIONAL overuse drops, suggesting that decisive causal judgment can be restored by structured verification rather than by changing the base model alone (Chang, 13 Jan 2026).

CausalT5K extends the same program at larger scale. It contains 5,147 validated cases across 10 domains, built through a human-machine collaborative pipeline involving 40 domain experts, iterative cross-validation cycles, and composite verification. Its three tiers map directly to Pearl’s ladder: Detection for association, Diagnosis for intervention, and Imagination for counterfactuals. The benchmark explicitly tests rung collapse, sycophantic drift, and Wise Refusals, and introduces the Four-Quadrant Control Landscape defined by Paranoia Rate and Sycophancy Ratio. It also isolates the Detection–Correction Gap: across five large models, detection rates lie between 77–91%, but dissonance rates remain 48–55%, meaning models often identify a causal flaw yet still fail to revise the final answer (Geng et al., 9 Feb 2026).

5. Architectures, internal mechanisms, and governance

Several papers interpret the ladder not only as an evaluation framework but as an architectural program. One proposal, Weight-Calculatism, is explicitly aimed at moving AI from “architects of experience” to “reasoners of principles.” It decomposes cognition into Logical Atoms and two operations, Pointing and Comparison, and makes decision-making explicit through

$P(Y\mid \mathrm{do}(X=x))$3

with

$P(Y\mid \mathrm{do}(X=x))$4

All values reduce to a finite set of Initial Weights, and the reasoning substrate is a graph-based cognitive library coordinated by a Global Workspace. The architecture is designed to make causal and value structure explicit, auditable, and editable. In a fire-escape scenario, “carry scientific notes” receives Weight $P(Y\mid \mathrm{do}(X=x))$5 while “carry canned food” receives Weight $P(Y\mid \mathrm{do}(X=x))$6; in an alien ecosystem scenario, overall similarity to known biology is computed as 0.106, leading the system to choose “remote monitoring” (Keyi, 27 Nov 2025).

This proposal is strongest, by its own characterization, on interpretability, causal abstraction at a qualitative level, and value grounding. It is weaker on formal causal inference in Pearl’s sense, explicit safety guarantees, and large-scale empirical validation (Keyi, 27 Nov 2025). That limitation is itself consistent with the ladder thesis: explicit mechanisms and value traceability are necessary components of trust, but they do not replace formal intervention and counterfactual machinery.

A more directly formal line of work uses causal influence diagrams (CIDs) to model AGI safety frameworks. CIDs represent chance nodes, decision nodes, utility nodes, causal arrows, and information links, making incentives structurally visible. In standard RL with a modifiable reward function, the path $P(Y\mid \mathrm{do}(X=x))$7 exposes a wireheading incentive. In current-RF optimization, the objective is anchored to an earlier reward parameter to break that incentive. In reward modeling, human preferences $P(Y\mid \mathrm{do}(X=x))$8 influence rewards through feedback data $P(Y\mid \mathrm{do}(X=x))$9 and the reward model $X$0, exposing the possibility that the agent may manipulate the feedback channel. In CIRL, by contrast, $X$1 directly parameterizes reward, and the agent infers it from human actions. Counterfactual oracles, debate, Supervised IDA, and CAIS are all expressed in the same language, allowing direct comparison of how human preferences, evaluation procedures, and agent actions are causally coupled (Everitt et al., 2019).

At the governance level, the AI-$X$2 program combines the three causal layers with five levels of trustworthy AGI: perception, reasoning, decision-making, autonomy, and collaboration trustworthiness. Perception trustworthiness relies mainly on Approximate Alignment; reasoning and decision-making require strong Intervenable capabilities; autonomy and collaboration increasingly depend on the Reflectable Layer, including world models, counterfactual reasoning, and value reflection. The same framework distinguishes endogenous trustworthiness, built into the architecture and training process, from exogenous trustworthiness, implemented through audits, tests, oversight, and governance mechanisms (Yang et al., 2024).

6. Controversies, limitations, and open problems

The ladder literature does not treat causality as a universal solution. Several tensions are described as structurally unavoidable. Privacy–utility trade-offs remain fundamental in many settings, and fairness is explicitly value-laden because different societies may disagree about which causal paths from a protected attribute are acceptable. Causality can make such conflicts explicit and support selective invariance, but it cannot eliminate normative disagreement (Binkyte et al., 4 May 2026).

A second limitation concerns identifiability and model specification. Explicit causal integration requires reasonably accurate causal graphs or mechanisms, yet causal discovery is constrained by hidden confounders, noise, and limited intervention data. Wrong causal assumptions may induce the wrong invariances and create new harms. Survey work likewise emphasizes that confounders are often latent and high-dimensional, that interpretability metrics remain proxy-based, and that most methods are still concentrated on narrow settings rather than unified models that are simultaneously robust, fair, interpretable, and privacy-preserving (Ganguly et al., 2023, Liu et al., 2023).

Foundation models create additional difficulties. One position paper notes concept superposition in FMs, which makes precise internal interventions difficult and often pushes causal control toward behavior-level constraints rather than fine-grained internal mechanisms. The same paper distinguishes explicit causal integration through SCMs and path constraints from implicit causal integration through multi-environment training, disentangled representations, counterfactual data augmentation, and robustness objectives. It argues that model scale alone does not remove the need for causal structure; it only changes where those assumptions are encoded (Binkyte et al., 4 May 2026).

Benchmark results add a further controversy: scaling and safety tuning do not monotonically improve trustworthy causal reasoning. T3 reports that larger or more safety-tuned models may become overly skeptical at L1, unstable under pressure at L2, or paralyzed by over-hedging at L3. CausalT5K likewise shows that static audit policies universally fail, because the same critique protocol can help a Volatile model and harm a Sycophantic one, and the same base model can shift quadrants depending on the judge (Chang, 13 Jan 2026, Geng et al., 9 Feb 2026).

Taken together, these results support a precise interpretation of the Causal Ladder of Trustworthy AGI. It is not only a hierarchy of query types. It is a hierarchy of trust conditions: from observational competence, to interventional robustness and controllability, to counterfactual reflection, calibrated refusal, and governance of long-horizon action. The recurring conclusion is that the ladder of causation is also a ladder of trustworthiness: ascending from Level 1 pattern fitting to Level 2 intervention and Level 3 counterfactual understanding is what separates high-capability but brittle systems from systems that can be robust, aligned, and safe under changing conditions (Fokoué, 22 May 2026).