Predictive-Causal Gap Explained

• Predictive-Causal Gap is a concept contrasting statistical prediction with intervention-valid causal inference, highlighting issues in treatment effect estimation.
• It reveals that models excelling in observed data prediction can falter under interventions, distribution shifts, and counterfactual analysis.
• Empirical studies across domains show that methods focusing solely on prediction often misalign with causal goals, necessitating techniques like importance weighting and causal discovery.

Searching arXiv for the cited papers to ground the article in the current records. The predictive-causal gap denotes a family of mismatches between success at prediction and validity for causal, interventional, or transportable inference. In some literatures, the term refers to the difference between minimizing factual prediction error and recovering treatment effects such as the ATE or CATE; in others, it denotes the failure of correlation-based predictors under intervention, operational change, or distribution shift; in yet others, it describes a mismatch between next-token prediction and genuine causal reasoning, or between predictive representations and representations of the system of interest (Doutreligne et al., 2023, Gonzalez et al., 2024, Chi et al., 26 Jun 2025, Liu, 6 May 2026). A central complication is that the gap is not uniformly construed as a fundamental divide. One influential reframing argues that it is often overstated, because causal inference can be viewed as prediction under distribution shift and selective label observation rather than as a wholly separate inferential enterprise (Fernández-Loría, 6 Apr 2025).

1. Conceptual definitions and competing framings

In causal inference proper, the gap is classically formulated as the difference between predicting observed outcomes and estimating unobserved counterfactual outcomes. Standard prediction minimizes outcome error on what is observed, whereas causal inference targets quantities that depend on both potential outcomes for each unit, even though only one is observed (Doutreligne et al., 2023). This is the sense in which a model can have low factual error yet poor treatment-effect estimation.

Applied work often gives the term a more operational meaning. In predictive maintenance, the gap is the difference between correlation-based prediction, which learns statistical patterns that co-occur with failures, and causal modeling, which encodes the physical mechanisms that actually generate failures, so that predictions remain valid under operational change and support actionable interventions (Taduri et al., 30 Nov 2025). In dynamical systems, the gap is the tension between a data-driven predictive culture emphasizing forecasting accuracy and a theory-/mechanism-driven explanatory culture emphasizing interpretable causal structure (Gonzalez et al., 2024). In stock prediction, it is the mismatch between source-domain correlations and invariant causal relationships that remain stable across domains and time (Xu et al., 26 Mar 2025).

A partially contrary framing appears in the claim that “causal inference isn’t special.” Under that view, the predictive-causal gap is overstated because causal inference has the same source-to-target generalization structure as predictive modeling under distribution shift; the distinctive feature is that labels are selectively observed through treatment assignment (Fernández-Loría, 6 Apr 2025). This perspective does not deny that counterfactual targets are conceptually distinct. Rather, it relocates the distinction from scientific logic to target definition and bias structure.

In machine learning for language and vision, the term is again specialized. For transformer-based LLMs, the gap is the difference between predicting likely continuations and reasoning about causal relations in fresh, unseen, counterfactual, or misleading settings (Chi et al., 26 Jun 2025). For Video-LLMs, it marks the difference between semantic perception and predictive world modeling, especially when models must explain anomalies via physical mechanisms rather than merely recognize that something “looks wrong” (Liu et al., 23 Feb 2026).

2. Formal statistical structure

A common formalization begins with the potential-outcomes setup. Each unit has two potential outcomes, $Y^1$ and $Y^0$, with treatment assignment $T \in \{0,1\}$; the individual causal effect is $Y^1-Y^0$. What is observed are conditional distributions such as $Y^1 \mid T=1$ and $Y^0 \mid T=0$, whereas causal targets require the marginal potential-outcome distributions $Y^1$ and $Y^0$ (Fernández-Loría, 6 Apr 2025). In randomized settings, the idealized approximation is

$$Y^1 \mid T=1 \approx Y^1, \qquad Y^0 \mid T=0 \approx Y^0,$$

which explains why difference-in-means works there; in observational settings, treatment assignment creates biased source samples.

The causal-model-selection literature makes the same gap explicit through risk functionals. With

$$\mu_a(x)=\mathbb E[Y\mid X=x, A=a], \qquad \tau(x)=\mu_1(x)-\mu_0(x),$$

standard predictive selection minimizes

$Y^0$0

whereas the unattainable oracle target is the treatment-effect error

$Y^0$1

Because $Y^0$2-risk is unobserved, proxy criteria are used; the recommended one is

$Y^0$3

with nuisance functions $Y^0$4 and $Y^0$5 (Doutreligne et al., 2023).

A related performative formulation targets the causal effect of predictions themselves. If deployed predictions $Y^0$6 affect eventual outcomes $Y^0$7, the relevant object is not $Y^0$8 but

$Y^0$9

the expected outcome if the prediction is intervened upon while covariates are held fixed (Mendler-Dünner et al., 2022). This makes the predictive-causal gap a problem of identifiability under endogenous prediction.

Time-series work extends the same issue to dynamic settings. The potential system formalizes counterfactual time-series branches and defines dynamic causal effects such as

$T \in \{0,1\}$0

Observed predictive contrasts become causal only under sequential assignment conditions such as branch-randomization or branch-sequential unconfoundedness (Carlson et al., 20 Mar 2026).

3. Mechanisms that generate the gap

Several mechanisms recur across domains. The first is selective label observation. In causal inference, one potential outcome is missing for each unit, and the missingness is induced by treatment assignment rather than by arbitrary censoring. The problem is therefore not merely extrapolation, but extrapolation from biased source samples (Fernández-Loría, 6 Apr 2025).

The second is spurious association under intervention or shift. In dynamical systems, classical feature selection may retain confounders, mediators, or merely predictive lags that are useful under the historical distribution but unstable under intervention (Gonzalez et al., 2024). In stock prediction, low signal-to-noise ratio and market nonstationarity encourage models to exploit unstable correlations rather than invariant causal factors (Xu et al., 26 Mar 2025). In causal-effect estimation from ML/DL, non-causal correlations may improve in-distribution fit while harming both explanation and out-of-distribution generalization (Pichler et al., 2023).

A third mechanism is limited overlap. When treated and untreated units occupy different regions of covariate space, predictive criteria that score only the observed outcome fail to control the accuracy of both $T \in \{0,1\}$1 and $T \in \{0,1\}$2 simultaneously (Doutreligne et al., 2023). Bayesian work on posterior predictive treatment assignment reframes the issue by asking which units have empirical support for the opposite treatment, thereby turning a predictive treatment model into a stochastic causal design stage (Zigler et al., 2017).

A fourth mechanism is prediction-induced concept shift. In performative systems, predictions shape outcomes, so historical $T \in \{0,1\}$3 pairs do not suffice for forecasting the effect of a new model. If the training-time predictor is deterministic, $T \in \{0,1\}$4 is generally not identifiable because $T \in \{0,1\}$5 is perfectly coupled to $T \in \{0,1\}$6 in the observed data (Mendler-Dünner et al., 2022).

A fifth mechanism is decision or representation mismatch after optimization. In fairness analysis, a score $T \in \{0,1\}$7 that satisfies a fairness notion need not retain that property after thresholding to $T \in \{0,1\}$8. The paper formalizes the distortion through the margin complement

$T \in \{0,1\}$9

showing how disparity in the binary predictor decomposes into disparity inherited from the true outcome and disparity introduced by thresholding (Plecko et al., 2024). In predictive representation learning, the mismatch becomes structural: when environment modes are slower or less noisy than system modes, predictive objectives may encode the environment rather than the system of interest (Liu, 6 May 2026).

Finally, in LLMs, the mechanism is the non-equivalence between sequential dependence in text and causal dependence in the world. The autoregressive objective

$Y^1-Y^0$0

optimizes continuation likelihood, not interventional validity, which helps explain why performance can drop sharply on fresh causal benchmarks (Chi et al., 26 Jun 2025).

4. Principal methodological responses

One major response is to reinterpret causal estimation as generalization under structured shift and to transfer familiar predictive tools. Under this view, inverse probability weighting (IPW) is analogous to importance weighting, while covariate adjustment and matching rebalance biased source data so that it better represents the target counterfactual population (Fernández-Loría, 6 Apr 2025). This does not eliminate assumptions; it makes them explicit through unconfoundedness and overlap.

A second response is causally constrained feature selection. If the causal graph is known, Pearl’s backdoor criterion can be used to restrict ML/DL models to adjustment sets that block confounding. Under such causal constraints, neural networks provided near unbiased effect estimates in the reported simulations, and causally constrained models generalized better when collinearity structures changed (Pichler et al., 2023). A related approach, MODE, learns $Y^1-Y^0$1, trains a predictive model only on those parents, and then computes feature-level causal effects from the model’s conditional probabilities, yielding what the paper calls a causally interpretable predictive model (Li et al., 2023).

A third family of methods combines causal discovery, expert refinement, and downstream prediction. In the data-center study, the workflow is: causal discovery on time series with Tigramite and a PC-family method, domain-expert refinement of the discovered graph, and predictive ML on feature sets such as Causal-lags and Causal-all (Gonzalez et al., 2024). This seeks predictors that are not merely accurate, but stable under intervention.

A fourth response addresses design and overlap directly. Posterior predictive treatment assignment uses posterior-predictive draws of treatment assignment to define stochastic inclusion in an unconfounded subset, thereby marginalizing over uncertainty in overlap assessment and sample selection (Zigler et al., 2017). For performative settings, “predicting from predictions” treats the logged prediction $Y^1-Y^0$2 as an input feature and learns on triples $Y^1-Y^0$3, but only under explicit identifiability conditions such as randomized predictions, overparameterization of the deployed predictor, or discrete prediction outputs (Mendler-Dünner et al., 2022).

A fifth response is to integrate predictive and causal nuisance modeling rather than treat them as separate tasks. In spatio-temporal biomedical data, an HMM for latent health states and an MTGCN for temporal trajectories are embedded inside a doubly robust estimator and a penalized empirical likelihood framework (Lee et al., 11 Jul 2025). This architecture is presented as a predictive-causal framework rather than a purely predictive or purely static causal estimator.

5. Empirical manifestations across domains

The term is operationalized differently across application areas, but the empirical pattern is recurrent: predictive adequacy under one criterion or domain does not guarantee causal adequacy, transportability, or operational usefulness.

Domain	Operationalization of the gap	Reported finding
Predictive maintenance	Correlation-based AI vs causal model	L5 achieved 70.2% savings, 92.1% precision, and 97% reduction in false alarms relative to L3 while matching 87.9% recall (Taduri et al., 30 Nov 2025)
Dynamical systems	Traditional feature selection vs causal features	Causal-discovery-based feature selection outperformed other approaches in more than 70% of simulation runs and degraded less after interventions on Cool_set (Gonzalez et al., 2024)
Stock prediction	Correlation fit vs invariant causal factors	Causal discovery models improved OOS performance; with ResNet1D, SR increased from 0.86 to 2.53 in one reported setting (Xu et al., 26 Mar 2025)
Predictive representation learning	Predictive encoder vs system representation	Mean causal fidelity was 0.49; only 2.5% of configurations exceeded 0.70; at $Y^1-Y^0$4, fidelity was approximately $Y^1-Y^0$5 with 92.1% lower prediction error than the causal representation (Liu, 6 May 2026)
LLM causal reasoning	Next-token prediction vs fresh causal reasoning	Models showed a significant drop on CausalProbe-2024; on CausalProbe-H, Claude 3 opus was under 70% exact match and LLaMA 2 7B chat was around 56.5% (Chi et al., 26 Jun 2025)
Video-LLMs	Semantic perception vs predictive world modeling	Performance dropped by more than 20% on causal tasks relative to ontological tasks (Liu et al., 23 Feb 2026)

These results do not all support the same theoretical conclusion. Some papers argue that causal structure improves robustness precisely because it encodes invariant mechanisms; others show that predictive objectives alone can systematically prefer the wrong variables. The shared empirical theme is narrower: standard predictive criteria are often misaligned with interventional validity, stability under shift, or real operational cost (Taduri et al., 30 Nov 2025, Liu, 6 May 2026).

6. Interpretation, controversy, and limits

The literature contains a genuine disagreement over whether the predictive-causal gap is fundamental. One position argues that the gap is overstated: causal inference is prediction with selective labels, distribution shift, and explicit assumptions, not a separate scientific logic (Fernández-Loría, 6 Apr 2025). Another line of work argues that the gap is substantive and sometimes structural: standard predictive selection, unconstrained feature learning, or autoregressive next-token objectives can systematically target the wrong object (Doutreligne et al., 2023, Chi et al., 26 Jun 2025, Liu, 6 May 2026).

The disagreement is partly terminological. In some settings, “predictive” means ordinary supervised learning on observed labels; in others, it means optimizing a deployment metric, minimizing future-state prediction error, or modeling semantic continuations. Likewise, “causal” may refer to treatment effects, mechanistic graphs, intervention robustness, physical law reasoning, or system-environment separation. This suggests that the phrase does not name one theorem or one failure mode, but a family of mismatches between observational fit and counterfactual or invariant interpretation.

The cited work also emphasizes strong caveats. Many methods assume that causal graphs or direct causes are known, or at least that domain experts can refine discovered structures (Li et al., 2023, Gonzalez et al., 2024). Several empirical studies are based on a single dataset, simulation environments, or synthetic nonlinear systems (Taduri et al., 30 Nov 2025, Gonzalez et al., 2024, Liu, 6 May 2026). Even where predictive and causal goals can be partially aligned, hyperparameter settings optimized for prediction need not be optimal for effect estimation, and regularization that stabilizes prediction can distort causal effects (Pichler et al., 2023).

A balanced interpretation is therefore limited but clear. The literature does not support the universal claim that prediction and causation are identical, nor the universal claim that they are categorically separate. It supports a more specific proposition: whenever the target is counterfactual, intervention-robust, mechanism-preserving, or transportable across altered environments, predictive success on observed data is insufficient unless the relevant bias structure, overlap conditions, causal variables, or system boundary are made explicit (Fernández-Loría, 6 Apr 2025, Mendler-Dünner et al., 2022, Carlson et al., 20 Mar 2026).