Runtime Confounding in Causal Models

• Runtime confounding is the presence of confounders in historical data that are missing, inadmissible, or altered at prediction time, leading to bias in causal estimates.
• Methodologies like doubly robust procedures and conformal prediction adjust for missing confounders, ensuring valid counterfactual predictions under deployment constraints.
• In recommender systems, runtime confounding causes biased scoring by influencing interaction likelihood, necessitating strategies like mixture-of-experts for unbiased recommendations.

Searching arXiv for recent and directly relevant papers on runtime confounding and adjacent formulations. Runtime confounding is used in several related ways in recent causal-inference and recommender-systems literature. In one formulation, all relevant confounders are captured in historical data, but some cannot be used at prediction time, so treatment assignment is unconfounded given $(V,Z)$ but not given the runtime-available variables $V$ alone (Coston et al., 2020). In another, a target population lacks confounders $\mathbf U$ that were observed in the source population, so naively discarding $\mathbf U$ can lead to severe miscoverage in counterfactual prediction intervals (Barnatchez et al., 4 Apr 2026). In recommender systems, runtime confounding also denotes inference-time bias caused by a confounding feature that directly affects whether an interaction happens, or by a deployed policy that begins to depend on a feature that downstream training still omits (He et al., 2022). Taken together, these formulations suggest a deployment-time mismatch between observational training and causal scoring or prediction (Merkov et al., 14 Aug 2025).

1. Core meanings and problem variants

The term has not been restricted to a single formalism. In decision-support settings, runtime confounding refers to the case where historical data contain all relevant factors, but some such factors are unavailable, impermissible, or undesirable in the final prediction model. The paper "Counterfactual Predictions under Runtime Confounding" states this through training ignorability,

$Y^a \perp A \mid V, Z,$

together with runtime confounding,

$Y^a \not\perp A \mid V,$

equivalently,

$$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$

(Coston et al., 2020).

In conformal counterfactual prediction, the term denotes a source–target setting in which $\mathbf V$ is always observed, $\mathbf U$ is observed only in the source population, and prediction intervals in the target population must depend only on $\mathbf V$. The target interval $V$0 is required to satisfy

$V$1

even though $V$2 is unavailable in the target population (Barnatchez et al., 4 Apr 2026).

In recommender systems, the same phrase has a more operational meaning. A confounding feature $V$3 is an item feature that has a direct effect on $V$4 independent of true preference; video length is the motivating example in short-video recommendation, because shorter videos are easier to finish even if the user does not like them. If such a feature is used observationally at scoring time, recommendations become biased toward “easy-to-interact” items (He et al., 2022). A related deployment-centered account argues that recommender systems can create confounding at runtime when a previously ignored observed feature begins to influence the action policy while downstream estimation still behaves as if that feature were ignorable (Merkov et al., 14 Aug 2025).

Setting	Runtime-confounding mechanism	Representative paper
Counterfactual prediction	Historical confounders unavailable or impermissible at prediction time	(Coston et al., 2020)
Counterfactual conformal prediction	Source-only confounders missing in the target population	(Barnatchez et al., 4 Apr 2026)
Causal recommendation	Inference-time scores contaminated by a confounding feature or its induced spurious correlation	(He et al., 2022)
Deployed recommender pipelines	Action policy starts using a feature that later training omits	(Merkov et al., 14 Aug 2025)

2. Formal causal structure and identification

The counterfactual-prediction formulation is centered on a binary intervention $V$5, runtime-available predictors $V$6, runtime-hidden confounders $V$7, observed outcome $V$8, and potential outcomes $V$9. The target is the conditional potential-outcome mean

$\mathbf U$0

With consistency and positivity, training ignorability implies

$\mathbf U$1

and therefore

$\mathbf U$2

The same paper contrasts this target with treatment-conditional regression, which estimates $\mathbf U$3 and incurs pointwise confounding bias

$\mathbf U$4

under runtime confounding (Coston et al., 2020).

The source–target formulation makes the missing-runtime-confounder structure explicit through the observed unit

$\mathbf U$5

with $\mathbf U$6 observed only when $\mathbf U$7. Its core assumptions are positivity,

$\mathbf U$8

consistency,

$\mathbf U$9

unconfoundedness in the source population,

$\mathbf U$0

source exchangeability,

$\mathbf U$1

and source positivity,

$\mathbf U$2

A key point in that formulation is that simply discarding $\mathbf U$3 would require the stronger condition

$\mathbf U$4

which is not assumed (Barnatchez et al., 4 Apr 2026).

The recommender formulation uses a different graph. Item features are split into $\mathbf U$5, a confounding feature, and $\mathbf U$6, other content features, with user features $\mathbf U$7 and interaction label $\mathbf U$8. The backdoor path

$\mathbf U$9

implies that learning either $Y^a \perp A \mid V, Z,$0 or $Y^a \perp A \mid V, Z,$1 from observational data produces biased scoring at inference time. The causal estimand becomes

$Y^a \perp A \mid V, Z,$2

and the runtime correction is

$Y^a \perp A \mid V, Z,$3

This shifts the deployed scoring rule from a factual predictor to an interventional predictor (He et al., 2022).

3. Prediction, debiasing, and uncertainty quantification

For counterfactual prediction with restricted runtime covariates, the central methodological response is a two-stage doubly robust procedure. The key pseudo-outcome is

$Y^a \perp A \mid V, Z,$4

which is then regressed on $Y^a \perp A \mid V, Z,$5 to estimate $Y^a \perp A \mid V, Z,$6. The pointwise error bound is product-form: $Y^a \perp A \mid V, Z,$7 For evaluation, the same paper proposes a doubly robust estimator of the mean squared error of a learned prediction function $Y^a \perp A \mid V, Z,$8,

$Y^a \perp A \mid V, Z,$9

and reports in a child-welfare study that treatment-conditional regression had MSE about $Y^a \not\perp A \mid V,$0, the plug-in method about $Y^a \not\perp A \mid V,$1, and the doubly robust method about $Y^a \not\perp A \mid V,$2 (Coston et al., 2020).

In runtime-confounded conformal prediction, the objective is not point prediction but valid target-population intervals. The paper "Debiased Machine Learning for Conformal Prediction of Counterfactual Outcomes Under Runtime Confounding" identifies the calibration threshold $Y^a \not\perp A \mid V,$3 through

$Y^a \not\perp A \mid V,$4

and uses both an identification formula with

$Y^a \not\perp A \mid V,$5

and a weighted formula with

$Y^a \not\perp A \mid V,$6

Its main estimator is built from the efficient influence curve

$Y^a \not\perp A \mid V,$7

and solves the empirical estimating equation

$Y^a \not\perp A \mid V,$8

Theorem 3 gives the coverage expansion

$Y^a \not\perp A \mid V,$9

where

$$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$0

The paper states that the naive method miscovers badly at all sample sizes, that miscoverage worsens as runtime confounding becomes more severe, and that the proposed DML method approaches the nominal $$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$1 coverage as $$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$2 grows; the weighted method also achieves near-nominal $$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$3 coverage, and the proposed DML intervals are often as narrow or narrower than the weighted intervals (Barnatchez et al., 4 Apr 2026).

A notable feature of both lines of work is that runtime confounding is treated as a deployment constraint rather than a failure of historical identifiability. The historical data can be rich enough to identify causal structure, while the deployed prediction rule is intentionally restricted.

4. Recommender systems and inference-time causal correction

The recommender literature gives runtime confounding a particularly operational interpretation. In "Addressing Confounding Feature Issue for Causal Recommendation," the confounding feature $$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$4 directly affects the interaction label $$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$5, so finished interactions do not necessarily indicate preference. The proposed framework, Deconfounding Causal Recommendation (DCR), trains a model to estimate $$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$6 but performs recommendation using the interventional quantity $$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$7. Direct computation of

$$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$8

requires one model evaluation for every possible confounding value $$A \not\perp Z \mid V \quad \text{and} \quad Y^a \not\perp Z \mid V$$9, so if $\mathbf V$0, inference becomes $\mathbf V$1 times more expensive. To reduce this cost, DCR introduces a mixture-of-experts architecture with a shared backbone

$\mathbf V$2

and expert heads

$\mathbf V$3

so that

$\mathbf V$4

Empirically, with $\mathbf V$5, DCR-MoE achieved the best recommendation accuracy on both datasets. On Kwai it reached Recall@10 $\mathbf V$6, MAP@10 $\mathbf V$7, and NDCG@10 $\mathbf V$8; on Wechat it reached Recall@3 $\mathbf V$9, MAP@3 $\mathbf U$0, and NDCG@3 $\mathbf U$1. Reported inference times were $\mathbf U$2s and $\mathbf U$3s for DCR-NFM, $\mathbf U$4s and $\mathbf U$5s for DCR-MoE, and $\mathbf U$6s and $\mathbf U$7s for the approximation-based DCR-NFM-A on Kwai and Wechat respectively (He et al., 2022).

A second recommender formulation emphasizes system evolution rather than static item features. The paper "Confounding is a Pervasive Problem in Real World Recommender Systems" uses variables $\mathbf U$8 for click outcome, $\mathbf U$9 for recommended action, $\mathbf V$0 for the feature currently used for personalization, and $\mathbf V$1 for an additional feature that may later be introduced. The causal target is

$\mathbf V$2

If $\mathbf V$3 does not affect the action, this simplifies to

$\mathbf V$4

But once the policy starts using $\mathbf V$5, and later training still omits it, $\mathbf V$6 becomes a confounder. The paper describes this as a temporal sequence: a randomized policy on Day 0, a policy using $\mathbf V$7 on Day 1, a policy using both $\mathbf V$8 and $\mathbf V$9 on Day 2, and then a reversion on Day 3 to training with only $V$00 on logs generated by a policy that depended on $V$01. It identifies feature engineering, A/B testing on shared logs, and modularization as mechanisms that can create runtime confounding in deployed systems (Merkov et al., 14 Aug 2025).

A common misconception is that recommender systems are safe from confounding because all inputs are “observed.” The recommender papers explicitly reject that view: an observed feature can become a confounder when it influences both action selection and outcome, and deployment changes can alter the causal graph without changing the training code (Merkov et al., 14 Aug 2025).

5. Relation to broader confounding methodologies

Runtime confounding sits within a broader literature on causal inference under confounding, but it is not reducible to any one classical problem. The instrumental-variables literature addresses a different obstacle: confounding by an unmeasured $V$02 when ordinary regression fails. In the simple structural equation

$V$03

ordinary least squares gives

$V$04

which is consistent only if $V$05. An instrumental variable $V$06 satisfying relevance, independence, and exclusion restriction yields

$V$07

and the appendix generalizes this to two-stage least squares (Marzban et al., 23 Jun 2025). This does not solve runtime confounding directly, but it addresses the adjacent case where confounders are not observed at all.

Safe decision-making under hidden confounding leads to yet another response. "Confounding-Robust Policy Improvement" assumes that policy value and regret may not be point-identifiable under unobserved confounding and therefore optimizes worst-case regret relative to a baseline policy $V$08. The method uses a marginal sensitivity model with odds-ratio bound

$V$09

and learns a policy by minimizing worst-case empirical regret over an uncertainty set of inverse propensity weights. The paper emphasizes safety relative to baseline rather than point identification of a fully personalized policy (Kallus et al., 2018).

Observed-confounding conformal prediction provides a finite-sample back-door analogue. "Conformal e-prediction in the presence of confounding" studies the graph

$V$10

and targets the interventional law

$V$11

It constructs a smoothed estimator $V$12 from empirical counts and proves

$V$13

which yields e-values and prediction regions for $V$14 under $V$15 (Vovk et al., 11 Mar 2026). This is not a runtime-confounding paper in the narrow sense, but it clarifies how prediction targets change once one moves from $V$16 to $V$17.

Causal discovery under confounding addresses a distinct but related question. LiNGAM-MMI replaces the standard LiNGAM requirement that one order achieve independent residuals with the objective

$V$18

where

$V$19

The method interprets larger residual dependence as stronger confounding and searches for the globally optimal order by a shortest-path formulation (Suzuki et al., 2024). This is adjacent to runtime confounding insofar as it treats confounding-aware causal structure as a prerequisite for later deployment.

6. Limitations, misconceptions, and practical significance

Several misconceptions recur across the literature. Runtime confounding is not the same as standard confounding in a single population, because the defining issue is often that causal adjustment is possible in training data but not in the deployed predictor (Coston et al., 2020). It is also not the same as target shift or a generic missing-covariate problem; the source–target conformal paper states that the key issue is that some confounders are available in training but not at runtime in the target site, and it attributes the problem to two simultaneous shifts: covariate shift across treatment levels within the source population and covariate shift between source and target populations in $V$20 (Barnatchez et al., 4 Apr 2026). Nor is runtime confounding restricted to unmeasured causes: in recommender systems, ignored observed features can become confounders when policy changes make them influence actions (Merkov et al., 14 Aug 2025).

The limitations are equally consistent. Runtime-confounding corrections often impose computational or modeling costs. In DCR, exact backdoor adjustment requires summing over all confounder values, which creates the runtime bottleneck that motivates the mixture-of-experts architecture; the approximation-based alternative is fastest but sacrifices accuracy (He et al., 2022). In conformal DML, a full conformal version without data splitting is possible but requires stronger Donsker-type conditions and is computationally heavier (Barnatchez et al., 4 Apr 2026). In IV-based inference, independence and exclusion restriction are not directly testable when confounders are unmeasured, so identification remains fundamentally a matter of theory and substantive knowledge (Marzban et al., 23 Jun 2025). In sensitivity-based policy learning, larger $V$21 gives stronger protection against hidden confounding but can be conservative if the real confounding is smaller (Kallus et al., 2018).

The practical significance is that runtime confounding converts an apparently ordinary prediction problem into a causal transport-and-deployment problem. Naive treatment-conditional regression can target the wrong quantity even when fit perfectly (Coston et al., 2020). Naively dropping source-only confounders can break interval validity (Barnatchez et al., 4 Apr 2026). Naively using observational recommender scores can over-recommend short videos or otherwise exploit “easy-to-interact” confounding values (He et al., 2022). Naively pooling logs across feature-mismatched recommender variants can entrench bias in A/B testing and modularized systems (Merkov et al., 14 Aug 2025). These results suggest that runtime confounding is best understood not as a narrow technical anomaly, but as a recurring mismatch between the variables that support causal identification during learning and the variables that remain available, admissible, or consistently used when decisions are made.