---
title: Confounded Causal Imitation Learning (C2L)
url: https://www.emergentmind.com/topics/confounded-causal-imitation-learning-c2l
type: topic
---

# Confounded Causal Imitation Learning (C2L)

Confounded Causal Imitation Learning (C2L) denotes imitation learning under confounding: expert demonstrations contain observational dependencies between observations and actions that need not remain valid once the learner is deployed. In this setting, behavioral cloning can attain low supervised loss while producing policies that fail under the learner-induced state distribution, because it estimates observational conditionals such as $P(a\mid o)$ rather than the interventional or otherwise deconfounded targets required for robust control. Across the literature, the same core problem appears as causal confusion or causal misidentification, imitation with unobserved confounders, sensory mismatch, temporally correlated noise, measurement error, and context-specific causal structure [1905.11979].

## 1. Causal confusion and the observational–interventional gap

In the canonical formulation, imitation learning is posed on a (partially observed) Markov decision process with states $s$, observations $o$, actions $a$, expert policy $\pi_E(a\mid o)$, learner policy $\pi_\theta(a\mid o)$, and dynamics $P(s' \mid s,a)$. Demonstrations $D_{\pi_E}$ are tuples $(o_t,a_t)$ drawn from trajectories generated by $\pi_E$ under the induced state distribution $d_{\pi_E}$. Behavioral cloning trains a discriminative mapping $f_\theta$ by minimizing
$$
L(\theta)=\mathbb{E}_{(o,a)\sim D_{\pi_E}}[\ell(f_\theta(o),a)],
$$
typically using cross-entropy for discrete actions or mean squared error for continuous actions. Deployment, however, evaluates the learner under its own induced state distribution $d_{\pi_\theta}$, so the relevant quantity is not training loss but performance under closed-loop interaction, written in the causal-confusion analysis as
$$
J(\pi_\theta)=\mathbb{E}_{s\sim d_{\pi_\theta}}[\ell(\pi_\theta(o(s)),a_E(s))].
$$
The mismatch between $d_{\pi_E}$ and $d_{\pi_\theta}$ is the source of failure [1905.11979].

The distinctive C2L pathology is causal misidentification. A cloned policy may exploit nuisance correlates in $o_t$ that are predictive in demonstrations because they are effects of past state and action, yet are not causes of the expert’s current action. The brake-light example illustrates the mechanism: an effect-of-action feature correlates with braking in demonstrations and can therefore deceive a non-causal learner. A central empirical and conceptual point is that more observational information can worsen performance, because additional features enlarge the space of spurious correlations that collapse under deployment shift [1905.11979].

The causal account is expressed through a structural causal model in which the observation factors are $X^t=[X_1^t,\dots,X_n^t]$, the expert action is $A^t$, and the confounder is $Z^t=[X^{t-1},A^{t-1}]$. Structural equations take the form
$$
Y_i=f_i(Y_{Pa(i;G)},E_i;\theta_G),
$$
with a DAG $G$ encoding which observation components are genuine parents of action. Observationally, behavioral cloning approximates $P(a\mid o)$ on expert data. Causally robust behavior requires the interventional query $P(a\mid do(o))$, computed on the mutilated graph where incoming edges into $o$ are removed. Under the faithfulness result in the original causal-confusion analysis, any learner that matches all interventional queries $p(A\mid do(X))$ must recover the true graph $\hat G=G^\*$ [1905.11979].

## 2. Imitability as a graphical property

A second line of work formulates C2L in a single-step Partially Observable Structural Causal Model (POSCM). Here, $X$ is the action variable, $Y$ is a latent reward, $O$ is the set of observed endogenous variables, $L$ is the set of latent endogenous variables, and $U$ denotes exogenous variables that may induce unobserved confounding. A policy $\pi\in\Pi$ intervenes stochastically on $X$, and the performance target is the causal quantity $\mathbb{E}[Y\mid do(\pi)]$. Because $Y$ is latent, $P(y\mid do(\pi))$ is generally not identifiable non-parametrically from $P(o)$ alone. The relevant question is therefore imitability: whether there exists a policy computable from $P(o)$ that matches the expert’s outcome distribution $P(y)$ for all SCMs compatible with the graph [2208.06267].

The central result is the complete $\pi$-backdoor criterion. For a graph $G$ and policy space $\Pi$, a set $Z\subseteq Pa(\Pi)$ is $\pi$-backdoor admissible if
$$
(Y \perp X \mid Z)_{G_{\underline{X}}}.
$$
Then $P(y)$ is imitable with respect to $\langle G,\Pi\rangle$ if and only if such a $Z$ exists, and the imitating policy is
$$
\pi(x\mid pa(\Pi))=P(x\mid z).
$$
This theorem gives a formal justification for behavior cloning only in the special case where the learner observes the expert’s relevant parents and there is no unobserved confounder entering $X$. When those conditions fail, naive cloning on predictive covariates can be biased even if validation accuracy is high [2208.06267].

When the graphical criterion fails, the same framework introduces practical imitability, or p-imitability, which conditions on both the graph and the realized observational distribution $P(o)$. The fallback mechanism uses surrogates and identifiable policy subspaces. If a surrogate set $S\subseteq O$ satisfies the required separation condition in the augmented graph and $P(s\mid do(\pi))$ is identifiable in a subspace $\Pi'$, then solving
$$
P(s\mid do(\pi))=P(s)
$$
is sufficient to imply
$$
P(y\mid do(\pi))=P(y).
$$
This yields front-door-style constructions even when reward is latent and no direct reward supervision is available [2208.06267].

Context-specific independence extends this picture by replacing DAGs with labeled DAGs whose edges can disappear in specific contexts. In that setting, the imitability decision problem is NP-hard, the necessary criterion requires classic imitability to hold in every context-induced DAG, and—under the structural assumption $Pa(C(\mathcal{L}))\subseteq C(\mathcal{L})$—that per-context condition is also sufficient. The resulting policies are context-gated mixtures of per-context imitators, rather than a single global adjustment rule [2306.00585].

## 3. Sequential C2L and temporal structure

In sequential decision-making, the imitator acts multiple times per episode, and confounding can propagate through both state evolution and observation availability. The sequential formulation models the environment as an SCM over a time-ordered set of variables, with demonstrator policy $\pi_D$, imitator policy $\pi_I=\{\pi_1,\dots,\pi_n\}$, observed covariates $Z_i\subseteq before(X_i)$ for each action $X_i$, and a latent performance variable $Y$ defined over the entire episode. Imitability requires a policy discernible from $P(O)$ such that
$$
P(Y)_M=P(Y\mid do(\pi))_M
$$
for all SCMs compatible with the causal diagram [2208.06276].

The key sequential result is the sequential $\pi$-backdoor criterion. For each action $X_i$, one constructs a manipulated graph $G_i'$ in which future actions are treated as controlled by the imitator. Then, for each $i$, either backdoor blocking must hold at $X_i$ in $(G_i')_{\underline{X_i}}$, or $X_i$ must cease to be an ancestor of $Y$ in $G_i'$. If suitable sets $Z_1,\dots,Z_n$ exist, the policy
$$
\pi_i(X_i\mid Z_i)=P(X_i\mid Z_i),\quad i=1,\dots,n
$$
exactly matches the demonstrator’s performance. The associated algorithm, FindOx, runs in polynomial time, constructs the maximal imitable subset of actions, and proves non-imitability when some actions cannot be included [2208.06276].

A notable consequence is that the standard sequential backdoor criterion of causal-effect identification is not sufficient for imitation. Earlier actions may be locally non-imitable yet made irrelevant globally by downstream policies that shield the final outcome from upstream mistakes. This distinction is specific to imitation: the objective is not merely to identify $P(Y\mid do(\pi))$, but to reproduce the demonstrator’s performance under observation mismatch and latent structure [2208.06276].

A separate unifying framework introduces two types of hidden confounders in sequential IL: expert-observed hidden variables $U_t^o$ and confounding noise $\epsilon_t$ hidden to both expert and learner. With confounding noise horizon $k$, additive action equation $a_t=\pi_E(s_t,U_t^o)+\epsilon_t$, and history $h_t=(s_1,a_1,\dots,s_t)$, the target becomes the history-dependent policy
$$
\pi_h(h_t)=\mathbb{E}[\pi_E(s_t,U_t^o)\mid h_t].
$$
The identification reduces to conditional moment restrictions,
$$
\mathbb{E}[a_t-\pi_h(h_t)\mid h_{t-k}]=0,
$$
and the imitation gap of DML-IL is bounded by
$$
J(\pi_E)-J(\hat\pi_h)\le T^2\big[c\,\mathcal{K}(\Pi,k)\,\epsilon_{CMR}+2\delta\big],
$$
which recovers earlier special cases when either $\delta=0$ or $\epsilon_t=0$ [2502.07656].

## 4. Deconfounding strategies: interventions, instruments, proxies, and causal features

One major strategy for C2L is direct intervention. In the causal-confusion framework, the learner trains a family of graph-parameterized policies by masking candidate parent sets:
$$
\pi_G(X)=f_\phi([X\odot G,G]),
$$
with shared parameters amortized across graphs. Two intervention modes are then used to identify the causal graph. In expert-query mode, the learner executes a mixture policy, scores states by disagreement
$$
D(X)=\mathbb{E}_G[D_{KL}(\pi_G(X)\,\|\,\pi_{mix}(X))],
$$
queries the expert on the most disagreeing states, and fits an energy-based posterior over graphs. In policy-execution mode, the learner executes $\pi_G$, records episodic return $R_G=\sum_t r_t$, and again fits an energy-based posterior $p(G)\propto \exp(\langle w,G\rangle)$, factorized into Bernoulli components under a linear energy model. The outcome is a graph whose parents remain predictive under interventions rather than mere observation [1905.11979].

A more restrictive but highly practical intervention regime masks observed nuisance variables in a disentangled latent representation. Initial-state interventions randomize $S_0$ with an everywhere-nonzero density, thereby removing the edge from latent seed $W$ into the initial state and breaking spurious forks that create false dependencies. The masking rule declares an observation coordinate removable when no state coordinate is dependent with both that observation and any action within a reaction horizon. Under the stated assumptions, the method is conservative: it does not incorrectly mask genuinely causal observations, and intervening on the initial state is provably strictly less conservative for a class of nuisance variables [2307.15980].

Visual imitation learning introduces another intervention style by augmenting demonstrations with human-provided causal cues. CIVIL uses physical markers and language prompts to construct image masks $M_t$, define masked images $X_t'$, and train a causal feature representation $\phi_t=[\phi_{explicit},\phi_{implicit}]$. The encoder is supervised explicitly on marker-derived positional features and implicitly through policy cloning on masked images, with objectives
$$
L_{civil}=L_{policy}+L_{explicit}
$$
in phase 1 and a distillation objective $L_{causal}$ in phase 2. The stated aim is to recover features that causally inform human actions while excluding distractors, and deployment uses the distilled encoder without requiring markers or prompts online [2504.17959].

Recent robotic work embeds similar masking ideas directly into high-capacity transformer policies. One such framework gates encoder features by a binary graph $g$, trains an ACT-style policy under randomized masks, and performs a post-training energy-based search over graphs using episodic reward. Its theoretical claim is that disentanglement is not necessary for learning the structural relationship from observations to action, because the SCM is uniquely solvable with respect to the action variable as long as there is no self-loop $A\to A$ [2507.22380].

A different family of methods identifies deconfounded policies from instruments or proxies. Under temporally correlated action noise, DoubIL uses a simulator to resample next states from past states and a first-stage policy, whereas ResiduIL enforces instrumental-variable moment conditions entirely offline through a minimax objective over residuals. Both treat the past state $Z_t=s_{t-1}$ as an instrument for the confounded relation between current state and action [2202.01312]. Multi-step confounding generalizes this by searching over lagged states $s_k$ as candidate instruments. The key diagnostic is the pseudo-variable residual
$$
R_{s_t,a_t\parallel s_k}:=a_t-l(s_t),
$$
together with the AB Criterion, which requires $R_{s_t,a_t\parallel s_k}\perp s_k$ when $s_k$ is a valid instrument. Under partial non-Gaussianity in linear models, or under a non-degenerate cross-derivative condition in nonlinear models, this criterion is necessary and sufficient for IV validity [2507.17309].

Measurement error leads to a proxy-based proximal formulation. CausIL treats the lagged state $Z_t=S_{t-1}$ as a treatment-inducing proxy and noisy measurement $W_{t-1}$ as an outcome-inducing proxy for the latent state $U_{t-1}$. The deconfounded target is
$$
\pi_{opt}(s)=\arg\max_{a\in\mathcal{A}} p(A_t^{(s)}=a),
$$
which is identified in the discrete case by
$$
P(A_t^{(s)})=P_{A\mid Z',s}(P_{W'\mid Z',s})^{-1}P_{W'}
$$
and in the continuous case by bridge functions estimated through an RKHS adversarial procedure. The target policy is robust to shifts in the measurement channel $P(W_{t-1}\mid U_{t-1})$ and to many dynamics shifts that preserve the marginal distribution of the latent state [2601.22206].

## 5. Empirical domains and reported findings

The earliest empirical evidence for C2L was obtained on modified control benchmarks, Atari, and realistic driving. Augmenting observations with previous actions or previous-action symbols produced policies with near-zero validation loss yet worse deployment reward than policies trained on the original observations, demonstrating causal misidentification directly. In MountainCar and Hopper, policy-execution interventions approached original performance after tens of episodes; on Hopper, Generative Adversarial Imitation Learning required approximately $1500$ episodes to match. Expert-query interventions improved rewards within few queries and outperformed dropout and DAgger under comparable query budgets; DAgger required hundreds of queries on MountainCar and tens of thousands on Hopper to close the gap. The same study also reported that entangled rotations of the MountainCar state reduced effectiveness, with rewards dropping from $-137$ to $-145$ in policy-execution mode and from $-140$ to $-165$ in expert-query mode [1905.11979].

Graphical single-step C2L was evaluated on highway driving and a front-door MNIST construction. In highD-based driving, causal imitation matched the expert outcome distribution with $L1$ distance $0.0018$, whereas behavior cloning using $\{Z,W\}$ failed with $L1=0.2937$. In the MNIST front-door setting, causal imitation achieved $L1=0.0634$ and behavior cloning $L1=0.1900$. In synthetic binary front-door models sampled uniformly, about $50\%$ of instances were p-imitable, and the average $L1$ was approximately $0.0016$ for the causal method versus approximately $0.0147$ for behavior cloning [2208.06267]. Sequential graphical C2L was then corroborated on randomized discrete SCMs and a HighD-based continuous experiment, where the sequential $\pi$-backdoor criterion achieved near-zero imitation error exactly in the cases predicted by theory and correctly identified non-imitable cases [2208.06276].

Context-specific structure was shown to change empirical feasibility as well as theory. In the synthetic economic model used for CSI-aware imitation, the causal algorithm achieved $\mathbb{E}[Y]=1.358$ and $D_{KL}=0.0007$, while two naive policies achieved $\mathbb{E}[Y]=1.194$ with $D_{KL}=0.0217$ and $\mathbb{E}[Y]=1.193$ with $D_{KL}=0.0219$. On random labeled DAGs, incorporating only three context variables significantly increased the fraction of imitable instances relative to the classic criterion [2306.00585].

Robotic and visual imitation studies extended the empirical scope of C2L. CIVIL outperformed BC, BYOL, VIOLA, Task-VIOLA, and CLIP on CALVIN tasks across demonstration budgets from $10$ to $120$. In Picking, CIVIL nearly always succeeded on unseen Center positions at $120$ demonstrations, whereas baselines were below $20\%$ success on Center despite above $50\%$ on seen Left and Right; the reported ANOVA statistic was $F(5,270)=179.84$, $p<0.001$. In the user study, CIVIL achieved above $77\%$ success versus about $40\%$ for BC under the same total time budget, with intuitive and seamless ratings significantly above neutral [2504.17959]. A separate ACT-based robotic study reported out-of-distribution transfer success of $0.82$ for Causal-ACT versus $0.23$ for ACT, with in-distribution transfer success of $0.96$ versus $0.89$; the best tuned domain-randomization baseline reached $0.91$ OOD transfer, while weaker randomization regimes ranged from $0.34$ to $0.61$ [2507.22380].

Instrumental and proximal methods were also evaluated beyond synthetic SCMs. DoubIL and ResiduIL compared favorably to behavioral cloning on LunarLander, HalfCheetahBulletEnv, and AntBulletEnv under temporally correlated noise, with lower MSE to $E[a\mid do(s)]$ and better returns under confounding and in noiseless generalization tests [2202.01312]. The IV-based C2L model with AB Criterion reported IV-identification accuracy that stabilized above $80\%$ and often above $90\%$ in continuous-control tasks, alongside policy improvements over BC, ResiduIL, and DoubIL in confounded settings [2507.17309]. Under measurement error and distribution shift, CausIL showed improved robustness relative to BC baselines on semi-simulated longitudinal data from the PhysioNet/Computing in Cardiology Challenge 2019 cohort, remaining stable under measurement-channel and dynamics shifts that degraded BC1 or BC2 [2601.22206].

## 6. Assumptions, limitations, and unresolved tensions

C2L methods rely on strong but explicit assumptions. Graphical approaches require a reasonably specified causal graph, admissible covariate sets, and positivity. Sequential graphical results require correct temporal ordering, c-components, and the availability of covariates before each action. Surrogate and proxy methods require either identifiable subspaces, valid surrogates, or completeness and rank conditions. Intervention-based methods require access to expert queries, environment interaction, episodic returns, or controllable initial-state interventions. Representation-based methods additionally assume that causal information is either disentangled or otherwise recoverable from masked features [2208.06267].

Several limitations recur across the literature. Passive causal discovery is unreliable in imitation learning because faithfulness can fail in control systems; in the original causal-confusion study, conditional mutual information $I(X_i^t;A^t\mid Z^t)$ was near-zero for both causal and nuisance variables in MountainCar, which rendered passive tests unusable [1905.11979]. Under CSI, even deciding imitability is NP-hard [2306.00585]. Under IV formulations, validity may be non-testable in linear Gaussian regimes: the AB Criterion cannot distinguish valid from invalid instruments when all relevant noise terms are Gaussian [2507.17309]. Under proximal formulations, very weak proxies or violations of completeness and positivity can destroy identification [2601.22206].

The literature also disagrees on how essential disentanglement is. Some methods explicitly depend on a disentangled representation to make graph search or masking tractable, and their reported performance degrades when latent factors are entangled [1905.11979]. Initial-state masking likewise assumes that nuisance variables are at least partially isolated in latent coordinates [2307.15980]. By contrast, the ACT-based causal-structure work argues that disentanglement is not necessary for learning the structural function into action, because the SCM is uniquely solvable with respect to $A$ [2507.22380]. This suggests that disentanglement is not a uniform prerequisite of C2L, but rather a method-dependent tradeoff between identifiability convenience, architectural simplicity, and search efficiency.

A final conceptual tension concerns the target itself. Some formulations pursue exact imitation of the expert’s outcome distribution, even with latent reward [2208.06267]. Others target causal parents of action [1905.11979], a history-dependent projection of expert behavior [2502.07656], or a policy robust to specific shifts such as measurement-channel change [2601.22206]. C2L is therefore not a single algorithmic recipe but a family of causal criteria and estimators for imitation under confounding. Its unifying claim is narrower and more precise: robust imitation requires distinguishing causal drivers of expert behavior from observational correlates, and that distinction generally cannot be recovered by standard behavioral cloning alone.

Source: https://www.emergentmind.com/topics/confounded-causal-imitation-learning-c2l