Confounded Causal Imitation Learning (C2L)

• Confounded Causal Imitation Learning (C2L) is a framework that addresses imitation learning under confounding, where standard behavioral cloning misidentifies causal drivers due to spurious observational correlations.
• C2L employs causal graphical models and intervention strategies—such as expert queries, state masking, and instrumental methods—to recover the true causal structure guiding expert actions.
• Empirical studies in simulated, driving, and robotic domains demonstrate that deconfounding techniques significantly improve policy performance under distribution shifts and measurement errors.

Confounded Causal Imitation Learning (C2L) denotes imitation learning under confounding: expert demonstrations contain observational dependencies between observations and actions that need not remain valid once the learner is deployed. In this setting, behavioral cloning can attain low supervised loss while producing policies that fail under the learner-induced state distribution, because it estimates observational conditionals such as $P(a\mid o)$ rather than the interventional or otherwise deconfounded targets required for robust control. Across the literature, the same core problem appears as causal confusion or causal misidentification, imitation with unobserved confounders, sensory mismatch, temporally correlated noise, measurement error, and context-specific causal structure (Haan et al., 2019).

1. Causal confusion and the observational–interventional gap

In the canonical formulation, imitation learning is posed on a (partially observed) Markov decision process with states $s$, observations $o$, actions $a$, expert policy $\pi_E(a\mid o)$, learner policy $\pi_\theta(a\mid o)$, and dynamics $P(s' \mid s,a)$. Demonstrations $D_{\pi_E}$ are tuples $(o_t,a_t)$ drawn from trajectories generated by $\pi_E$ under the induced state distribution $s$0. Behavioral cloning trains a discriminative mapping $s$1 by minimizing

$s$2

typically using cross-entropy for discrete actions or mean squared error for continuous actions. Deployment, however, evaluates the learner under its own induced state distribution $s$3, so the relevant quantity is not training loss but performance under closed-loop interaction, written in the causal-confusion analysis as

$s$4

The mismatch between $s$5 and $s$6 is the source of failure (Haan et al., 2019).

The distinctive C2L pathology is causal misidentification. A cloned policy may exploit nuisance correlates in $s$7 that are predictive in demonstrations because they are effects of past state and action, yet are not causes of the expert’s current action. The brake-light example illustrates the mechanism: an effect-of-action feature correlates with braking in demonstrations and can therefore deceive a non-causal learner. A central empirical and conceptual point is that more observational information can worsen performance, because additional features enlarge the space of spurious correlations that collapse under deployment shift (Haan et al., 2019).

The causal account is expressed through a structural causal model in which the observation factors are $s$8, the expert action is $s$9, and the confounder is $o$0. Structural equations take the form

$o$1

with a DAG $o$2 encoding which observation components are genuine parents of action. Observationally, behavioral cloning approximates $o$3 on expert data. Causally robust behavior requires the interventional query $o$4, computed on the mutilated graph where incoming edges into $o$5 are removed. Under the faithfulness result in the original causal-confusion analysis, any learner that matches all interventional queries $o$6 must recover the true graph $o$7 (Haan et al., 2019).

2. Imitability as a graphical property

A second line of work formulates C2L in a single-step Partially Observable Structural Causal Model (POSCM). Here, $o$8 is the action variable, $o$9 is a latent reward, $a$0 is the set of observed endogenous variables, $a$1 is the set of latent endogenous variables, and $a$2 denotes exogenous variables that may induce unobserved confounding. A policy $a$3 intervenes stochastically on $a$4, and the performance target is the causal quantity $a$5. Because $a$6 is latent, $a$7 is generally not identifiable non-parametrically from $a$8 alone. The relevant question is therefore imitability: whether there exists a policy computable from $a$9 that matches the expert’s outcome distribution $\pi_E(a\mid o)$0 for all SCMs compatible with the graph (Zhang et al., 2022).

The central result is the complete $\pi_E(a\mid o)$1-backdoor criterion. For a graph $\pi_E(a\mid o)$2 and policy space $\pi_E(a\mid o)$3, a set $\pi_E(a\mid o)$4 is $\pi_E(a\mid o)$5-backdoor admissible if

$\pi_E(a\mid o)$6

Then $\pi_E(a\mid o)$7 is imitable with respect to $\pi_E(a\mid o)$8 if and only if such a $\pi_E(a\mid o)$9 exists, and the imitating policy is

$\pi_\theta(a\mid o)$0

This theorem gives a formal justification for behavior cloning only in the special case where the learner observes the expert’s relevant parents and there is no unobserved confounder entering $\pi_\theta(a\mid o)$1. When those conditions fail, naive cloning on predictive covariates can be biased even if validation accuracy is high (Zhang et al., 2022).

When the graphical criterion fails, the same framework introduces practical imitability, or p-imitability, which conditions on both the graph and the realized observational distribution $\pi_\theta(a\mid o)$2. The fallback mechanism uses surrogates and identifiable policy subspaces. If a surrogate set $\pi_\theta(a\mid o)$3 satisfies the required separation condition in the augmented graph and $\pi_\theta(a\mid o)$4 is identifiable in a subspace $\pi_\theta(a\mid o)$5, then solving

$\pi_\theta(a\mid o)$6

is sufficient to imply

$\pi_\theta(a\mid o)$7

This yields front-door-style constructions even when reward is latent and no direct reward supervision is available (Zhang et al., 2022).

Context-specific independence extends this picture by replacing DAGs with labeled DAGs whose edges can disappear in specific contexts. In that setting, the imitability decision problem is NP-hard, the necessary criterion requires classic imitability to hold in every context-induced DAG, and—under the structural assumption $\pi_\theta(a\mid o)$8—that per-context condition is also sufficient. The resulting policies are context-gated mixtures of per-context imitators, rather than a single global adjustment rule (Jamshidi et al., 2023).

3. Sequential C2L and temporal structure

In sequential decision-making, the imitator acts multiple times per episode, and confounding can propagate through both state evolution and observation availability. The sequential formulation models the environment as an SCM over a time-ordered set of variables, with demonstrator policy $\pi_\theta(a\mid o)$9, imitator policy $P(s' \mid s,a)$0, observed covariates $P(s' \mid s,a)$1 for each action $P(s' \mid s,a)$2, and a latent performance variable $P(s' \mid s,a)$3 defined over the entire episode. Imitability requires a policy discernible from $P(s' \mid s,a)$4 such that

$P(s' \mid s,a)$5

for all SCMs compatible with the causal diagram (Kumor et al., 2022).

The key sequential result is the sequential $P(s' \mid s,a)$6-backdoor criterion. For each action $P(s' \mid s,a)$7, one constructs a manipulated graph $P(s' \mid s,a)$8 in which future actions are treated as controlled by the imitator. Then, for each $P(s' \mid s,a)$9, either backdoor blocking must hold at $D_{\pi_E}$0 in $D_{\pi_E}$1, or $D_{\pi_E}$2 must cease to be an ancestor of $D_{\pi_E}$3 in $D_{\pi_E}$4. If suitable sets $D_{\pi_E}$5 exist, the policy

$D_{\pi_E}$6

exactly matches the demonstrator’s performance. The associated algorithm, FindOx, runs in polynomial time, constructs the maximal imitable subset of actions, and proves non-imitability when some actions cannot be included (Kumor et al., 2022).

A notable consequence is that the standard sequential backdoor criterion of causal-effect identification is not sufficient for imitation. Earlier actions may be locally non-imitable yet made irrelevant globally by downstream policies that shield the final outcome from upstream mistakes. This distinction is specific to imitation: the objective is not merely to identify $D_{\pi_E}$7, but to reproduce the demonstrator’s performance under observation mismatch and latent structure (Kumor et al., 2022).

A separate unifying framework introduces two types of hidden confounders in sequential IL: expert-observed hidden variables $D_{\pi_E}$8 and confounding noise $D_{\pi_E}$9 hidden to both expert and learner. With confounding noise horizon $(o_t,a_t)$0, additive action equation $(o_t,a_t)$1, and history $(o_t,a_t)$2, the target becomes the history-dependent policy

$(o_t,a_t)$3

The identification reduces to conditional moment restrictions,

$(o_t,a_t)$4

and the imitation gap of DML-IL is bounded by

$(o_t,a_t)$5

which recovers earlier special cases when either $(o_t,a_t)$6 or $(o_t,a_t)$7 (Shao et al., 11 Feb 2025).

4. Deconfounding strategies: interventions, instruments, proxies, and causal features

One major strategy for C2L is direct intervention. In the causal-confusion framework, the learner trains a family of graph-parameterized policies by masking candidate parent sets:

$(o_t,a_t)$8

with shared parameters amortized across graphs. Two intervention modes are then used to identify the causal graph. In expert-query mode, the learner executes a mixture policy, scores states by disagreement

$(o_t,a_t)$9

queries the expert on the most disagreeing states, and fits an energy-based posterior over graphs. In policy-execution mode, the learner executes $\pi_E$0, records episodic return $\pi_E$1, and again fits an energy-based posterior $\pi_E$2, factorized into Bernoulli components under a linear energy model. The outcome is a graph whose parents remain predictive under interventions rather than mere observation (Haan et al., 2019).

A more restrictive but highly practical intervention regime masks observed nuisance variables in a disentangled latent representation. Initial-state interventions randomize $\pi_E$3 with an everywhere-nonzero density, thereby removing the edge from latent seed $\pi_E$4 into the initial state and breaking spurious forks that create false dependencies. The masking rule declares an observation coordinate removable when no state coordinate is dependent with both that observation and any action within a reaction horizon. Under the stated assumptions, the method is conservative: it does not incorrectly mask genuinely causal observations, and intervening on the initial state is provably strictly less conservative for a class of nuisance variables (Pfrommer et al., 2023).

Visual imitation learning introduces another intervention style by augmenting demonstrations with human-provided causal cues. CIVIL uses physical markers and language prompts to construct image masks $\pi_E$5, define masked images $\pi_E$6, and train a causal feature representation $\pi_E$7. The encoder is supervised explicitly on marker-derived positional features and implicitly through policy cloning on masked images, with objectives

$\pi_E$8

in phase 1 and a distillation objective $\pi_E$9 in phase 2. The stated aim is to recover features that causally inform human actions while excluding distractors, and deployment uses the distilled encoder without requiring markers or prompts online (Dai et al., 24 Apr 2025).

Recent robotic work embeds similar masking ideas directly into high-capacity transformer policies. One such framework gates encoder features by a binary graph $s$00, trains an ACT-style policy under randomized masks, and performs a post-training energy-based search over graphs using episodic reward. Its theoretical claim is that disentanglement is not necessary for learning the structural relationship from observations to action, because the SCM is uniquely solvable with respect to the action variable as long as there is no self-loop $s$01 (Chen et al., 30 Jul 2025).

A different family of methods identifies deconfounded policies from instruments or proxies. Under temporally correlated action noise, DoubIL uses a simulator to resample next states from past states and a first-stage policy, whereas ResiduIL enforces instrumental-variable moment conditions entirely offline through a minimax objective over residuals. Both treat the past state $s$02 as an instrument for the confounded relation between current state and action (Swamy et al., 2022). Multi-step confounding generalizes this by searching over lagged states $s$03 as candidate instruments. The key diagnostic is the pseudo-variable residual

$s$04

together with the AB Criterion, which requires $s$05 when $s$06 is a valid instrument. Under partial non-Gaussianity in linear models, or under a non-degenerate cross-derivative condition in nonlinear models, this criterion is necessary and sufficient for IV validity (Zeng et al., 23 Jul 2025).

Measurement error leads to a proxy-based proximal formulation. CausIL treats the lagged state $s$07 as a treatment-inducing proxy and noisy measurement $s$08 as an outcome-inducing proxy for the latent state $s$09. The deconfounded target is

$s$10

which is identified in the discrete case by

$s$11

and in the continuous case by bridge functions estimated through an RKHS adversarial procedure. The target policy is robust to shifts in the measurement channel $s$12 and to many dynamics shifts that preserve the marginal distribution of the latent state (Bo et al., 29 Jan 2026).

5. Empirical domains and reported findings

The earliest empirical evidence for C2L was obtained on modified control benchmarks, Atari, and realistic driving. Augmenting observations with previous actions or previous-action symbols produced policies with near-zero validation loss yet worse deployment reward than policies trained on the original observations, demonstrating causal misidentification directly. In MountainCar and Hopper, policy-execution interventions approached original performance after tens of episodes; on Hopper, Generative Adversarial Imitation Learning required approximately $s$13 episodes to match. Expert-query interventions improved rewards within few queries and outperformed dropout and DAgger under comparable query budgets; DAgger required hundreds of queries on MountainCar and tens of thousands on Hopper to close the gap. The same study also reported that entangled rotations of the MountainCar state reduced effectiveness, with rewards dropping from $s$14 to $s$15 in policy-execution mode and from $s$16 to $s$17 in expert-query mode (Haan et al., 2019).

Graphical single-step C2L was evaluated on highway driving and a front-door MNIST construction. In highD-based driving, causal imitation matched the expert outcome distribution with $s$18 distance $s$19, whereas behavior cloning using $s$20 failed with $s$21. In the MNIST front-door setting, causal imitation achieved $s$22 and behavior cloning $s$23. In synthetic binary front-door models sampled uniformly, about $s$24 of instances were p-imitable, and the average $s$25 was approximately $s$26 for the causal method versus approximately $s$27 for behavior cloning (Zhang et al., 2022). Sequential graphical C2L was then corroborated on randomized discrete SCMs and a HighD-based continuous experiment, where the sequential $s$28-backdoor criterion achieved near-zero imitation error exactly in the cases predicted by theory and correctly identified non-imitable cases (Kumor et al., 2022).

Context-specific structure was shown to change empirical feasibility as well as theory. In the synthetic economic model used for CSI-aware imitation, the causal algorithm achieved $s$29 and $s$30, while two naive policies achieved $s$31 with $s$32 and $s$33 with $s$34. On random labeled DAGs, incorporating only three context variables significantly increased the fraction of imitable instances relative to the classic criterion (Jamshidi et al., 2023).

Robotic and visual imitation studies extended the empirical scope of C2L. CIVIL outperformed BC, BYOL, VIOLA, Task-VIOLA, and CLIP on CALVIN tasks across demonstration budgets from $s$35 to $s$36. In Picking, CIVIL nearly always succeeded on unseen Center positions at $s$37 demonstrations, whereas baselines were below $s$38 success on Center despite above $s$39 on seen Left and Right; the reported ANOVA statistic was $s$40, $s$41. In the user study, CIVIL achieved above $s$42 success versus about $s$43 for BC under the same total time budget, with intuitive and seamless ratings significantly above neutral (Dai et al., 24 Apr 2025). A separate ACT-based robotic study reported out-of-distribution transfer success of $s$44 for Causal-ACT versus $s$45 for ACT, with in-distribution transfer success of $s$46 versus $s$47; the best tuned domain-randomization baseline reached $s$48 OOD transfer, while weaker randomization regimes ranged from $s$49 to $s$50 (Chen et al., 30 Jul 2025).

Instrumental and proximal methods were also evaluated beyond synthetic SCMs. DoubIL and ResiduIL compared favorably to behavioral cloning on LunarLander, HalfCheetahBulletEnv, and AntBulletEnv under temporally correlated noise, with lower MSE to $s$51 and better returns under confounding and in noiseless generalization tests (Swamy et al., 2022). The IV-based C2L model with AB Criterion reported IV-identification accuracy that stabilized above $s$52 and often above $s$53 in continuous-control tasks, alongside policy improvements over BC, ResiduIL, and DoubIL in confounded settings (Zeng et al., 23 Jul 2025). Under measurement error and distribution shift, CausIL showed improved robustness relative to BC baselines on semi-simulated longitudinal data from the PhysioNet/Computing in Cardiology Challenge 2019 cohort, remaining stable under measurement-channel and dynamics shifts that degraded BC1 or BC2 (Bo et al., 29 Jan 2026).

6. Assumptions, limitations, and unresolved tensions

C2L methods rely on strong but explicit assumptions. Graphical approaches require a reasonably specified causal graph, admissible covariate sets, and positivity. Sequential graphical results require correct temporal ordering, c-components, and the availability of covariates before each action. Surrogate and proxy methods require either identifiable subspaces, valid surrogates, or completeness and rank conditions. Intervention-based methods require access to expert queries, environment interaction, episodic returns, or controllable initial-state interventions. Representation-based methods additionally assume that causal information is either disentangled or otherwise recoverable from masked features (Zhang et al., 2022).

Several limitations recur across the literature. Passive causal discovery is unreliable in imitation learning because faithfulness can fail in control systems; in the original causal-confusion study, conditional mutual information $s$54 was near-zero for both causal and nuisance variables in MountainCar, which rendered passive tests unusable (Haan et al., 2019). Under CSI, even deciding imitability is NP-hard (Jamshidi et al., 2023). Under IV formulations, validity may be non-testable in linear Gaussian regimes: the AB Criterion cannot distinguish valid from invalid instruments when all relevant noise terms are Gaussian (Zeng et al., 23 Jul 2025). Under proximal formulations, very weak proxies or violations of completeness and positivity can destroy identification (Bo et al., 29 Jan 2026).

The literature also disagrees on how essential disentanglement is. Some methods explicitly depend on a disentangled representation to make graph search or masking tractable, and their reported performance degrades when latent factors are entangled (Haan et al., 2019). Initial-state masking likewise assumes that nuisance variables are at least partially isolated in latent coordinates (Pfrommer et al., 2023). By contrast, the ACT-based causal-structure work argues that disentanglement is not necessary for learning the structural function into action, because the SCM is uniquely solvable with respect to $s$55 (Chen et al., 30 Jul 2025). This suggests that disentanglement is not a uniform prerequisite of C2L, but rather a method-dependent tradeoff between identifiability convenience, architectural simplicity, and search efficiency.

A final conceptual tension concerns the target itself. Some formulations pursue exact imitation of the expert’s outcome distribution, even with latent reward (Zhang et al., 2022). Others target causal parents of action (Haan et al., 2019), a history-dependent projection of expert behavior (Shao et al., 11 Feb 2025), or a policy robust to specific shifts such as measurement-channel change (Bo et al., 29 Jan 2026). C2L is therefore not a single algorithmic recipe but a family of causal criteria and estimators for imitation under confounding. Its unifying claim is narrower and more precise: robust imitation requires distinguishing causal drivers of expert behavior from observational correlates, and that distinction generally cannot be recovered by standard behavioral cloning alone.