---
title: Behavioral Cloning from Observation (BCO)
url: https://www.emergentmind.com/topics/behavioral-cloning-from-observation-bco
type: topic
---

# Behavioral Cloning from Observation (BCO)

Behavioral Cloning from Observation (BCO) is a class of imitation learning methods for recovering control policies by observing only expert state trajectories, without access to their action sequences. BCO leverages self-supervision to infer actions via an inverse dynamics model, enabling behavioral cloning in settings where (state, action) expert pairs are unavailable. This framework underpins a corpus of methods that address sample efficiency, local minima, and domain transfer when learning from observation-only demonstrations.

## 1. Formal Problem Statement and Original Methodology

BCO is situated in Markov Decision Processes (MDPs), seeking to learn a policy $\pi(a|s)$ from a collection of state-only expert trajectories $D_{\rm demo} = \{\zeta^k\}_{k=1}^K$, where each $\zeta^k = (s_0^k, s_1^k, ..., s_{T_k}^k)$ and no expert actions are observed. The key assumption is that learners and experts share the same state and action spaces, but the environment dynamics $T$ are unknown. BCO introduces a self-supervised two-phase pipeline [1805.01954]:

**Phase 1: Inverse Dynamics Model Learning**
- The agent collects tuples $(s, a, s')$ by rolling out an exploration (typically random) policy pre-demonstration.
- An inverse dynamics model $M_\theta(a | s, s')$ is trained via maximum likelihood to predict actions given adjacent states.

**Phase 2: Imitation from Observation**
- For each transition $(s_t, s_{t+1})$ in the expert’s demonstrations, the model infers a pseudo-action $\hat a_t = \arg\max_a M_\theta(a | s_t, s_{t+1})$.
- Standard behavioral cloning is then performed: train $\pi_\phi(a|s)$ to predict pseudo-labels $\hat a_t$ from states $s_t$.

Optionally, BCO may be iterated with a small post-demonstration interaction budget to improve $M_\theta$ and $\pi_\phi$ in alternation [1805.01954].

## 2. Advances in Model Design and Sampling Strategies

The original BCO is susceptible to sub-optimal solutions due to poor action inference when dynamics are underexplored. Recent extensions address these shortcomings:

**Augmented Behavioral Cloning from Observation (ABCO)** implements two main advances [2004.13529]:
- **Goal-aware, Win/Loss Sampling:** At each iteration, data for retraining the inverse dynamics model are sampled in proportion to the demonstrator’s success probability in the environment, ensuring a curriculum from random exploratory to increasingly expert-like states.
- **Self-Attention Enhancements:** Incorporate self-attention modules into both inverse dynamics and policy networks, improving the modeling of long-range dependencies in both vector and image states.

**Imitating Unknown Policies via Exploration (IUPE)** combines balanced sampling (maintaining a mix of pre- and post-demonstration data), softmax-based stochastic exploration, and self-attention layers to encourage policy exploration and prevent collapse onto trivial policies [2008.05660]. Instead of always selecting maximum likelihood actions, actions are sampled from the inferred distribution, increasing exploration and, empirically, performance on high-dimensional control and visual domains.

**Concurrent Training Approaches (BCO*)** further eliminate the inefficiency of accumulating large random exploration buffers by interleaving policy improvement and inverse dynamics model retraining. Transitions for inverse model learning are continually collected on-policy, leading to a tighter feedback loop and dramatic reductions in sample complexity [2008.01205].

## 3. Adversarial and Self-Supervised Extensions

Standard BCO can fall into local minima where the policy performs “no action” loops due to imperfect action inference. To address this, adversarial frameworks have been proposed:

**Self-Supervised Adversarial Imitation Learning (SAIL)** augments the BCO pipeline with two additional components: a discriminator $D$ that distinguishes expert from learner trajectories based solely on states, and a forward-dynamics model $G$. The imitation policy, inverse model, and $G$ are trained jointly in a minimax game where D provides both an automated goal-criterion and adversarial pressure to avoid trivial (no-action) local minima [2304.10914]. The combined objective encourages the policy to generate state sequences that a discriminator cannot distinguish from the expert, thereby avoiding collapse to non-progressive behavior.

Experimental results demonstrate that SAIL matches or exceeds earlier BCO-style and adversarial approaches (GAIL, GAIfO, IUPE) on a suite of Gym tasks, particularly by eliminating the need for manual goal selection and user intervention.

## 4. Robustness, Partial Observability, and Causal Confounds

BCO and its relaxations are sensitive to dataset bias and nuisance correlates, especially under partial observability. In such settings, behavioral cloning from observation histories can yield “copycat” agents that infer the expert’s previous action rather than correct next action, relying on action autocorrelation in expert data [2010.14876]. This causal confusion leads to poor generalization under distribution shift.

To combat this, adversarial feature learning is employed: an encoder-decoder policy is trained alongside a target-conditioned adversary (TCA) that ensures the learned representation erases excess information about previous actions (nuisance variable) while retaining information necessary to predict the next action. An information bottleneck (KL penalty) is also imposed on the latent representation. This method achieves state-of-the-art offline performance on partially observed MuJoCo domains, demonstrating that careful regularization is essential for robust BCO in generalized settings.

## 5. Domain Transfer and Observation Mapping

Advancing beyond shared-environment imitation, BCO principles have been adapted for cross-domain policy transfer with disjoint observation spaces [2310.08836]. Here, unpaired CycleGANs learn mappings between source and target observations, with cycle-consistency losses ensuring invertibility. The mapped observations allow cloning a source-domain policy into the target, even under severe visual and semantic mismatch (including sim-to-real transfer). Only a small static set of target-domain images is needed, and no action or further real-world rollouts are required for BC. Reported success rates approach 90–94% with as few as 4,000 real images.

## 6. Empirical Evaluation and Comparative Results

BCO and its variants have been systematically evaluated on classic control (CartPole, Acrobot, MountainCar), maze navigation (Gym-Maze), and deep continuous domains (MuJoCo locomotion, TurtleBot real-world sim-to-real). Key metrics include Average Episodic Reward (AER) and Normalized Performance $P$ (scaled between random and expert returns). Across these benchmarks:

- Vanilla BCO, ABCO, and IUPE typically reach or exceed performance of baseline Behavioral Cloning (using ground-truth actions), GAIL, and related methods using only expert states and minimal environment interaction [2004.13529, 2008.05660].
- SAIL outperforms BCO and adversarial IfO baselines, particularly in avoiding high-variance loops and action-inactivity collapse [2304.10914].
- Methods introducing attention, balanced sampling, and joint optimization display substantially superior learning curves, especially on large maze and visually complex tasks.

| Method         | CartPole P | MountainCar P | Acrobot P | Maze 10x10 P |
|----------------|------------|---------------|-----------|--------------|
| BC             | 1.000      | 1.000         | 1.000     | 1.000        |
| BCO            | 1.000      | 0.948         | 0.980     | -0.416       |
| ABCO           | 1.000      | 1.289         | 1.071     | 0.860        |
| IUPE           | 1.135      | 1.314         | 1.086     | 1.000        |
| SAIL           | 1.000      | 0.990         | 0.990     | --           |

All reported values and further details are reproduced directly from the referenced experimental tables [2004.13529, 2008.05660, 2304.10914].

## 7. Limitations and Future Directions

Current BCO methodologies face several constraints:
- Inverse model quality depends on coverage and diversity of pre/post-demo exploration.
- Partitioning states into agent-specific and task-specific components often requires domain knowledge [1805.01954].
- Adversarial extensions add system complexity and impose capacity constraints on the discriminator to avoid overfitting on small demo sets [2304.10914].
- Iterative labeling loops may accumulate approximation errors if the inverse model fails to generalize.

Active research directions include:
- Scaling BCO to pixel-based observations and learning directly from raw video [2304.10914].
- Employing more advanced discriminator architectures (e.g., transformers) and data augmentation to increase robustness [2304.10914].
- Extending BCO frameworks to multi-modal actions, partial observability, and large-scale domain adaptation for broader applicability [2008.01205, 2310.08836].

A plausible implication is that future advances in semi-supervised learning, domain adaptation, and representation robustness will further close the remaining performance gap between observation-only and action-augmented imitation learning, as well as enable robust policy transfer from unstructured demonstration data at scale.

Source: https://www.emergentmind.com/topics/behavioral-cloning-from-observation-bco