Implicit Behavioral Cloning (IBC)

• Implicit Behavioral Cloning (IBC) is an imitation learning approach that represents policies via energy functions to capture multimodality and discontinuities in action mappings.
• It utilizes contrastive training and iterative inference methods like Langevin dynamics and derivative-free optimization to select expert-like actions.
• IBC has shown improved performance in robotic manipulation, urban driving, and fleet learning, despite challenges in computational cost and robustness.

Implicit Behavioral Cloning (IBC) is an imitation-learning paradigm in which the policy is represented implicitly by an energy function over state–action pairs, and action selection is performed by optimization or sampling in action space rather than by direct feed-forward regression. In its canonical form, IBC defines a conditional energy-based model (EBM) with policy density $\pi_\theta(a \mid s) \propto \exp(-E_\theta(s,a))$, so that low-energy actions are interpreted as expert-compatible actions; training is typically contrastive, and inference is iterative (Florence et al., 2021). The framework was introduced to address multimodal, discontinuous, and set-valued action mappings in robotic policy learning, and later work extended or adapted the idea to interactive fleet learning under heterogeneous supervision, off-policy robot motion planning with Dynamic Movement Primitives (DMPs), social nonverbal cue generation, multimodal driving, and robustness analysis under adversarial perturbations (Datta et al., 2023, Zhang et al., 2023, Martin-Ozimek et al., 18 Jan 2025, Antonelo et al., 18 Sep 2025, Patil et al., 6 Feb 2025).

1. Conceptual basis and motivation

The central departure of IBC from conventional behavior cloning is representational. Standard explicit BC learns a continuous mapping $a = F_\theta(o)$, usually by minimizing an MSE loss, while IBC defines the policy as the composition of an energy function and an optimization operator,

$\hat a = \arg\min_a E_\theta(o,a).$

This reformulation was motivated by the observation that many robotic policies are not well modeled as single-valued smooth functions: the same observation may admit multiple valid actions, and valid action sets may change discontinuously across state boundaries. In such settings, explicit regressors interpolate across modes and can produce invalid intermediate actions, whereas EBMs can place multiple low-energy basins and sharp transitions in action space (Florence et al., 2021).

The original IBC study argued that this distinction matters particularly for multimodal and discontinuous policies, including tasks with visual inputs, high-dimensional actions, contact-rich dynamics, and combinatorial structure. It also contrasted IBC with Mixture Density Networks (MDNs), noting that parametric mixture models can become brittle under high-dimensional visual conditioning because mixture count, variance structure, and temperature must all be tuned, whereas an energy landscape can represent multimodality through multiple minima without fixing a parametric mode count (Florence et al., 2021).

Subsequent applications preserved this motivation but instantiated it in distinct operational settings. In interactive fleet learning, IBC was used because heterogeneous human supervisors may demonstrate different valid strategies for the same state, so a multimodal conditional action model is preferable to a single regressed action (Datta et al., 2023). In urban driving, the same motivation appears in scenarios such as T-intersections, where turning left or right may both be valid, and standard BC was reported to suffer mode collapse, producing “in-between” commands (Antonelo et al., 18 Sep 2025). These later uses reinforce the original claim that IBC is fundamentally a response to multimodal action structure rather than merely a training trick.

2. Formalism, training objectives, and inference procedures

In the canonical EBM formulation, IBC defines

$$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$

Maximum-likelihood training would require computing $Z_\theta(s)$, which is intractable in continuous action spaces, so practical IBC implementations typically replace exact normalization with a contrastive surrogate. The original formulation used an InfoNCE-style loss in which an expert action $a_i$ is contrasted with sampled negatives $\{\tilde a_i^j\}$:

$$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$

The training objective minimizes $-\log \tilde p_\theta$ over the dataset, thereby lowering the energy of demonstrated actions relative to counter-examples (Florence et al., 2021).

Negative actions have been generated in several ways. The original derivative-free variant sampled negatives uniformly within buffered action bounds; the Langevin variant refined negatives with stochastic gradient Langevin dynamics (SGLD) and added an $\ell_\infty$ gradient penalty with margin $a = F_\theta(o)$0 for stability (Florence et al., 2021). In IIFL, training likewise used InfoNCE with 8 counter-examples per positive, an additional gradient penalty, and Langevin dynamics with 100 iterations and a polynomially decaying learning-rate schedule from $a = F_\theta(o)$1 to $a = F_\theta(o)$2; inference used 512 counter-examples (Datta et al., 2023). In the social-gaze replication, the implementation followed the public IBC code path with a contrastive EBM, uniform negatives over dataset-derived action bounds, and optional Langevin refinement, explicitly not using denoising score matching (DSM) (Martin-Ozimek et al., 18 Jan 2025).

Inference is correspondingly iterative. Three families appear in the literature. The first is derivative-free optimization (DFO), a CEM-like procedure that samples actions, evaluates energies, converts them to softmax weights, resamples, adds Gaussian noise, clips to bounds, shrinks the noise scale, and returns the best or a sampled low-energy candidate (Florence et al., 2021). The second is autoregressive derivative-free inference, in which one energy model is trained per action dimension and inference proceeds coordinate-wise, improving scalability to higher-dimensional actions (Florence et al., 2021). The third is Langevin inference,

$a = F_\theta(o)$3

which is used both for sampling negatives during training and for action refinement at test time (Florence et al., 2021, Datta et al., 2023).

A notable later variant departs from the original conditional-EBM formulation. In the robot motion-planning framework that combines IBC with DMPs and DDPG, the policy is a deterministic actor $a = F_\theta(o)$4 producing a DMP forcing action, and “IBC” is realized as a value-aware regularizer driven by the critic:

$a = F_\theta(o)$5

with

$a = F_\theta(o)$6

and combined actor loss

$a = F_\theta(o)$7

This formulation does not use Langevin or score-based sampling; actions are produced directly by the actor and perturbed only by Gaussian exploration noise (Zhang et al., 2023). This suggests that “IBC” in later robotics work may denote either the original conditional EBM with inference-time optimization or a broader class of implicit, value-gated imitation objectives.

3. Representational and theoretical properties

A major part of the original IBC contribution was theoretical. The paper states that for any set-valued function $a = F_\theta(o)$8 whose graph is closed, there exists a continuous energy function $a = F_\theta(o)$9 such that

$\hat a = \arg\min_a E_\theta(o,a).$0

for all $\hat a = \arg\min_a E_\theta(o,a).$1 (Florence et al., 2021). The construction uses the distance to the graph of $\hat a = \arg\min_a E_\theta(o,a).$2, yielding a continuous, $\hat a = \arg\min_a E_\theta(o,a).$3-Lipschitz energy whose minima coincide with the valid action set. A related approximation result states that if $\hat a = \arg\min_a E_\theta(o,a).$4 approximates $\hat a = \arg\min_a E_\theta(o,a).$5 with bounded error $\hat a = \arg\min_a E_\theta(o,a).$6, then any inferred minimizer lies within $\hat a = \arg\min_a E_\theta(o,a).$7 of the true graph (Florence et al., 2021).

These results formalize the intuition that implicit policies can represent multi-valued and discontinuous mappings without requiring a continuous regressor to interpolate across incompatible labels. In the original exposition, this was linked to empirical behavior such as piecewise-linear extrapolation near data boundaries and preservation of sharp transitions, in contrast to explicit regressors that average across modes (Florence et al., 2021). A plausible implication is that IBC’s empirical gains in manipulation and control are not solely due to optimization heuristics, but also to the fact that the hypothesis class matches the geometry of set-valued action mappings more closely.

Later studies supplied concrete energy-landscape illustrations of this claim. In CARLA, the learned energy $\hat a = \arg\min_a E_\theta(o,a).$8 over $\hat a = \arg\min_a E_\theta(o,a).$9 exhibited multiple minima at T-intersections, with distinct low-energy basins corresponding to different valid maneuvers under the same scene, while red-light scenes produced a sharp unimodal minimum near $$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$0 (Antonelo et al., 18 Sep 2025). The study further reported that minima were flatter in multimodal contexts and sharper in unimodal stopping contexts, and that repeated DFO runs clustered around low-energy regions, directly visualizing the multi-basin interpretation of IBC (Antonelo et al., 18 Sep 2025).

4. Variants, extensions, and system integrations

One major extension of IBC is Implicit Interactive Fleet Learning (IIFL), which embeds an IBC policy inside a DAgger-style fleet-learning system with $$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$1 robots, $$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$2 remote supervisors, synchronous timesteps, and a single shared EBM policy trained on aggregated multi-human data (Datta et al., 2023). The key addition is uncertainty-aware human allocation using a bootstrap ensemble of two EBMs and Jeffreys divergence,

$$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$3

where $$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$4 and $$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$5. Because the partition functions cancel symmetrically, IIFL estimates

$$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$6

using Langevin samples from each EBM, and ranks robots for intervention by uncertainty and hard-reset status (Datta et al., 2023). The extension is significant because it treats multimodality not only as a representation problem but also as a supervisory-allocation problem under distribution shift.

A second line of work integrated IBC into RL-based robot motion planning via DMPs. In that framework, the policy outputs the DMP forcing term $$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$7 rather than raw trajectories, and DMP dynamics enforce smoothness and convergence:

$$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$8

IBC enters as a critic-gated actor regularizer, combined with demonstration initialization for replay (DIR), dual buffers, and demo-heavy critic refinement early in training (Zhang et al., 2023). In this setting, DMPs reduce the effective planning space, while IBC only imitates demonstration actions that the current critic judges better than the actor’s action. The paper explicitly contrasts this with explicit BC, which penalizes action mismatch regardless of value and was reported to overfit the demonstrations (Zhang et al., 2023).

A third extension, CLIC, reinterprets the weakness of vanilla IBC as an assumption problem: “the expert action is optimal and all others are negative.” CLIC replaces pointwise labels by desired action sets inferred from interactive corrections and optimizes a KL divergence between the current policy and a Bayes target over sampled actions (Li et al., 11 Feb 2025). For energy-based policies, the sampled-set policy is defined by an energy softmax, and the target may be policy-weighted,

$$\pi_\theta(a \mid s) = \frac{\exp(-E_\theta(s,a))}{Z_\theta(s)}, \qquad Z_\theta(s) = \int \exp(-E_\theta(s,a))\, da.$$9

which yields conservative updates under noisy feedback (Li et al., 11 Feb 2025). Theoretical analysis in that work states that, under finite optimal-action sets and persistent correction of suboptimal actions, the estimated overall desired set converges to the optimal-action set or a subset thereof (Li et al., 11 Feb 2025). Relative to classical IBC, this reframes imitation as desired-set alignment rather than single-label attraction.

5. Empirical performance across domains

The empirical record for IBC is heterogeneous but extensive. The original study reported that implicit policies often outperformed explicit MSE and MDN baselines on simulated pushing, sweeping, bi-manual manipulation, D4RL human-expert tasks, and several real-robot RGB-only manipulation problems (Florence et al., 2021). On D4RL kitchen-complete, for example, Explicit BC (MSE) achieved $Z_\theta(s)$0 normalized return, while Implicit BC (EBM) achieved $Z_\theta(s)$1; on the real-robot Insert-Blue task with 1 mm tolerance, EBM achieved $Z_\theta(s)$2 success versus $Z_\theta(s)$3 for MSE, and on Sort-Blue-from-Yellow it achieved $Z_\theta(s)$4 versus $Z_\theta(s)$5 (Florence et al., 2021).

Later domain-specific studies reported more nuanced outcomes. In social gaze generation, IBC replicated endpoint-reaching performance and produced smoother trajectories, but explicit MSE BC fit the observed trajectories more closely in $Z_\theta(s)$6 (Martin-Ozimek et al., 18 Jan 2025). In route-free urban driving, standard IBC underperformed both BC and the proposed DA-IBC on free-navigation distance, even though DA-IBC preserved the multimodal EBM formulation and improved negative sampling and DFO initialization (Antonelo et al., 18 Sep 2025). In interactive settings, however, IBC-derived systems often benefited from explicit uncertainty handling or iterative feedback aggregation, as in IIFL and CLIC (Datta et al., 2023, Li et al., 11 Feb 2025).

Setting	Representative result	Paper
D4RL and real-robot manipulation	kitchen-complete: MSE $Z_\theta(s)$7, EBM $Z_\theta(s)$8; Insert-Blue: EBM $Z_\theta(s)$9, MSE $a_i$0	(Florence et al., 2021)
Off-policy motion planning with DMPs	Training $a_i$1: IBC-DMP $a_i$2; test collision rate $a_i$3: $a_i$4	(Zhang et al., 2023)
Interactive fleet learning	$a_i$5 higher success rate in simulation and $a_i$6 higher ROHE on the physical task over explicit IFL/IBC baselines	(Datta et al., 2023)
Nonverbal cue generation	ASM: IBC $a_i$7, MSE $a_i$8; IBC smoother by SPARC, MSE higher in yaw/pitch $a_i$9	(Martin-Ozimek et al., 18 Jan 2025)
Multimodal urban driving	Average distance traveled: BC $\{\tilde a_i^j\}$0, IBC $\{\tilde a_i^j\}$1, DA-IBC $\{\tilde a_i^j\}$2	(Antonelo et al., 18 Sep 2025)
Interactive imitation with desired action sets	Accurate-feedback average SR: CLIC-Circular $\{\tilde a_i^j\}$3, Diffusion $\{\tilde a_i^j\}$4, IBC $\{\tilde a_i^j\}$5	(Li et al., 11 Feb 2025)

These results support two simultaneous conclusions. First, the original EBM formulation can outperform explicit regression by a large margin when the task truly exhibits discontinuity, multi-modality, combinatorial structure, or tight contact precision (Florence et al., 2021). Second, later studies indicate that vanilla IBC is not uniformly dominant: performance depends on inference design, negative-sample quality, feedback noise, and whether the deployment setting requires adaptation beyond static offline imitation (Zhang et al., 2023, Li et al., 11 Feb 2025).

6. Limitations, robustness, and ongoing directions

A recurrent limitation of IBC is computational cost. Because inference requires inner-loop optimization or sampling, IBC is slower than explicit feed-forward BC. In the original D4RL study, training plus evaluation on TPUv3 was reported as approximately $\{\tilde a_i^j\}$6 hours for IBC with 100 Langevin iterations versus approximately $\{\tilde a_i^j\}$7 hours for explicit MSE, although reported CQL training could take approximately $\{\tilde a_i^j\}$8 hours on similar tasks; on real-robot vision models, deployment inference on RTX 2080 Ti was approximately $\{\tilde a_i^j\}$9 ms for IBC versus approximately $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$0 ms for MSE (Florence et al., 2021). In IIFL, the overhead was more severe relative to explicit IFL: training step time was approximately $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$1 slower, inference approximately $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$2 slower, and uncertainty estimation approximately $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$3 slower, even though uncertainty remained sub-second for 100 robots (Datta et al., 2023).

A second limitation is that multimodal representation alone does not solve distribution shift or supervision noise. In the 2D navigation benchmark with wind shift, both BC and IBC achieved $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$4 success, whereas IIFL achieved $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$5 success after interactive aggregation, showing that the fleet-learning mechanism rather than the static implicit policy was responsible for recovery under shift (Datta et al., 2023). CLIC makes a related criticism from the standpoint of label quality: because vanilla IBC treats the demonstrated action as the sole positive and all others as negatives, expressive EBMs may overfit noisy or suboptimal labels, producing unstable training and sharp spurious minima; CLIC addresses this with desired-action sets, softened observation models, and policy-weighted Bayes targets (Li et al., 11 Feb 2025).

A third limitation is security and robustness under observation perturbations. A dedicated adversarial study found that IBC, while sometimes more robust than explicit BC, remained highly vulnerable to white-box PGD and universal adversarial perturbations. On Lift, clean IBC success was approximately $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$6 and dropped to approximately $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$7 under untargeted PGD; on Push-T, clean success was approximately $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$8 and dropped to approximately $$\tilde p_\theta(a_i \mid s_i,\{\tilde a_i^j\}) = \frac{\exp(-E_\theta(s_i,a_i))} {\exp(-E_\theta(s_i,a_i)) + \sum_j \exp(-E_\theta(s_i,\tilde a_i^j))}.$$9 (Patil et al., 6 Feb 2025). Randomized smoothing improved PGD robustness for IBC to approximately $-\log \tilde p_\theta$0 on both Lift and Push-T, but on Lift it also reduced clean success from approximately $-\log \tilde p_\theta$1 to approximately $-\log \tilde p_\theta$2, illustrating a robustness–accuracy trade-off and the difficulty of applying classification-style smoothing to continuous multimodal control (Patil et al., 6 Feb 2025).

Domain-specific failure modes also recur. In the DMP-based motion-planning variant, state-dependent forcing can alter closed-loop poles and may sacrifice inherent DMP stability, producing odd-shaped trajectories when actions saturate; the same framework reported sensitivity to action noise, reward shaping, and overtraining under off-policy bootstrapping (Zhang et al., 2023). In the social-interaction replication, extension from gaze to full pose degraded due to PD overshoot and increased action-space complexity (Martin-Ozimek et al., 18 Jan 2025). In multimodal driving, DA-IBC and standard IBC exhibited more traffic-light infractions than BC, and the method remained compute-intensive because each control step required $-\log \tilde p_\theta$3 forward passes of the energy network (Antonelo et al., 18 Sep 2025).

Current research directions are correspondingly diverse. The original IBC paper identified structured proposals, adaptive negative sampling, uncertainty estimation, and tighter integration with reward-aware algorithms as open questions (Florence et al., 2021). IIFL proposed extending other multimodal policy families, including Behavior Transformers and Diffusion Policies, to interactive fleet learning, while the social-gaze work suggested DSM and annealed Langevin dynamics as future alternatives to the contrastive code path (Datta et al., 2023, Martin-Ozimek et al., 18 Jan 2025). Robustness work recommended adversarial training directly on the energy function and harder negative sampling near realistic off-policy modes, while CLIC identified the integration of non-intervention data and evaluative feedback as unresolved (Patil et al., 6 Feb 2025, Li et al., 11 Feb 2025). Taken together, these directions indicate that IBC has become less a single algorithm than a family of implicit policy-learning methods centered on energy shaping, iterative inference, and multimodal action selection.