Diffusion Steering via Behavioral Cloning (DSBC)

• The paper introduces DSBC, a method that enhances diffusion-based behavioral cloning by integrating self-guidance and adaptive chunking without extra reward signals.
• The approach leverages DDPM frameworks and past state negation to steer policy sampling, improving action fidelity, temporal consistency, and task responsiveness.
• Empirical results demonstrate over 23% improvement in success rates and reduced compute by 16× in robotic manipulation tasks under varying noise conditions.

Diffusion Steering via Behavioral Cloning (DSBC) refers to a set of techniques for steering diffusion-based policies in generative behavior cloning (GBC), with an emphasis on robot learning in multi-task settings. The core approach involves a diffusion policy that synthesizes action trajectories (“chunks”) using denoising diffusion probabilistic models (DDPMs). DSBC specifically augments standard DDPM-based behavior cloning by introducing self-guidance for adaptive policy steering and adaptive chunking for online control refinement. These innovations aim to improve action fidelity, temporal consistency, and task responsiveness, all without requiring external reward signals or retraining. The framework substantially improves success rates and robustness in both simulated and real-world robotic manipulation tasks (So et al., 14 Oct 2025).

1. Diffusion Policy Formulation in Behavioral Cloning

DSBC builds on the DDPM framework by treating expert action chunks $A_0 \in \mathbb{R}^n$ ($n = H \cdot d_a$; $H$ is chunk length, $d_a$ action dimensions) as data. The forward process is a Markov chain:

$$q(A_k|A_{k-1}) = \mathcal{N}(A_k; \sqrt{1-\beta_k}A_{k-1}, \beta_k I),$$

with closed-form marginal

$$q(A_k|A_0) = \mathcal{N}(A_k; \sqrt{\bar\alpha_k}A_0, (1-\bar\alpha_k)I),$$

where $\alpha_k = 1-\beta_k$, $\bar\alpha_k = \prod_{i=1}^k\alpha_i$.

The reverse denoising network $\epsilon_\theta$ is trained to predict injected noise in

$$A_k = \sqrt{\bar\alpha_k}A_0 + \sqrt{1-\bar\alpha_k}\epsilon, \quad \epsilon \sim \mathcal{N}(0,I),$$

minimizing the denoising-score matching loss

$n = H \cdot d_a$0

At inference, chunk generation proceeds by iterative denoising from $n = H \cdot d_a$1:

$n = H \cdot d_a$2

where $n = H \cdot d_a$3, $n = H \cdot d_a$4 controlled by $n = H \cdot d_a$5. The decoded chunk yields actions $n = H \cdot d_a$6; either all ($n = H \cdot d_a$7; open-loop) or first $n = H \cdot d_a$8 steps are executed before replanning.

The process can be formulated as a probability-flow ODE for continuous-time generalization:

$n = H \cdot d_a$9

with the score function approximated by $H$0.

2. Self-Guidance: Diffusion Steering via Past State Negation

Instead of steering with external classifiers or rewards, DSBC leverages a novel self-guidance objective using the model’s own predictions at past and current states. This approach directs the reverse process to prefer action samples that are likely under the current observation $H$1 and unlikely under the prior observation $H$2. The intuition is to amplify modes adaptable to recent environmental changes.

The revised noise estimation for denoising is:

$H$3

where $H$4 is the guidance weight.

This corresponds to sampling from the reweighted distribution:

$H$5

Each denoising step uses $H$6 in place of the standard noise estimation.

The full chunk sampling pseudocode is: $$q(A_k|A_{k-1}) = \mathcal{N}(A_k; \sqrt{1-\beta_k}A_{k-1}, \beta_k I),$$4 No additional losses or finetuning are required; self-guidance is implemented at inference by a second forward pass with past state.

3. Adaptive Chunking for Temporal Consistency and Reactivity

Chunking in DSBC refers to generating action trajectories in blocks (chunks) and selectively replanning to balance the trade-off between temporal consistency (smooth execution) and reactivity (adaptation to new states).

Open-loop execution (OL, $H$7): execute a multi-step chunk without replanning, resulting in high consistency but potential reaction delay. Closed-loop execution (CL, $H$8): replan at every timestep, offering high reactivity but potentially jittery action sequences.

Adaptive chunking dynamically selects when to replan by maintaining a FIFO action queue $H$9. At each timestep:

• Sample a new candidate chunk $d_a$0 using self-guidance.
• Compare the first step $d_a$1 of $d_a$2 to $d_a$3 via cosine similarity.
• If $d_a$4 (threshold), append the remaining chunk and continue (open-loop).
• If $d_a$5, reset $d_a$6 (force closed-loop adaptation).

This mechanism allows the policy to preserve consistency during routine phases and to react quickly in precision-demanding or state-changing phases.

Execution Mode	Replanning Frequency	Pros	Cons
Open-loop	Low ($d_a$7)	Consistency	Delayed reactivity
Closed-loop	High ($d_a$8)	Reactivity	Temporal jitter
Adaptive	Dynamic (sim-based)	Balanced	Requires similarity

4. Behavioral Cloning Steering: Mechanism and Theoretical Interpretation

In vanilla diffusion BC, actions are sampled directly from $d_a$9, a procedure that can result in low-fidelity outliers. Self-guidance introduces a form of “negative-score” steering, biasing sampling toward actions adapting to the most current state and away from stale or inconsistent modes.

The revised guidance achieves a Taylor-like blend:

$$q(A_k|A_{k-1}) = \mathcal{N}(A_k; \sqrt{1-\beta_k}A_{k-1}, \beta_k I),$$0

implying an implicit interpolation between current and extrapolated future trajectories (when $$q(A_k|A_{k-1}) = \mathcal{N}(A_k; \sqrt{1-\beta_k}A_{k-1}, \beta_k I),$$1).

Key advantages emerge:

• Improved action fidelity, with a reduction in “bad” actions (low-probability under $$q(A_k|A_{k-1}) = \mathcal{N}(A_k; \sqrt{1-\beta_k}A_{k-1}, \beta_k I),$$2).
• Enhanced proactiveness and future-aware behavior without external models or reward signals.
• All improvements accrue at inference, needing only a second forward pass.

5. Empirical Results and Performance Benchmarks

Extensive experiments demonstrate the merits of the combined self-guidance (SG) and adaptive chunking (AC) strategies. Six simulated manipulation tasks (Push-T, Square, Lift, Can, Transport, Kitchen) are tested under both stochastic (h=1, with temporally correlated action noise) and static (h=H/2, no noise) regimes.

• Average gain over vanilla Diffusion Policy: +23.25% success rate.
• Gain over Bidirectional Test-time Search (BID): +12.27% with 16× reduced compute.
• On Push-T (static), success: DP=0.667, SG+AC=0.817.
• Robustness: SG+AC exhibits graceful degradation under noise, consistently outperforming all baselines across noise levels $$q(A_k|A_{k-1}) = \mathcal{N}(A_k; \sqrt{1-\beta_k}A_{k-1}, \beta_k I),$$3.

In real-world SO-100 arm pick-and-place tasks using a 3-camera setup and 30 Hz DDIM-10 solver:

• Vanilla DP yields 40–50% success rate (20 trials).
• SG+AC achieves 70–80% success under both static and moving-target conditions.

These empirical results validate the practical effectiveness of DSBC techniques for improving both reliability (action fidelity) and reactivity (state adaptation), without requiring retraining or additional reward mechanisms (So et al., 14 Oct 2025).

6. Significance and Context within the Diffusion-Based RL Paradigm

DSBC addresses two core challenges in diffusion policy BC: the stochasticity-induced sampling errors and the temporal lag of open-loop execution. By introducing self-guidance leveraging past observations, it eliminates the need for external reward or classifier signals, while adaptive chunking provides a simple, similarity-driven criterion for planning updates.

This suggests DSBC may serve as a model for principled, low-overhead policy steering across a broad range of robotic learning environments. The methodology generalizes to other domains where rapid adaptation and high-fidelity action generation are critical, and highlights the potential of inference-time guidance and chunked planning in diffusion-action models.