---
title: 'DynaGuide: Steering Diffusion Policies'
url: https://www.emergentmind.com/topics/dynaguide
type: topic
---

# DynaGuide: Steering Diffusion Policies

Searching arXiv for the specified paper and closely related uses of the name "DynaGuide".
arxiv_search(query="2506.13922 DynaGuide Steering Diffusion Policies with Active Dynamic Guidance", max_results=5)
Searching arXiv now.
{"query":"2506.13922 DynaGuide Steering Diffusion Policies with Active Dynamic Guidance","max_results":5}
DynaGuide is a steering method for diffusion policies that uses guidance from an external dynamics model during the diffusion denoising process, leaving the base diffusion policy unchanged and introducing a decoupling between the policy’s action prior and the steering objective [2506.13922]. In the formulation reported for robot manipulation, DynaGuide is designed for settings in which large visuomotor policies have already acquired rich skills from human or self-supervised play data, but deployment requires inference-time control over outcomes such as opening rather than closing a drawer, or preferring one object over another, without retraining or fine-tuning the entire policy. The method is presented as a modular approach that can steer off-the-shelf pretrained diffusion policies, support arbitrarily many positive and negative objectives, enhance underrepresented behaviors, and remain robust when objectives are novel, noisy, or low-quality [2506.13922].

## 1. Problem setting and conceptual basis

The immediate problem addressed by DynaGuide is the mismatch between the breadth of behaviors represented in a pretrained diffusion policy and the narrower, situational objectives that arise at deployment time. Common steering approaches, such as goal-conditioning, require the robot policy to be trained with a distribution of test-time objectives in mind, and they typically admit only a single goal vector or image as input [2506.13922]. In the reported motivation, this makes such approaches brittle when inference-time goals are new, novel, or low-quality.

DynaGuide takes a different approach: it leaves the base diffusion policy unchanged and injects a gradient “push” from a separately trained latent dynamics model during each denoising step [2506.13922]. The dynamics model encodes long-horizon visual outcome objectives rather than the action prior itself. The paper characterizes this as a decoupling between “what the robot can do” and “what we want it to do,” with the former residing in the base policy and the latter in the external guidance model [2506.13922].

This separation is central to the method’s stated properties. Because the guidance signal is external, DynaGuide can be applied to off-the-shelf pretrained diffusion policies, can fuse multiple desired and undesired outcomes, and can amplify behaviors that are rare in the base policy’s training distribution [2506.13922]. A plausible implication is that the method treats steering as an inference-time control layer rather than as a property that must be embedded into the policy during training.

## 2. Inference algorithm and mathematical formulation

DynaGuide operates during diffusion-policy inference. Its inputs are the current observation $o_t$, a positive guidance set $\mathcal G^+$ of desired outcome images, a negative guidance set $\mathcal G^-$ of undesired outcome images, a pretrained diffusion policy $\epsilon_p(a^k,o_t)$, a latent dynamics model $h_\theta(\phi(o_t),a)$, an image embedder $\phi$, a guidance strength $s$, a variance $\sigma$, and a number of stochastic refinements $M$ [2506.13922].

The procedure initializes $a^K \sim \mathcal N(0,I)$ and iterates from diffusion step $K$ down to $1$. At each step, the policy predicts noise, the guidance metric is evaluated from the dynamics model’s far-future latent prediction, the gradient of that metric with respect to the current noisy action is backpropagated, and the resulting gradient is combined with the policy’s denoising signal under DDIM guidance [2506.13922]. The denoising update is

$$
\hat \epsilon(a^k,o_t) = \epsilon_p(a^k,o_t) - s\sqrt{1-\bar\alpha_k}\,\nabla_{a^k} d,
$$

where $\bar\alpha_k=\prod_{l=1}^k \alpha_l$ is the DDIM noise schedule [2506.13922].

The guidance metric itself is defined through latent distances between predicted future outcomes and the embeddings of positive and negative guidance images. Let $\hat z_{t+H}=h_\theta(\phi(o_t),a)$ and let $z_i=\phi(g)$ for each guidance image $g$. Then

$$
d(a;o_t,\mathcal G^+,\mathcal G^-)
=
\log \sum_{i\in\mathcal G^+} \exp\bigl[-\|z_i-\hat z_{t+H}\|_2^2/\sigma\bigr]
-
\log \sum_{j\in\mathcal G^-} \exp\bigl[-\|z_j-\hat z_{t+H}\|_2^2/\sigma\bigr].
$$

This log-sum-exp construction is the basis for multi-objective steering, because it naturally fuses any number of positive and negative conditions [2506.13922].

The dynamics model is trained separately with the objective

$$
L(\theta)=\mathbb E_{(o_t,o_{t+H},a)} \|\phi(o_{t+H})-h_\theta(\phi(o_t),a)\|_2^2.
$$

The reported interpretation is that DynaGuide leverages classifier-style diffusion guidance, but replaces the classifier with a latent dynamics model that predicts far-future outcomes [2506.13922].

## 3. Decoupled guidance and steering capabilities

The decoupling of the dynamics model from the base policy is the defining design choice of DynaGuide. The paper identifies four consequences of this separation: modular steering of any pretrained diffusion policy at inference time, multi-objective steering through the log-sum-exp metric, robustness to mismatched or noisy objectives, and improved access to rare behaviors [2506.13922].

For multi-objective steering, DynaGuide can mix several desired $g^+$ and undesired $g^-$ conditions, such as “press button OR open drawer AND avoid closing” [2506.13922]. Because the objectives are represented as outcome images in the latent space used by the dynamics model, the system is not restricted to a single goal vector or single goal image.

For robustness, the paper argues that guidance conditions $\phi(g)$ and the policy inputs remain in-distribution even if $g$ mismatches the real scene, so latent errors average out [2506.13922]. This is used to explain why DynaGuide remains effective when guidance images are underspecified or low-quality.

For rare or underrepresented behaviors, the paper contrasts diffusion guidance with sampling-based rankers. Diffusion guidance is described as sculpting the denoising path directly into low-probability regions, whereas sampling-based rankers need many draws from $\pi$ to hit rare modes [2506.13922]. This suggests that DynaGuide is not merely selecting among candidate actions already favored by the policy, but modifying the trajectory of denoising itself.

## 4. Experimental evaluation

The experimental program spans simulated CALVIN tasks, multi-objective steering, underrepresented-behavior tests, and real-robot experiments on an ARX5 platform with a UMI policy [2506.13922]. In simulation, success rate is defined as the fraction of rollouts in which the target behavior occurs within 400 steps.

| Setting | Main result | Context |
|---|---|---|
| ArticulatedParts | DynaGuide $\approx 70\%$ success vs. base $\approx 8\%$ | Outperforms sampling by $2\times$ and ITPS by $1.5\times$ |
| MovableObjects | DynaGuide $\approx 60\%$ vs. sampling $\approx 40\%$ | Goal-conditioning collapses to $<10\%$ on OOD goals |
| Underspecified guidance | DynaGuide retains $\approx 54\%$ | Goal-conditioning $<10\%$, a $5.4\times$ improvement |
| Multi-objective steering | DynaGuide obtains $\approx 80\%$ joint success | Nearly zero unwanted behaviors |
| Rare behavior regime | DynaGuide reaches $\approx 40\%$ | Base policy does Switch-On $<5\%$, sampling reaches $\approx 25\%$ |

The simulated 3D CALVIN tasks are divided into ArticulatedParts and MovableObjects. ArticulatedParts includes opening or closing a drawer, flipping a switch, and pressing a button; MovableObjects includes pushing red or blue cubes that are randomly placed [2506.13922]. Baselines are the base policy without steering, a goal-conditioned policy trained on final-image goals, a sampling-based ranking method denoted GPC that samples $\pi$ five times and picks the action maximizing $d$, and position guidance denoted ITPS for fixed objects [2506.13922].

On ArticulatedParts, DynaGuide reaches approximately $70\%$ success, compared with approximately $8\%$ for the base policy, while goal-conditioning reaches approximately $95\%$ in-distribution [2506.13922]. On MovableObjects, goal-conditioning collapses to below $10\%$ because of out-of-distribution goals, while DynaGuide stays at approximately $60\%$ and sampling remains at approximately $40\%$ [2506.13922]. When guidance images are underspecified by a random robot pose in $g^+$, goal-conditioning remains below $10\%$ while DynaGuide retains approximately $54\%$, yielding the reported $5.4\times$ improvement [2506.13922].

In multi-objective steering, DynaGuide obtains approximately $80\%$ joint success with nearly zero unwanted behaviors, whereas sampling loses approximately $30$ percentage points [2506.13922]. In the underrepresented-behavior experiment, $99\%$ of Switch-On examples are removed from the base policy’s training data. The base policy then performs Switch-On in less than $5\%$ of cases, sampling guidance rises to approximately $25\%$, and DynaGuide rises to approximately $40\%$ even in the $1\%$ data regime [2506.13922].

The real-robot experiments use an off-the-shelf cup-to-saucer diffusion policy. The dynamics model is trained on public UMI cup data together with custom two-cup demos and mouse demos [2506.13922]. In CupPreference, two cups are equidistant and the system is steered to choose red rather than grey over 20 trials. In HiddenCup, the red cup is occluded behind the grey cup, and DynaGuide picks the red cup $80\%$ of the time versus $15\%$ for the base policy. In NovelBehavior, steering toward touching a mouse, a behavior never seen by the base policy, doubles mouse contacts [2506.13922].

## 5. Ablations, sensitivity, and operational constraints

The paper reports several sensitivity analyses. Guidance strength $s$ that is too small produces no steering, whereas guidance that is too large produces incoherent trajectories; the reported optimal interval is $[1.2,1.8]$ [2506.13922]. For the variance $\sigma$ in the guidance metric, small $\sigma$ sharpens one mode and overfits, while large $\sigma$ blunts the signal; the typical range is $[20,40]$ [2506.13922].

The number of stochastic repeats also matters. The reported choice $M=4$ stabilizes high-$s$ guidance through MCMC-style resampling [2506.13922]. With respect to the number of conditions $|\mathcal G|$, the method is reported to remain robust even down to $1$–$5$ conditions, although more conditions produce a smoother log-sum-exp objective [2506.13922].

The most important ablation concerns the dynamics model’s training distribution. The paper states that the dynamics model must be pretrained with noised actions; otherwise, $\nabla_a d$ is meaningless on out-of-distribution $a^k$ and steering fails [2506.13922]. This constraint follows directly from the fact that DynaGuide evaluates and differentiates the guidance metric on noisy action samples encountered during denoising rather than on clean actions alone.

## 6. Naming, scope, and related uses of “DynaGuide”

The name “DynaGuide” is not unique to diffusion-policy steering. In robotics motion planning, “DynaGuide” denotes a guided sampling-based kinodynamic planner for navigation in unknown or partially observed environments [2306.09229]. In reinforcement learning, Guided Dyna-Q is also referred to as DynaGuide and integrates automated planning into Dyna-Q to focus exploration on plannable transitions [2004.11456]. In text-to-image diffusion, “DynaGuide” denotes a framework for dynamic classifier-free diffusion guidance via online feedback [2509.16131]. In computer vision, “DynaGuide” names a dual-guidance framework for unsupervised semantic segmentation that combines global pseudo-labels with local boundary refinement [2602.13020].

These works are unrelated in method and application domain. A plausible implication is that citation by arXiv identifier is necessary for disambiguation whenever the term appears in bibliographies, search systems, or cross-domain discussion.

Within the diffusion-policy setting, DynaGuide is defined by three interlocking features: classifier-style guidance during denoising, a separately trained latent dynamics model that predicts far-future outcomes, and inference-time steering without fine-tuning the base policy [2506.13922]. In that sense, its contribution is not only a steering heuristic but a reorganization of policy control into a modular relation between action prior and outcome preference.

Source: https://www.emergentmind.com/topics/dynaguide