---
title: Action-Guided Diffusion Policy (DP-AG)
url: https://www.emergentmind.com/topics/action-guided-diffusion-policy-dp-ag
type: topic
---

# Action-Guided Diffusion Policy (DP-AG)

Action-Guided Diffusion Policy (DP-AG) most specifically denotes a diffusion-policy imitation learner in which latent perception is updated during action denoising, so that action generation and perceptual representation form a closed perception–action loop rather than a one-way conditioning pipeline. In the formulation introduced in "Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies," DP-AG augments standard Diffusion Policy (DP) with variational latent encoding, an action-guided stochastic differential equation in latent space, and a cycle-consistent contrastive objective, with the stated goal of improving continuity, adaptability, and robustness in partially observable or dynamic manipulation settings [2509.25822]. In adjacent literature, however, closely related guidance-based methods use the same acronym or analogous language for different mechanisms, including bidirectional decoding for cross-chunk coherence, self-guidance from prior action chunks, affordance guidance, policy guidance, advantage guidance, and skill guidance; the term is therefore context-dependent rather than fully standardized [2508.12189].

## 1. Conceptual position within diffusion-policy research

Standard behavioral cloning maps $o_t \mapsto a_t$ directly, and standard Diffusion Policy models the conditional action distribution $p_\theta(a_t \mid z_t)$ by iteratively denoising action noise while keeping the observation feature $z_t=f_\psi(o_t)$ fixed over the whole action-generation horizon. DP-AG retains the diffusion-policy machinery but rejects the assumption that perception should remain static during denoising. Its central claim is that intermediate action predictions should feed back into latent observation updates, so that the model continuously reinterprets the same observation in light of the action it is currently generating [2509.25822].

This design is motivated by the perception–action gap in ordinary imitation learning and by active-perception and sensorimotor viewpoints. In the paper’s high-level contrast, DP smooths actions by denoising while perception remains fixed; DP-AG smooths actions and refines latent perception through action-guided latent dynamics. The significance of that shift is greatest in ambiguous, dynamic, or partially observable regimes, where a fixed latent can yield brittle or discontinuous behavior [2509.25822].

A recurrent source of confusion is that “action-guided diffusion policy” is used in more than one sense in the 2025–2026 literature. In Self-GAD, for example, “Action-Guided Diffusion Policy / bidirectional decoding (DP-AG)” refers to multi-sample coherence maintenance across action chunks in closed-loop control, not to latent perceptual evolution. This terminological overlap reflects a broader family resemblance—guidance is injected into diffusion-based control—but not a single canonical mechanism [2508.12189].

## 2. Architectural formulation and latent dynamics

The DP-AG pipeline comprises five parts: perception, variational latent encoding, action diffusion, VJP-guided latent evolution, and cycle-consistent alignment. Perception is modeled with a variational posterior rather than a deterministic encoder,
$$
q_\phi(z_t \mid o_t)=\mathcal{N}(\mu_\phi(z_t), \sigma_\phi^2(z_t)),
$$
with reparameterized latent sample
$$
z_t=\mu_\phi(z_t)+\sigma_\phi(z_t)\odot \epsilon,\qquad \epsilon\sim\mathcal{N}(0,I).
$$
The paper interprets this as yielding uncertainty-aware perceptual features rather than a frozen deterministic backbone [2509.25822].

Action generation is then performed by a conditional diffusion denoiser over noisy action samples. The paper writes the denoising process as
$$
\hat{a}_t^{0}=\hat{a}_t^{K}-\sum_{k=1}^{K} g(k)\,\epsilon_\theta(\hat{a}_t^k,z_t,k),
$$
with the standard diffusion-policy denoising objective
$$
\mathcal{L}_{\text{DP}} = \mathbb{E}_{(o_t,a_t)\sim\mathcal D,\; k\sim \mathcal U(1,K)} \left[ \left\|\epsilon_\theta(a_t^k,f_\psi(o_t),k)-\epsilon\right\|_2^2 \right].
$$
Up to this point the model still resembles ordinary DP; the distinctive element is that the latent observation is not held constant while denoising proceeds [2509.25822].

The key addition is an action-guided latent stochastic differential equation,
$$
d\tilde z_t^{k} = \mathrm{VJP}(\hat a_t^k, z_t)\,dt + \sigma_\phi(z_t)\,dW_t,
$$
whose drift is computed from the diffusion model’s noise prediction through a Vector-Jacobian Product,
$$
\mathrm{VJP}(\hat a_t^k, z_t) = \left(\frac{\partial \epsilon_\theta(\hat a_t^k,z_t,k)}{\partial z_t}\right)^\top \epsilon_\theta(\hat a_t^k,z_t,k).
$$
The paper also gives a reparameterized latent update,
$$
\tilde z_t^{k} = \mu_\phi(z_t)+\gamma\,\sigma_\phi(z_t)\odot \mathrm{VJP}(\hat a_t^k,z_t),
$$
where $\gamma$ controls the strength of action-guided latent drift. The stated intuition is that the denoiser’s sensitivity to latent perception becomes a structured force that moves the latent toward action-relevant features; the model effectively “looks again” at the observation through the lens of the action currently being denoised [2509.25822].

## 3. Training objective, variational interpretation, and continuity results

VJP-guided latent motion alone does not guarantee that the refined latent remains aligned with either the original observation or the denoising trajectory. DP-AG therefore introduces a cycle-consistent contrastive loss defined on pairs of noise predictions from the static latent $z_t$ and the evolved latent $\tilde z_t^k$:
$$
\varepsilon_k = \epsilon_\theta(\hat a_t^k,z_t,k), \qquad \tilde\varepsilon_k = \epsilon_\theta(\hat a_t^k,\tilde z_t^k,k).
$$
The paper uses an InfoNCE objective,
$$
\mathcal{L}_{\text{cont}} = -\frac{1}{B}\sum_{i=1}^{B} \log \frac{ \exp\left(\mathrm{sim}(\varepsilon_k^i,\tilde\varepsilon_k^i)/\tau\right) }{ \sum_{j\neq i}\exp\left(\mathrm{sim}(\varepsilon_k^i,\tilde\varepsilon_k^j)/\tau\right) },
$$
with $\mathrm{sim}(u,v)=\frac{u^\top v}{\|u\|\,\|v\|}$ and temperature $\tau$. This creates a bidirectional consistency loop: static latent $\rightarrow$ noise prediction, noise prediction $\rightarrow$ latent update, updated latent $\rightarrow$ refined noise prediction [2509.25822].

The total training objective is
$$
\mathcal{L}_{\text{DP-AG}} = \mathcal{L}_{\text{DP}} + \lambda_{\text{cont}}\mathcal{L}_{\text{cont}} + \lambda_{\text{KL}}\mathcal{L}_{\text{KL}}.
$$
Its KL term arises from a variational treatment of the evolved latent. The paper defines a posterior over evolved latents $q_\phi(\tilde z_t^k\mid z_t,\hat a_t^k)$, a Gaussian prior $p(\tilde z_t^k\mid z_t)=\mathcal N(z_t,I)$, and derives an ELBO for $\log p(\varepsilon_k\mid z_t)$ via Jensen’s inequality. The KL regularizer keeps the evolving latent anchored to the original observation while still allowing action-driven adaptation [2509.25822].

The paper makes two explicit theoretical claims. First, the action-guided latent model is variationally principled through the ELBO above. Second, the contrastive objective implies continuity: a sufficiently small InfoNCE loss yields a lower bound on positive-pair similarity, and under an $L$-Lipschitz assumption on $\epsilon_\theta$ in $z$, this induces a bound on latent-trajectory change. The intended consequence is that smoother latent updates produce smoother action trajectories. This moves the contrastive term beyond heuristic alignment and situates it as a continuity-inducing constraint on the coupled perception–action dynamics [2509.25822].

## 4. Empirical profile and benchmark behavior

The reported empirical picture has three layers: a synthetic continuity test, simulation benchmarks, and real-world UR5 manipulation. On an irregular spiral regression dataset, adding VJP-guided latent evolution reduces MSE from **0.0095 to 0.0052**, a **45.3% improvement**, and produces much smoother latent trajectories. In simulation, DP-AG is evaluated on Robomimic tasks including Lift, Can, Square, Transport, and Tool Hang, on Franka Kitchen, and on Push-T in both image and keypoint variants, including a Dynamic Push-T setting with a moving disturbance ball [2509.25822].

On Push-T, the paper reports target coverage of **0.87 img / 0.95 kp** for DP and **0.93 img / 0.99 kp** for DP-AG. On Dynamic Push-T, DP is reported at **0.65** and DP-AG at **0.80**, summarized in the paper as about **6% on Push-T** and **13% on Dynamic Push-T**. Across Robomimic and Franka Kitchen, DP-AG is described as achieving near-perfect success on most tasks and generally matching or exceeding DP, with the clearest gains on harder tasks and dynamic conditions [2509.25822].

Real-world evaluation uses three UR5 manipulation tasks: Painting, Candy Push, and Peg-in-Hole. Compared to DP, DP-AG improves Painting by reducing jerk from **0.083 to 0.032** and increasing IoU from **68.9% to 92.1%**. On Candy Push, success rises from **65.0% to 90.0%**, while jerk drops from **0.107 to 0.039**. On Peg-in-Hole, DP fails completely, whereas DP-AG reaches **85.0% success** with much smoother motion. The Peg-in-Hole result is emphasized as a case where blocked motions trigger latent updates that help infer the correct 3D alignment from 2D RGB cues [2509.25822].

Ablation results localize the contribution of the coupling terms. Removing the contrastive loss drops Push-T success from **0.93 to 0.85** and slows convergence from about **100 epochs to 300**. The strength of the VJP term matters: if it is too weak, the model reverts toward standard DP; if it is too strong, training destabilizes. The KL weight also matters, with small KL working best, while moderate/high values around **0.8** for temperature $\tau$ work best in the reported setup. Efficiency overhead is modest: on Push-T, per-epoch time increases from **114.2 s** to **119.5 s**, inference latency is essentially unchanged, and convergence improves from about **200 epochs** to **100 epochs** [2509.25822].

## 5. Relation to other guidance-based diffusion policies

The surrounding literature uses guidance in several different ways, and these variants help clarify what is distinctive about DP-AG proper.

| Method | Guidance signal | Relation to DP-AG |
|---|---|---|
| Self-GAD | prior decision | guides the proposal distribution at each diffusion step |
| AffordDP | transferable affordance | adds affordance guidance during diffusion sampling |
| PGD | target policy guidance | guides synthetic trajectories toward more on-policy data |
| AGD-MBRL | advantage estimates | reweights sampling with $\sigma(A)$ or $\exp(A)$ |
| CFG-DP | timestep input and CFG | biases generation toward termination actions |
| SDP | primitive skill | constructs a single-skill policy to generate skill-aligned actions |
| D3P | state-aware adaptor | changes denoising computation per action |

The core distinction is mechanistic. DP-AG in the strict sense updates latent perception during denoising via a VJP-guided SDE and binds that update to the action denoiser through a cycle-consistent contrastive loss. Self-GAD pursues the different objective of cross-chunk coherence in closed-loop action chunking: it is described as a more efficient alternative to Action-Guided Diffusion Policy / bidirectional decoding, and it guides the proposal distribution at each diffusion step based on the prior decision rather than evolving perceptual latents [2508.12189].

AffordDP moves guidance into a differentiable affordance loss. It conditions on a static affordance contact point and a dynamic post-contact trajectory, then adds a gradient correction toward the transferred static affordance during DDIM sampling, with guidance active only when the gripper is close enough to the target contact point. The paper presents this as a way to generalize to unseen instances and categories while keeping generated actions within the learned action manifold [2412.03142].

In reinforcement learning, Policy-Guided Diffusion uses target-policy action gradients to steer full-trajectory diffusion toward more on-policy synthetic offline data, while AGD-MBRL replaces policy-only or reward-only steering with advantage-based guidance and proves a reweighting interpretation through Sigmoid Advantage Guidance (SAG) and Exponential Advantage Guidance (EAG). Both methods guide reverse diffusion during sampling without changing the diffusion training objective, but their domain is trajectory generation for RL rather than visuomotor imitation in manipulation [2404.06356] [2604.09035].

Other variants alter the meaning of guidance again. CFG-DP combines conditional and unconditional denoisers with a timestep-dependent classifier-free guidance factor to improve temporal progression and termination in a humanoid screwing task; SDP converts observation and language into a discrete primitive skill and conditions a single-skill diffusion policy on that skill; D3P learns a state-aware adaptor that allocates denoising steps per action so that crucial actions receive more computation than routine ones; and GDNB uses Feynman–Kac correctors with a rare-shell guiding potential to mine frontier trajectories, repair them with trajectory optimization, and retrain the diffusion policy on the repaired data [2510.09786] [2601.01948] [2508.06804] [2606.08743].

A common misconception is therefore that “action guidance” always means post-hoc action reranking or classifier-like steering of the reverse sampler. The literature shows at least four distinct loci of guidance: latent-state evolution, sampling-time action correction, temporal-coherence shaping, and computation-allocation control. DP-AG in the narrow sense refers to the first of these.

## 6. Limitations, ambiguities, and open directions

The limitations of DP-AG itself are explicit. It still relies on demonstration data, so bias or suboptimality in demonstrations can transfer to the policy. Most of the main evaluation is offline imitation learning, even though online results are also provided. VJP-based training adds complexity relative to plain DP. The paper identifies future work on sim-to-real transfer and broader integration with world models, and the appendix suggests that the perception–action feedback idea could be combined with flow matching and online learning rather than remaining tied to DDPM-style policies [2509.25822].

Broader guidance-based diffusion work identifies additional fault lines. Self-GAD is motivated by the computational expense of repeated multi-sample search for cross-chunk coherence; D3P is motivated by the cost of allocating the same denoising budget to every action; CFG-DP reports sensitivity to timestep estimation and manual tuning of $\lambda_{\text{max}}$; AffordDP depends on the quality of transferred affordances, semantic correspondence, and registration; AGD-MBRL depends on critic accuracy because guidance is only as reliable as the estimated advantage; and GDNB notes uncertainty about long-run behavior over many rounds, along with hyperparameter sensitivity in both its guidance potential and trajectory-repair stage [2508.12189] [2508.06804] [2510.09786] [2412.03142] [2604.09035] [2606.08743].

A second limitation is nomenclatural. The acronym DP-AG is used most rigorously for the VJP-guided perception–action loop of "Act to See, See to Act," but other papers use “Action-Guided Diffusion Policy / bidirectional decoding (DP-AG)” for cross-chunk coherence search or use the term more loosely for any diffusion policy whose reverse process is shaped by an auxiliary guidance signal. This suggests that the field has converged more strongly on the family of ideas than on a single stable name [2508.12189].

Taken together, the literature indicates that action-guided diffusion is best understood as a design pattern for modifying diffusion-based control with structured feedback during generation. Within that pattern, DP-AG proper occupies a distinctive position: it treats perception as a dynamic latent state rather than a fixed conditioning vector, and it uses the action denoiser’s own gradients to update that latent in a cycle-consistent way. The surrounding variants show that the same broader principle can also be applied to coherence maintenance, affordance transfer, temporal termination, offline RL data generation, model-based planning, skill abstraction, computational efficiency, and rare-behavior discovery [2509.25822].

Source: https://www.emergentmind.com/topics/action-guided-diffusion-policy-dp-ag