Continuous Diffusion for Actions

• Continuous diffusion for actions is a framework that uses iterative noise injection and learned reverse denoising processes to model multimodal continuous control actions.
• It leverages mathematical formulations like DDPMs and continuous-time SDEs to refine actions for robotics, video action segmentation, and hybrid planning.
• Empirical results show improved shared autonomy, high-dimensional manipulation, and real-time performance, demonstrating its practical effectiveness.

Continuous diffusion for actions denotes a class of methods that model the generation, refinement, and sampling of continuous control actions via denoising diffusion probabilistic models (DDPMs) or their continuous-time SDE/ODE equivalents. This approach leverages iterative noise injection and denoising to model highly multimodal continuous distributions over actions, trajectories, or action sequences, with applications ranging from shared autonomy and real-time robot control to video action segmentation and hybrid symbolic-continuous planning.

1. Mathematical Foundation of Continuous Diffusion for Actions

Given a continuous action space $\mathcal{A}\subseteq\mathbb{R}^d$, and demonstration data $D=\{(s_i, a_i)\}$ or sequences $\xi: [0,1] \to \mathcal{A}$, continuous diffusion models construct a Markov chain or SDE that iteratively corrupts an action or action trajectory with noise, and a learned reverse process (often parameterized as a neural network) that denoises toward likely modes under the data distribution.

Forward (Noising) Process

The canonical discrete-time DDPM formulation for actions is

$$q(a_k \mid a_{k-1}) = \mathcal{N}\left(a_k; \sqrt{\alpha_k} a_{k-1}, (1-\alpha_k) I \right), \quad k = 1,...,K$$

with cumulative $\bar\alpha_k = \prod_{i=1}^k \alpha_i$ and closed-form

$$q(a_k \mid a_0) = \mathcal{N}\left(a_k; \sqrt{\bar\alpha_k} a_0, (1-\bar\alpha_k) I\right)$$

for action $a_0$ sampled from demonstrations (Yoneda et al., 2023, Chi et al., 2023, Hou et al., 2024).

The continuous-time SDE view writes

$$\mathrm{d}a_t = f(t) a_t \,\mathrm{d}t + g(t)\, \mathrm{d}W_t$$

with drift/variance schedules ensuring $a_t$ becomes isotropic Gaussian as $t\!\to\!1$ (Abyaneh et al., 2 Jan 2026, Zhao et al., 2024).

Reverse (Denoising) Process

The learned parameterization is

$$p_\theta(a_{k-1} \mid a_k, s) = \mathcal{N}(a_{k-1}; \mu_\theta(a_k, k, s), \sigma_k^2 I)$$

where

$$\mu_\theta(a_k, k, s) = \frac{1}{\sqrt{\alpha_k}} \left[a_k - \frac{1-\alpha_k}{\sqrt{1-\bar\alpha_k}} \epsilon_\theta(a_k, k, s)\right]$$

and $\epsilon_\theta$ is trained to predict the injected noise (Yoneda et al., 2023, Chi et al., 2023, Jiang et al., 13 May 2025).

The equivalent SDE/ODE for reverse-time generative sampling is

$$\mathrm{d}a_t = [f(t)a_t - \tfrac{1}{2} g(t)^2 \nabla_{a_t} \log p_t(a_t \mid s)] \mathrm{d}t$$

where the "score" $\nabla_{a_t} \log p_t(a_t\mid s)$ is learned via noise prediction or direct score matching (Abyaneh et al., 2 Jan 2026, Zhao et al., 2024).

This formalism underpins nearly all applications of continuous diffusion for actions.

2. Architectural and Algorithmic Instantiations

Action-Conditioned Denoisers

Architectures vary across deployments:

• MLPs with state concatenation and time embeddings (Yoneda et al., 2023, Jiang et al., 13 May 2025)
• CNNs or Transformers for high-dimensional action-sequences, allowing visual and language conditioning (Chi et al., 2023, Hou et al., 2024)
• Time-unified denoisers where time embedding is removed and a single velocity field is learned (Niu et al., 11 Jun 2025)
• Noise-relaying buffers for efficiency and responsiveness in streaming control (Chen et al., 18 Feb 2025)
• Scalar-valued potential networks for bottleneck-gradient representations enabling density calculation (Chen et al., 2024)

Sampling strategies include full reverse chains, DDIM/accelerated steps, Euler/Heun ODE integration, and flow-matching for streaming (Jiang et al., 28 May 2025).

Optimization and Fine-tuning

Optimization objectives follow the standard denoising score-matching loss

$$\mathcal{L}_\mathrm{simple} = \mathbb{E}_{a_0, t, \epsilon}\left\| \epsilon - \epsilon_\theta\left(\sqrt{\bar\alpha_t} a_0 + \sqrt{1-\bar\alpha_t}\ \epsilon,\, t,\, s\right)\right\|^2$$

with additional terms for regularization, KL to expert behavior, action discrimination, or Q-function alignment as needed (Jiang et al., 13 May 2025, Chen et al., 2024, Zhao et al., 3 Feb 2025, Abyaneh et al., 2 Jan 2026).

Fine-tuning can involve adaptive RL surrogates (e.g., ADPO), continuous-time RL (e.g., policy gradients for score-controls), or DPO-style preference alignment with Q-values (Chen et al., 2024, Zhao et al., 3 Feb 2025, Jiang et al., 13 May 2025).

3. Applications and Empirical Impact

Shared and Assisted Control

(Yoneda et al., 2023) demonstrates a principled means to interpolate control authority between user and agent through a forward-diffusion ratio $\gamma$. Small $\gamma$ values significantly improve task success and safety in shared autonomy, with rigorous bounds on authority allocation:

• $\gamma=0$: raw user action
• $\gamma=1$: full autonomous demonstration Empirically, assistance ($\gamma\approx0.2{-}0.4$) outperforms unassisted pilots while maintaining user intent.

High-Dimensional Visuomotor Policy Learning

(Chi et al., 2023, Hou et al., 2024) show that sequence-level (chunked) action diffusion, coupled with rich visual/language context, delivers SOTA results in manipulation, robust multimodal policy expressiveness, and stability in high-dimensional settings, outperforming both conventional BC and discrete action policies.

Real-Time and Streaming Control

Buffer-relaying (Chen et al., 18 Feb 2025) and streaming ODE/flow policy (Jiang et al., 28 May 2025) formulations circumvent costly trajectory-level denoising or mode-bouncing—each action can be sampled and executed as soon as denoising completes, matching or surpassing DP performance with order-of-magnitude latency improvements.

Temporal Action Segmentation and Prediction

Diffusion models (e.g., DiffAct (Liu et al., 2023) and DiffAnt (Zhong et al., 2023)) enable stochastic, multimodal forecasting of temporally extended action sequences in video—improving on deterministic and classical iterative-refinement methods particularly in ambiguous, long-horizon regimes.

Hybrid Planning and Compositionality

Joint discrete/continuous diffusion enables simultaneous symbolic and continuous planning, as shown in (Høeg et al., 26 Sep 2025), substantially increasing robustness and task generalization relative to pure trajectory diffusion or vanilla symbolic planners.

Policy Alignment and Regularization

Continuous diffusion policies support alignment to arbitrary Q-functions via tractable density representations, facilitating preference-based and RL fine-tuning akin to LLM alignment frameworks (Chen et al., 2024, Zhao et al., 3 Feb 2025). Contractive regularization (Abyaneh et al., 2 Jan 2026) ensures stability and robustness under data scarcity or solver mismatches.

4. Computational and Practical Considerations

• Sample Efficiency: Streaming and time-unified methods lower neural function evaluations per action, enabling real-time use (NFEs/a ≈ 1) (Chen et al., 18 Feb 2025, Jiang et al., 28 May 2025, Niu et al., 11 Jun 2025).
• Inference Complexity: Time-unified velocity fields (Niu et al., 11 Jun 2025), streaming-flow policies (Jiang et al., 28 May 2025), and contractive regulation (Abyaneh et al., 2 Jan 2026) allow for drastically reduced denoising steps without significant accuracy loss.
• Multimodal Expressiveness: All diffusion-based policies can represent complex, multimodal behavior, with empirical demonstrations on diverse robotic and video datasets (Chi et al., 2023, Zhong et al., 2023, Shi et al., 2024).
• Architectural Compatibility: Existing MLP, CNN, transformer, and flow-matching architectures are directly extensible to diffusion-action paradigms with minimal changes (Chi et al., 2023, Hou et al., 2024, Jiang et al., 28 May 2025).
• Density Calculation: Bottleneck-gradient representation provides direct access to action densities, enabling direct likelihood/regret optimization in policy alignment (Chen et al., 2024).

5. Limitations and Extensions

• Marginal vs. Joint Distributions: Streaming approaches guarantee per-time marginal fidelity but not coherence over entire trajectory segments, which can induce unanticipated compositional behaviors (Jiang et al., 28 May 2025).
• Responsiveness vs. Consistency: Monolithic diffusion rollouts ensure high long-horizon consistency but may lack real-time reactivity; noise-relaying and streaming strategies directly address this for sensorimotor loops (Chen et al., 18 Feb 2025).
• Global Constraints: Diffusion-CCSP (Yang et al., 2023) shows that factor-graph composition of constraint-specific diffusion models yields scalable generalization but can require multiple sampling/backtracking cycles for hard global constraints.
• Fine-Tuning Dynamics: KL or contractive penalties are essential in RL fine-tuning to avoid deviation from pretrained score fields and preserve generative validity (Abyaneh et al., 2 Jan 2026, Zhao et al., 3 Feb 2025, Zhao et al., 2024).
• Data Dependence: Robustness to limited demonstrations is enhanced by contractive regularization and time-unification, but naive diffusion policies degrade sharply in sparse regimes (Abyaneh et al., 2 Jan 2026, Niu et al., 11 Jun 2025).

6. Representative Empirical Results

Application Domain	Notable Result	Source
Shared Autonomy	+25% success with $\gamma=0.2{-}0.4$	(Yoneda et al., 2023)
Robotic Manipulation	83.8% RLBench single-view SOTA	(Niu et al., 11 Jun 2025)
Latency (Push-T, RoboMimic)	3.5–4.5 ms per action, SFP/Buffer	(Jiang et al., 28 May 2025, Chen et al., 18 Feb 2025)
Action Segmentation (50Salads)	F1@10=90.1, +1.2pp over SOTA	(Liu et al., 2023)
Offline RL (D4RL)	83.7 average (EDA, full-data)	(Chen et al., 2024)
Contractive DP (D4RL)	65.7 avg return vs 61.2 baseline	(Abyaneh et al., 2 Jan 2026)

Across these methods, continuous diffusion for actions provides a unified, highly expressive, and principled modeling framework that achieves or exceeds state-of-the-art accuracy in diverse continuous control, video understanding, and planning tasks.

7. Theoretical Insights and Control-Theoretic Extensions

Continuous diffusion for actions admits interpretation within mathematical control theory:

• Score as Control/Policy: The learned score function $s_\theta(t, a)$ can be interpreted as a continuous control or action for the diffusion SDE, enabling the application of policy gradient, HJB PDEs, and RLHF-style optimization (Zhao et al., 2024, Zhao et al., 3 Feb 2025).
• Contractivity: Enforcing negativity of the symmetric part of the Jacobian of the score field guarantees exponential stability of the generative process, critical for robustness in control regimes (Abyaneh et al., 2 Jan 2026).
• Hybridization with Discrete Planning: Coupled discrete-continuous diffusion models enable combinatorial symbolic-concrete search that cannot be replicated by either approach alone (Høeg et al., 26 Sep 2025).
• Reinforcement Learning Integration: Continuous-time RL formulations allow the seamless integration of reward models, Q-alignment, and regularization in both policy optimization and value estimation, with sound theoretical convergence and monotonicity guarantees (Zhao et al., 2024, Zhao et al., 3 Feb 2025, Chen et al., 2024).

These perspectives not only illuminate the empirical strengths of diffusion-action models but also provide a rigorous analytical basis for future extensions—including sample-efficient fine-tuning, real-time adaptive control, and compositionally-aware planning over joint symbolic-continuous domains.