Scalable Diffusion Policy

Updated 26 May 2026

Scalable Diffusion Policy is a class of algorithms that models high-dimensional, multimodal action distributions using denoising diffusion probabilistic models for robotics and control.
Architectural innovations such as Transformer-based models, noncausal attention, and implicit scene supervision enable stable training and substantial performance gains in tasks like single-robot manipulation and multi-agent coordination.
Empirical results demonstrate that scaling model parameters and training data leads to significant improvements, with success rates rising from 78% to over 86% and real-time inference maintained even for billion-parameter models.

Scalable diffusion policy refers to a class of action-generation algorithms in robotics and control that utilize denoising diffusion probabilistic models (DDPMs), or their variants, to model complex, high-dimensional, and multimodal action distributions. Scalability in this context denotes both the ability of the architecture and algorithm to leverage large datasets and models—yielding monotonic performance improvements with scale—and to maintain real-time inference and robust training as task, data, and model complexity increase. Scalable diffusion policies have demonstrated state-of-the-art results in single-robot manipulation, multi-agent coordination, language-driven control, and sim-to-real transfer, with practical implementations supporting models up to a billion parameters and training regimes encompassing thousands of trajectories or parallel environments (Xia et al., 17 Dec 2025, Zhu et al., 2024, Zhang et al., 21 May 2026, Vatnsdal et al., 21 Sep 2025, Ma et al., 1 Feb 2025).

1. Foundations of Diffusion Policy and Scalability Challenges

Diffusion policy represents an agent's action (or action sequence) distribution via a stochastic process: the policy maps noise (e.g., Gaussian) to actions through an iterative denoising chain, with the entire process conditioned on sensory observations such as images, point clouds, or proprioceptive states (Chi et al., 2023, Xia et al., 17 Dec 2025, Zhu et al., 2024). The forward (noising) and reverse (denoising/policy) chains are defined as: $q(x_t|x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I)$

$p_\theta(x_{t-1}|x_t, c) = \mathcal{N}(x_{t-1}; \mu_\theta(x_t, t, c), \Sigma_\theta(x_t, t, c))$

Scalability bottlenecks appear in training and inference as model size, action dimensionality, horizon length, and demonstration datasets increase. Issues include gradient instability, slow convergence, compounding action prediction error, and high inference latency. Early diffusion policies scaled to tens of millions of parameters before hitting optimization pathologies, such as gradient-norm explosion with deeper Transformers (Zhu et al., 2024). Standard approaches also performed poorly when deployed in domains requiring precise spatial reasoning or physical-dynamics understanding due to weak geometric supervision (Xia et al., 17 Dec 2025).

2. Architectural Innovations Enabling Scalability

Several lines of research have unlocked practical scaling for diffusion policies:

Transformer-based Models: DiT (Diffusion Transformer) and variants enable larger effective capacity by leveraging self-attention, cross-attention to observations, and receding-horizon sequence prediction. However, vanilla causal-masked Transformers suffered from gradient imbalance at depth, which was addressed via adaptive per-block normalization (AdaLN) in ScaleDP (Zhu et al., 2024) and blockwise affine observation embeddings in ISS Policy (Xia et al., 17 Dec 2025).
Noncausal (Bidirectional) Attention: Removing strict autoregressive masking within each sampled action chunk allows all tokens to attend bidirectionally during prediction, reducing compounding errors and improving multimodal output fidelity (Zhu et al., 2024).
Full Condition Modulation: FiLM- or AdaLN-style per-block modulation (scaling and shifting) with observation and timestep embeddings improves optimization stability and enables deep architectures to remain trainable at large scale (Zhu et al., 2024, Xia et al., 17 Dec 2025, Wang et al., 13 Feb 2025).
Implicit Scene Supervision Modules: Auxiliary networks enforce action-to-scene geometric consistency by predicting future point cloud embeddings from predicted actions and penalizing deviation from the observed geometry, providing dense 3D signal for better generalization and faster convergence (Xia et al., 17 Dec 2025).
Multistream Attention for Multimodal Inputs: In language-driven and multimodal settings (e.g., SCRIPT (Zhang et al., 21 May 2026)), joint token-level attention across actions, states, and language improves alignment and semantic downstream controllability.

Summary Table: Core Design Principles for Scalable Diffusion Policy

Principle	Function	Representative Work
AdaLN/FiLM modulation	Gradient stability at depth	ScaleDP, ISS Policy
Noncausal attention	Reduced compounding errors, multimodality	ScaleDP, SCRIPT
Geometric supervision	Long-horizon spatial consistency	ISS Policy
Multistream transformers	Joint semantic-control coupling	SCRIPT

3. Training Algorithms and Objectives

Training objectives for scalable diffusion policy split into supervised imitation (behavior cloning) and reinforcement learning paradigms.

Supervised Diffusion Loss: Most approaches minimize mean-squared error between predicted noise and injected noise under the perturbed trajectory, optionally combined with geometric or task-aware auxiliary losses: $\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_{t, x_0, \epsilon}\left[\|\epsilon - \epsilon_\theta(\sqrt{\bar{\alpha}_t}x_0 + \sqrt{1-\bar{\alpha}_t}\epsilon, t, c)\|_2^2\right]$ with full loss incorporating implicit scene supervision: $\mathcal{L}(\theta, \psi) = \mathcal{L}_{\mathrm{diff}}(\theta) + \lambda\,\mathcal{L}_{\mathrm{iss}}(\theta, \psi)$ (Xia et al., 17 Dec 2025).
Auxiliary Scheduled Sampling: To bridge training and inference distribution gaps, scheduled sampling gradually shifts the ISS module's conditioning from ground-truth to model predictions (Xia et al., 17 Dec 2025).
Imitation + RL Hybridization: Several methods freeze the diffusion backbone and train a lightweight RL controller to steer the initial noise (e.g., ExpertGen), enabling reward optimization over a manifold constrained by expert-like motion (Xu et al., 16 Mar 2026).
On-policy and Online RL Extensions: For sample-efficient and scalable online RL, reweighted score-matching (RSM) and conditional PPO (DP-CPPO) break the O(K) log-likelihood bottleneck by "aligning" PPO steps with single diffusion denoising kernels, or by reweighting score loss via Q-values (DPMD, SDAC). This allows scaling to hundreds of thousands of steps and millions of parameters without sample complexity blowup (Liu et al., 5 Mar 2026, Ma et al., 1 Feb 2025, Choi et al., 5 Jan 2026).

4. Empirical Analysis: Scalability with Model Capacity, Data, and Task Complexity

Quantitative evaluations on robotic manipulation and coordination benchmarks demonstrate consistent scaling trends:

Model Size Scaling: On MetaWorld (single-arm and dexterous tasks), DiT size was scaled from 45M to 215M parameters, yielding monotonic gains in average SR₅ (top-5 success rate) from 78.1% (small) to 86.2% (large) (Xia et al., 17 Dec 2025). In real-world stacking, ScaleDP with 1B parameters yielded up to a 36pp absolute improvement on single-arm and 75pp on bimanual tasks relative to 10M-parameter DP-T baselines (Zhu et al., 2024).
Data Scaling: ISS Policy improves from 69.3% (10 demos) to 83.7% (50 demos); this scaling trend holds in both simulation (Adroit/MetaWorld) and real-robot settings (Xia et al., 17 Dec 2025).
Sample and Inference Efficiency: With DDIM acceleration, denoising steps at inference can be reduced by up to 80% (e.g., from 100 to 10 steps), maintaining <10% success drop and reducing latency to ~50 ms for large models (Xia et al., 17 Dec 2025, Zhu et al., 2024). D3P achieves a 2.2× speedup over fixed-step DPPO without success degradation by allocating denoising steps adaptively per action type (Yu et al., 9 Aug 2025).
Generalization: Larger models exhibit improved visual robustness (object shape, color, lighting) and better performance on unseen objects and configurations (Zhu et al., 2024, Xia et al., 17 Dec 2025).

5. Practical Implementation and Reproducibility Strategies

Reproducible scalable diffusion policy implementations share several elements (Xia et al., 17 Dec 2025, Zhu et al., 2024):

Point-cloud normalization (centered unit cube) and random subsampling (e.g., 2048 pts).
Cosine learning rate decay, with linear warmup (e.g., 5-epoch) and batch sizes tuned to hardware constraints.
Scheduled sampling probabilities annealed over initial epochs.
Clear train/val/test splits with seeded environment initializations.
Extensive ablation studies: removing AdaLN, ISS, or noncausal attention leads to double-digit drops in success rates, confirming necessity for scaling (Xia et al., 17 Dec 2025, Zhu et al., 2024).
Public release of code, pretrained weights, and data in official repositories.

6. Multidomain Generalization and Future Research

The scalable diffusion policy paradigm extends beyond single-agent control to:

Decentralized Multi-Agent Coordination: Spatial-transformer diffusion models efficiently coordinate coverage in robot swarms, maintaining permutation and translation equivariance while scaling to dozens of agents and large action spaces (Vatnsdal et al., 21 Sep 2025).
Language-Driven Humanoid Control: Joint action-state-text diffusion policies (e.g., SCRIPT) scale to >1B parameters, coupling physical trajectories with semantic tokens via multistream self-attention, nonlinear temporal memory, and hybrid RL fine-tuning (Zhang et al., 21 May 2026).
Sim-to-Real Transfer: RL-steered initial noise (ExpertGen) constrains exploration to safe action manifolds while achieving robust sim-to-real transfer in challenging assembly and manipulation tasks (Xu et al., 16 Mar 2026).
Online RL in Continuous/Discrete Domains: Algorithmic advances such as reweighted score matching (DPMD, SDAC), conditional PPO alignment, and policy gradient for discrete diffusion models (SEPO) facilitate scaling to high-dimensional, combinatorial action spaces with amortized or simulation-free EBM training (Ma et al., 1 Feb 2025, Liu et al., 5 Mar 2026, Ma et al., 26 Sep 2025, Zekri et al., 3 Feb 2025).
Coverage Control, Macro-Actions, and Ensemble RL: Policy-level diffusion scales to combinatorial sequences, large macro-actions, and decentralized interaction through regularization and expressivity of the diffusion kernel (Ma et al., 26 Sep 2025, Vatnsdal et al., 21 Sep 2025).

Anticipated directions include greater integration of geometric consistency, adaptive computation, hierarchical planning, multimodal fusion (language, tactile, vision), and cross-domain transfer using modular, scalable diffusion policy backbones.

7. Summary Table: Empirical Scaling Benchmarks

Model / Study	Domain	Params (M)	Data (#demos)	Inference (ms)	Success (%)
ISS Policy (large) (Xia et al., 17 Dec 2025)	MetaWorld/Adroit	215	50	52.7	86.2
ScaleDP (huge) (Zhu et al., 2024)	MetaWorld+Real	1,000	100	70–90	+21.6 over DT
SCRIPT (huge) (Zhang et al., 21 May 2026)	Language-humanoid	1,230	550k traj	~1000 (batch)	FID↓0.645, Prec@1↑0.464
MADP (Vatnsdal et al., 21 Sep 2025)	Coverage	N agents	100k traj	agent-local	state of art
D3P (Yu et al., 9 Aug 2025)	Real robot	10–100	50+	33.7 Hz	100

Scaling is demonstrated across model size, demonstration count, and control complexity, frequently with monotonic or near-monotonic improvements in key metrics.

Scalable diffusion policy thus constitutes a robust, extensible family of methods for high-dimensional, multimodal robotic and control systems, characterized by architectural innovations (AdaLN, noncausal attention, geometric consistency modules), efficient training and inference procedures, and the ability to generalize across data scales, tasks, and embodiment domains (Xia et al., 17 Dec 2025, Zhu et al., 2024, Zhang et al., 21 May 2026, Vatnsdal et al., 21 Sep 2025, Ma et al., 1 Feb 2025).