Constrained Diffusion Algorithms

• Constrained diffusion algorithms are methods that adjust diffusion dynamics to enforce explicit constraints while maintaining the integrity of the underlying distribution.
• They integrate techniques such as dynamic programming, projection, and dual training to achieve exact or near-exact constraint satisfaction during reverse sampling.
• Applications include constrained decoding for language models, safe planning in control systems, and accurate inverse problem solving, yielding improved validity and performance.

Constrained diffusion algorithm denotes a family of methods that modify diffusion dynamics, reverse-time samplers, or diffusion-style denoisers so generated objects satisfy explicit constraints while preserving, approximating, or deliberately reshaping the underlying model distribution. Across recent literature, the term covers exact constrained decoding for diffusion LLMs, constrained sampling for inverse problems and safe control, training procedures that align diffusion models with feasibility or fairness requirements, and geometry-aware diffusion processes defined directly on constrained sets (Suresh et al., 29 May 2025, Jayaram et al., 2024, Zhang et al., 14 Jun 2025, Khalafi et al., 2024, Fishman et al., 2023). A broader adjacent usage treats “energy-constrained diffusion” as a deterministic representation-propagation principle underlying MLPs, GNNs, and Transformers rather than generative sampling, indicating that the phrase names a methodological motif rather than a single canonical algorithm (Wu et al., 2024).

1. Constraint models and optimization objectives

The central formal distinction in constrained diffusion is the nature of the feasible set. In diffusion LLM inference, constraints are often formal languages. DINGO defines a user constraint $R$ as a regular expression over characters, compiles it to a character-level DFA, and then lifts that automaton to token level so that blockwise diffusion predictions can be constrained at the token vocabulary actually used by the model. The associated inference objectives are constrained MAP,

$\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$

and distribution-preserving constrained sampling from

$p_\theta(y \mid y \in L(R)).$

That formulation is specific to block-parallel diffusion LLMs whose working distribution factorizes across positions at inference time (Suresh et al., 29 May 2025).

Other constrained diffusion formulations use generic equality and inequality sets. The landing framework writes the feasible set as

$$\Sigma := \{x \in \mathbb{R}^d \mid h(x)=0,\ g(x)\le 0\},$$

with smooth equality constraints $h$ and inequality constraints $g$, and develops diffusion processes that remain on or are driven back toward $\Sigma$ even when $\Sigma$ is nonconvex (Jeon et al., 20 Apr 2026). In direct trajectory optimization, equality-constrained diffusion instead treats nonlinear dynamics and boundary conditions as equality constraints in a nonlinear program and evolves primal variables and Lagrange multipliers by a Langevin-like augmented-Lagrangian system (Kurtz et al., 2024). In inverse problems, CDIM imposes either exact linear constraints $Ax=y$ in the noiseless case or an exact constraint on the residual distribution in the noisy case (Jayaram et al., 2024).

A separate line of work constrains the learned distribution itself. Dual training formulates diffusion training as minimizing divergence to the data distribution subject to distribution-level constraints such as KL divergence to desired reference distributions or expectation constraints on generated attributes (Khalafi et al., 2024). Alignment and composition of diffusion models use reverse-KL or forward-KL constraints to enforce reward thresholds or proximity to several pretrained models simultaneously, producing reward-tilted, product, or mixture target distributions depending on the formulation (Khalafi et al., 26 Aug 2025).

2. Exact constrained inference for diffusion LLMs

The most explicit “constrained diffusion algorithm” in the decoding sense is "DINGO: Constrained Inference for Diffusion LLMs" (Suresh et al., 29 May 2025). Its starting point is the mismatch between traditional constrained decoding and diffusion LLMs. Autoregressive methods prune invalid next tokens on prefixes, but diffusion LLMs predict a block of tokens in parallel. Per-position masking does not account for cross-position compatibility, so it cannot preserve the correct conditional distribution when constraints couple multiple positions, such as brace matching or mutual exclusion.

DINGO resolves that mismatch by dynamic programming over block positions and automaton states. For MAP decoding it uses a Viterbi-style recurrence,

$$V_t(s') = \max_{s \in Q} \max_{a \in \Sigma: \delta(s,a) \ni s'} \left(V_{t-1}(s) + \log p_t(a)\right),$$

and for normalization and exact sampling it uses forward messages

$\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$0

Backward messages then yield ancestral constrained sampling exactly from the conditional distribution under the factorized block model. The stated complexity is

$\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$1

with memory $\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$2 (Suresh et al., 29 May 2025).

The algorithm’s guarantees are correspondingly strong but assumption-dependent. Under the factorization $\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$3, DINGO returns a valid prefix compatible with the regular expression, its MAP dynamic program maximizes the model probability among valid strings, and its forward-backward sampler is distribution-preserving for $\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$4 (Suresh et al., 29 May 2025). Empirically, the method achieves up to a 68 percentage point improvement over unconstrained inference, reaches 100% schema-valid and syntactically valid outputs on JSON-Mode-Eval across all evaluated models, and improves symbolic-math accuracy while maintaining 100% syntactic validity (Suresh et al., 29 May 2025).

The regular-language setting has since been extended to context-free grammars for diffusion LLMs by reducing additive infilling to emptiness of the intersection between a CFG and a regular language. The resulting product-grammar construction supports multi-region infilling and practical CFG-constrained diffusion decoding for C++, JSON, and SMILES. Reported outcomes include near-perfect syntactic correctness, with JSON and SMILES reaching 100.0% under constrained decoding with completion fallback, while median completion overhead remains about 30% for diffusion LLMs (Mündler et al., 13 Aug 2025).

3. Projection, primal–dual control, and test-time constrained sampling

Outside language decoding, a dominant pattern is to take a pretrained diffusion model and modify reverse-time sampling at inference. "Constrained Diffusers for Safe Planning and Control" formulates reverse diffusion as a Langevin-like step and then inserts one of three constraint handlers: projection onto the feasible set, primal–dual updates with Lagrange multipliers, or augmented Lagrangian updates with slack variables. The projected method enforces feasibility at every reverse step; the primal–dual and ALM variants trade exact per-step feasibility for lower computational cost while remaining close to the base sampler (Zhang et al., 14 Jun 2025).

The same design pattern appears in joint generation. Projected Coupled Diffusion adds a coupling cost $\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$5 to coordinate multiple pretrained diffusion models and applies a projection operator after each diffusion step, so that the joint sample both exhibits the desired interaction and lies in the hard constraint set. For convex kinematic constraints, the projection is implemented efficiently by ADMM; for convex-hull latent constraints, it is solved by mirror descent on the simplex (Luan et al., 14 Aug 2025).

Several methods replace exact projection with guided local optimization. Trust Sampling reinterprets loss-guided diffusion as a constrained optimization performed at each diffusion level, allowing multiple inner gradient steps while the proxy objective is still reliable. Trust is modulated by the variance at the current diffusion level, and an estimate of the state manifold based on the predicted noise norm is used to terminate the inner loop when the iterate departs from the model’s state manifold (Huang et al., 2024). Fast constrained sampling in pretrained diffusion models pushes this idea further by avoiding backpropagation through the denoiser. It observes that one can replace the usual $\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$6 guidance with a numerically estimated Jacobian–vector product $\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$7 via two forward passes, using

$\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$8

which produces substantial speed-ups and lower memory usage relative to backprop-based training-free methods (Graikos et al., 2024).

Inverse-problem solvers make the same intervention on the Tweedie estimate. CDIM performs DDIM updates but inserts inner optimization on $\hat{y} = \arg\max_{y \in L(R)} p_\theta(y),$9 so that the final sample exactly satisfies $p_\theta(y \mid y \in L(R)).$0 in the noiseless linear case, or exactly matches a prescribed residual distribution in the noisy case (Jayaram et al., 2024). For direct trajectory optimization, equality-constrained diffusion replaces projection by a coupled Langevin evolution in primal variables and multipliers,

$p_\theta(y \mid y \in L(R)).$1

thereby embedding nonlinear dynamics constraints directly into the diffusion process rather than enforcing them by shooting rollouts (Kurtz et al., 2024).

4. Training-time alignment and model-level constrained diffusion

A second major class constrains diffusion during training rather than only at inference. In trajectory optimization, constraint-aware diffusion models introduce a hybrid loss that augments denoising with a violation penalty measured on one-step-denoised predictions. The violation term is normalized by the ground-truth mean violation at the same noise level, so that late, inherently noisy steps are not penalized as strongly as near-data steps (Li et al., 2024). "Aligning Diffusion Model with Problem Constraints for Trajectory Optimization" uses the same principle and explicitly emphasizes re-weighting by diffusion-step violation statistics to align predicted violations with ground-truth corruption statistics at each step (Li et al., 1 Apr 2025). Both formulations improve feasibility metrics but do not provide hard guarantees at inference.

Dual-constrained training instead treats the entire generative model as the object of constrained optimization. "Constrained Diffusion Models via Dual Training" minimizes divergence to the data distribution subject to KL or expectation constraints on the generated distribution, with projected dual ascent on nonnegative multipliers. In the unparametrized case, the optimum is characterized as sampling from a mixture

$p_\theta(y \mid y \in L(R)).$2

which makes explicit that constraints reshape the target distribution rather than merely rejecting samples (Khalafi et al., 2024). "Composition and Alignment of Diffusion Models using Constrained Learning" applies the same Lagrangian logic to reward alignment and composition, yielding reward-tilted solutions for alignment and product or mixture distributions for composition depending on whether reverse-KL or forward-KL constraints are used (Khalafi et al., 26 Aug 2025).

Some papers move the constraint into the geometry of the diffusion process itself. "Diffusion Models for Constrained Domains" defines forward noising either by a logarithmic-barrier metric or by reflected Brownian motion, thereby ensuring trajectories remain in an inequality-constrained domain and deriving reverse-time SDEs and implicit score matching tools adapted to the boundary conditions (Fishman et al., 2023). "Mirror Diffusion Models for Constrained and Watermarked Generation" pushes diffusion into a dual Euclidean space through a mirror map and returns to the primal constrained set via the inverse mirror map, so feasibility is achieved by construction for convex domains such as simplices and $p_\theta(y \mid y \in L(R)).$3-balls (Liu et al., 2023). The more recent landing framework generalizes this to nonconvex equality-and-inequality sets by replacing projection with a landing drift

$p_\theta(y \mid y \in L(R)).$4

which drives constraint violations to decay exponentially and can be combined with overdamped or underdamped dynamics (Jeon et al., 20 Apr 2026).

A separate but related usage appears in "Transformers from Diffusion: A Unified Framework for Neural Message Passing", where energy-constrained diffusion is not a generative sampler but an explicit Euler discretization of a diffusion PDE whose layerwise couplings are chosen so that a global energy decreases monotonically. In that setting, “constrained diffusion” names an energy-descent principle that induces MLP, GNN, and Transformer layers, rather than constraint satisfaction in generative sampling (Wu et al., 2024).

5. Guarantees, empirical behavior, and representative applications

A key divide in the literature is between exact or by-construction guarantees and soft empirical alignment. Exact or formal guarantees appear when the constraint is integrated into the sampler in a mathematically explicit way. DINGO proves correctness, MAP optimality, and exact conditional sampling under block-factorized diffusion-LM inference (Suresh et al., 29 May 2025). CFG-constrained decoding guarantees syntactic validity up to timeout through repeated feasibility checks over CFG–regular-language intersection (Mündler et al., 13 Aug 2025). CDIM gives exact final satisfaction of linear equality constraints in noiseless inverse problems and exact residual-distribution matching under the assumed noise model in the noisy case (Jayaram et al., 2024). Mirror diffusion, constrained-domain diffusions based on barrier or reflected processes, projected samplers, and landing-based methods all preserve feasibility either by construction, by zero-flux boundary dynamics, or by explicit projection (Liu et al., 2023, Fishman et al., 2023, Zhang et al., 14 Jun 2025, Jeon et al., 20 Apr 2026).

By contrast, loss-aligned training methods generally improve but do not guarantee feasibility. Constraint-aware trajectory diffusion lowers pre-solver violations and raises feasible ratios, but the paper explicitly notes that constraints are not guaranteed at inference because satisfaction is induced only implicitly through the training loss (Li et al., 2024). The DDDAS-inspired constraint-aligned DDPM for trajectory optimization likewise reports substantial reductions in violations while preserving warm-start quality, yet remains penalty-based rather than exact (Li et al., 1 Apr 2025).

Applications now span symbolic language generation, safe control, inverse problems, structured data, and scientific computing. DINGO achieves 100% schema-valid JSON and 100% syntactic validity on symbolic-math constraints (Suresh et al., 29 May 2025). Trust Sampling improves image restoration metrics such as FID and LPIPS under super-resolution, inpainting, and deblurring, while also handling high-level motion constraints in 3D human motion generation (Huang et al., 2024). CDC, a training-free neurosymbolic framework for code generation, uses localized denoising corrections to improve compile rate, pass@1, and security metrics, including raising HumanEval-X C++ pass@1 from 34.1% to 65.2% for Dream-Coder 7B and increasing LLMSecEval+ sec@1 from 54.7% to 80.7% (Shao et al., 16 May 2026). In materials science, terminal-set constrained diffusion for Bi$p_\theta(y \mid y \in L(R)).$5Te$p_\theta(y \mid y \in L(R)).$6 point defects substantially lowers RMSD, RDF discrepancy, and surrogate forces relative to conditional or projected baselines by enforcing physical constraints only in the low-noise terminal regime (Cui et al., 22 Feb 2026).

6. Limitations, misconceptions, and research directions

A common misconception is that constrained diffusion always means projection onto a feasible set during reverse sampling. The literature is broader. Some methods perform exact dynamic programming over automata; some alter the diffusion geometry itself through barrier, reflection, mirror, or landing constructions; some solve primal–dual or augmented-Lagrangian updates; and some only regularize training so that feasibility becomes more likely rather than guaranteed (Suresh et al., 29 May 2025, Fishman et al., 2023, Liu et al., 2023, Zhang et al., 14 Jun 2025, Khalafi et al., 2024).

Another misconception is that stronger constraints automatically imply distribution preservation. In fact, preserving the model’s constrained conditional law is difficult. DINGO achieves it only under the explicit independence assumption on block positions (Suresh et al., 29 May 2025). CDIM’s exactness depends on linear measurements and correct residual-noise specification (Jayaram et al., 2024). Trust-sampling, fast-guidance, CDC, and other training-free guidance methods intentionally bias the sampler and may trade diversity or realism for feasibility (Huang et al., 2024, Graikos et al., 2024, Shao et al., 16 May 2026). Projection-heavy methods can also distort the marginal distribution or induce mode collapse, as observed for narrow feasible latent hulls in coupled image generation (Luan et al., 14 Aug 2025).

The principal open problems are therefore structural. Regular expressions are insufficient for many program-analysis tasks, motivating CFG-based and eventually semantic constrained decoding (Mündler et al., 13 Aug 2025). Large DFAs, dense vocabularies, and high-dimensional projections create substantial runtime pressure, even when reported overhead is “marginal” in moderate settings (Suresh et al., 29 May 2025). Surrogate mismatch remains central in scientific and control domains: force models, RDF discrepancies, static analyzers, or learned reward functions may not perfectly encode the true constraint, so a sampler can become “constraint-satisfying” only with respect to an approximation (Cui et al., 22 Feb 2026, Khalafi et al., 26 Aug 2025, Shao et al., 16 May 2026). Nonconvex equality-and-inequality sets also remain delicate; the landing framework addresses this without iterative projection, but still relies on smoothness, active-set regularity, and step-size control (Jeon et al., 20 Apr 2026).

Taken together, the literature suggests that “constrained diffusion algorithm” is best understood as a design space organized by three questions: how the feasible set is represented, where the constraint enters the diffusion pipeline, and what guarantee is required. Formal-language decoding, projected and primal–dual reverse samplers, dual-constrained training, and geometry-aware diffusions answer those questions differently, and their differences are not incidental; they determine whether the method offers exact feasibility, calibrated constrained sampling, or only empirical reduction of violations.