Continuous Diffusion Models

• Continuous diffusion models are stochastic processes evolving in continuous time and state, defined by tools such as stochastic differential equations and continuous-time Markov chains.
• They are applied across science and engineering—from modeling particle dynamics and cell migration to enabling high-fidelity generative modeling in images, language, and graphs.
• Recent advancements extend these models to hybrid, infinite-dimensional, and mixed discrete-continuous data, addressing challenges in scalability, sampling efficiency, and unified theoretical frameworks.

A continuous diffusion model is a mathematical and computational framework that describes stochastic processes evolving over continuous time and, in many cases, continuous state spaces. These models underpin a wide array of scientific and engineering domains, including physics, biology, ecology, generative modeling, and robotics. Their versatility derives from the ability to elegantly characterize both microscopic and macroscopic phenomena via stochastic differential equations (SDEs), continuous-time Markov chains (CTMCs), or related constructs. Recent research extends these models to discrete and hybrid data, infinite-dimensional functional spaces, mixed-type data, and compositional planning scenarios, significantly broadening their domain of applicability.

1. Mathematical Foundations of Continuous Diffusion Models

Continuous diffusion models are fundamentally described by stochastic processes that evolve in continuous time. The canonical example is the stochastic differential equation

$dX_t = f(X_t, t) dt + g(X_t, t) dW_t,$

where $X_t$ represents the system state at time $t$, $f$ is the drift function, $g$ is the diffusion coefficient (possibly a matrix), and $W_t$ is standard Brownian motion. For pure (linear) diffusion without drift, $f=0$ and $g$ is constant, as in the Wiener process. More generally, processes such as the Langevin diffusion add drift to target a stationary distribution $\pi(x)$, e.g.,

$$dX_t = \frac{\gamma^2}{2}\nabla \log \pi(X_t) dt + \gamma dW_t,$$

ensuring that $X_t$ samples from $\pi(x)$ as $t \to \infty$ (Michelot et al., 2018).

For continuous-time Markov chains (CTMCs) operating in discrete (often categorical) state spaces, the Kolmogorov forward equation governs the evolution of the probability of state $x$ via a generator matrix $Q_t$:

$$\frac{d}{dt} q_{t|s}(x_t | x_s) = \sum_{x} q_{t|s}(x|x_s) Q_t(x, x_t) \qquad [2211.16750, 2406.06449]$$

Recent advances generalize these frameworks to infinite-dimensional (functional) spaces using operator-valued SDEs and consider mapping between discrete categorical states and continuous (e.g., hyperspherical) manifolds for LLMing or hybrid tasks (Franzese et al., 2023, Zhang et al., 2023, Jo et al., 17 Feb 2025).

2. Model Classes and Their Domains

Model classes can be distinguished by domain and formalism:

Model Type	State Space	Formulation
Langevin/Brownian diffusion	$\mathbb{R}^n$	SDE with drift/diffusion; stationary distributions
CTMC-based (jump/diffusion on discrete states)	finite/categorical	Rate matrix evolution; jump processes
Hybrid (continuous + discrete)	mixed	Coupled SDE and CTMC/MD4-like masking
Functional (infinite-dimensional)	$L^2(M)$ etc.	Operator SDE; operator-valued Wiener process
Parameterized SDE (FP-Diffusion)	$\mathbb{R}^n$	Metric, symplectic/antisymmetric parameterizations

Each class addresses specific phenomena, from granular media diffusion (Amitai et al., 2015), molecular transport (Merunka et al., 2017), and cell movement (Bao et al., 2018), to generative modeling in high dimension (Du et al., 2022, Franzese et al., 2023, Zhang et al., 2023) and symbolic-continuous combined planning (Høeg et al., 26 Sep 2025).

3. Model Adaptations: Extensions and Generalizations

Continuous diffusion models have evolved along several methodological axes:

• Parameterization and Flexibility: Early score-based models used fixed SDEs; recent frameworks (e.g., FP-Diffusion) parameterize drift and diffusion in spatially anisotropic or Hamiltonian-inspired ways to tailor the stationary distribution and sample paths. These parameterizations leverage Riemannian geometry and symplectic forms to improve convergence and tractability (Du et al., 2022).
• Infinite-Dimensional Extension: Functional diffusion extends SDEs to operator-valued drift/diffusion, enabling modeling on function spaces (e.g., for continuous images, SDFs, deformation fields) and requiring generalizations of the Girsanov theorem and sampling theory for practical implementation (Franzese et al., 2023, Zhang et al., 2023).
• Continuous-Time Discrete-State Diffusion: CTMC-based approaches construct transition-rate driven diffusions on categorical or graph-structured domains, permitting analytical reverse processes and unbiased conditional probability matching (categorical score matching) (Sun et al., 2022, Siraudin et al., 10 Jun 2024). This is critical in graph generation, symbolic planning, and categorical data (e.g., language) (Dieleman et al., 2022, Jo et al., 17 Feb 2025).
• Mixed-Type and Hybrid Models: Continuous diffusion has been adapted to synthesize mixed-type tabular data (using loss calibration, feature-specific noise, score interpolation) (Mueller et al., 2023), and to simultaneously model symbolic (discrete plan) and continuous (trajectory) components of planning problems, applying coupled loss terms and bidirectionally informed denoising (Høeg et al., 26 Sep 2025).
• Continuous Conditional Generative Modeling: Models such as CCDM address data generation conditioned on continuous variables, with innovations like vicinal denoising losses, specialized label embeddings, and efficient conditional sampling (Ding et al., 6 May 2024).

4. Core Applications and Empirical Insights

Applications of continuous diffusion models are diverse:

• Physical and Biological Systems: Modeling finite-size particle diffusion in porous media reveals limitations of isotropic continuous-time random walks (CTRW). Introduction of anisotropy and memory effects (e.g., DA model) is necessary to capture topology-induced anomalous diffusion (Amitai et al., 2015). In EPR spectroscopy, continuous diffusion models relate EPR parameters to relative radical motion, with kinetic equation derivations providing more accurate diffusion coefficients, particularly in supercooled water (Merunka et al., 2017).
• Ecological Movement and Habitat Selection: Langevin diffusion SDEs are used to mechanistically link animal motion to resource selection, accommodating irregular sampling and environmental covariate gradients (Michelot et al., 2018).
• Cell Movement and Aggregation Phenomena: Nonlinear parabolic PDEs derived from discrete random-walk models describe aggregation-diffusion cell migration, with sharp distinctions between regimes where solutions exist or fail (diffusion vs. aggregation regions) (Bao et al., 2018).
• Generative Modeling (Images, Language, Graphs, Tables, Functions): Continuous diffusion models in generative tasks support flexible noising and denoising, enable high-fidelity synthesis in images and text (even for mixed and categorical data), and underpin advances in continuous-time graph modeling, functional data generation, and hybrid planning (Du et al., 2022, Shabani et al., 2022, Dieleman et al., 2022, Franzese et al., 2023, Siraudin et al., 10 Jun 2024, Jo et al., 17 Feb 2025, Mueller et al., 2023, Høeg et al., 26 Sep 2025).
• Robotic Planning and Constraint Satisfaction: Compositional continuous diffusion solvers use Langevin-style annealed dynamics in energy-based factor graphs, enabling high success rates for long-horizon, tightly constrained robotics scenarios, especially where modular composition and parallel inference are required (Yang et al., 2023).

5. Methodological Innovations: Training, Sampling, and Inference

Continuous diffusion models frequently innovate beyond classical score-matching SDE/ODE training and ancestral sampling:

• Score Interpolation and Categorical Matching: Embedding categorical data in continuous space and using cross-entropy losses or conditional probability ratio matching enable score-based learning where gradients of discrete likelihoods are ill-defined (Dieleman et al., 2022, Sun et al., 2022, Mueller et al., 2023).
• Bridge and Geodesic Diffusion: On statistical manifolds, continuous flows along geodesics with proper metric adjustment (e.g., Fisher–Rao metric) bridge original discrete transitions via continuous paths, improving iterative refinement and enabling simulation-free training when radial symmetry is leveraged (Jo et al., 17 Feb 2025).
• Adaptive and Feature-Specific Noise Scheduling: In mixed-type or highly heterogeneous data (e.g., tabular), feature-wise and type-wise noise schedules adapt the diffusion process so as to balance learning and generation across modalities (Mueller et al., 2023).
• Conditional and Vicinal Losses: Loss functions that weight sample-pairs by label proximity or apply classifier-free guidance during sampling yield improved conditional generative quality, especially on sparse or imbalanced label distributions (Ding et al., 6 May 2024).
• Integration with Planning and Symbolic Reasoning: Hybrid models use parallel or coupled diffusion streams with cross-modality conditioning and imputation, promoting coherent planning over combinatorially large symbolic and continuous action spaces (Høeg et al., 26 Sep 2025).
• Simulation-Free and Analytical Sampling: Riemannian normal approximations, time-predictor mechanisms, and analytical bridging between discrete and continuous approaches reduce computational burden and optimize inference control (Jo et al., 17 Feb 2025, Li et al., 28 May 2025).

6. Challenges, Limitations, and Future Directions

Several open challenges and limitations remain:

• Scalability: Continuous diffusion models (especially for graphs and functions) face quadratic or higher computational cost with respect to state size or discretization, motivating research into sparsification and decomposition (Siraudin et al., 10 Jun 2024).
• Extension to Infinite-Dimensional, Non-Euclidean, and Hybrid Spaces: While functional and manifold-based approaches address some aspects, practical implementations require additional work to handle arbitrary domains and attribute types (Zhang et al., 2023, Franzese et al., 2023).
• Sampling Efficiency: Diffusion models generally exhibit slower sampling compared to GANs or flow models. Accelerated solvers and approximate ODE-based reverse dynamics are active areas of research (Ding et al., 6 May 2024).
• Continual and Compositional Learning: Maintaining generative fidelity across tasks (avoiding catastrophic forgetting in streaming/continual settings) requires novel consistency objectives and hierarchical regularization strategies (Liu et al., 17 May 2025).
• Unified Theoretical Frameworks: Recent advances in bridging discrete and continuous paradigms via dual-time or non-simultaneous processes point to a unification of prior approaches, yet further generalizations and deeper understanding are advocated (Li et al., 28 May 2025).

7. Conclusion

Continuous diffusion models constitute a foundational class of methods for modeling stochastic dynamical systems, providing flexible tools for simulation, inference, and generative modeling across physics, biology, engineering, and machine learning. Contemporary research synthesizes advances from stochastic calculus, information geometry, and deep learning to produce powerful models for continuous, discrete, and hybrid data types. Practical innovations—in parameterization, loss function design, domain adaptation, and learning methodologies—continue to extend the reach and efficacy of continuous diffusion frameworks, with open challenges related to scalability, efficiency, unification, and continual operation representing the next frontiers.