Multinomial Diffusion Equation (MDE)

• The Multinomial Diffusion Equation (MDE) is a discrete-time, finite-difference model that simulates particle diffusion while conserving mass and capturing multinomial fluctuations.
• It accurately reproduces ensemble statistics, including higher cumulants like skewness and kurtosis, especially in low-density regimes where continuum models fail.
• The framework extends to generative modeling for one-hot categorical data, bridging physical simulations with machine learning applications.

The Multinomial Diffusion Equation (MDE) is a finite-difference, discrete-time model of diffusion that captures stochastic fluctuations resulting from particle-level discreteness. Unlike classical deterministic or continuum stochastic formulations, the MDE provides a particle-conserving, synchronously updated evolution on an Eulerian grid, accurately reproducing ensemble statistics—including higher cumulants characteristic of multinomial fluctuations—across a broad range of regimes. The MDE has recently emerged as a foundation both for physically accurate simulation of discrete particle diffusion (Balter et al., 2010) and as a machine learning generative model for categorical data within the denoising diffusion probabilistic modeling paradigm (Hoogeboom et al., 2021).

1. Microscopic Dynamics of the Multinomial Diffusion Equation

The MDE models a 1D periodic spatial domain of length $L$ partitioned into $M$ voxels of size $\Delta x = L/M$. Let $N_i^t$ denote the integer number of particles in voxel $i$ at time $t$, with total $N_0 = \sum_i N_i^t$ fixed. At each discrete timestep $\Delta t$, each particle in voxel $i$ independently hops to its left neighbor with probability $k = D\,\Delta t/\Delta x^2$, to the right with probability $M$0, or remains in place with probability $M$1 (with the constraint $M$2, $M$3).

Let $M$4 and $M$5 denote the numbers of particles moving from voxel $M$6 to $M$7 and $M$8, respectively, during $M$9. Conditional on $\Delta x = L/M$0, the pair $\Delta x = L/M$1 follows a trinomial law: $\Delta x = L/M$2 The update rule, expressing conservation and nearest-neighbor coupling, reads: $\Delta x = L/M$3 This preserves total mass and strictly enforces discrete particle counts, unlike continuum or stochastic partial differential equation approaches (Balter et al., 2010).

2. Continuum and Stochastic Limits

In the thermodynamic limit ($\Delta x = L/M$4, $\Delta x = L/M$5), bin exchange statistics can be approximated using the Central Limit Theorem. The mean and variance of outgoing hops satisfy: $\Delta x = L/M$6 Neglecting higher-order covariances and using $\Delta x = L/M$7, the macroscopic update, after combining Gaussian noise contributions, yields: $\Delta x = L/M$8 In the continuum ($\Delta x = L/M$9, $N_i^t$0, $N_i^t$1 fixed), this converges to the stochastic diffusion equation (SDE): $N_i^t$2 where $N_i^t$3 is space-time white noise. The MDE thus bridges particle-based and macroscopic stochastic diffusion models (Balter et al., 2010).

3. Equilibrium and Fluctuations: Ensemble Statistics

At equilibrium, the MDE yields a multinomial distribution over $N_i^t$4 with equal bin probabilities $N_i^t$5. The resulting statistics are:

• $N_i^t$6
• $N_i^t$7
• $N_i^t$8 for $N_i^t$9

This translates in densities to

• $i$0

The SDE steady-state is Gaussian. The MDE, however, exactly reproduces all factorial moments (i.e., all cumulants) of the multinomial, including non-vanishing skewness and kurtosis at low $i$1, where the SDE instead predicts vanishing higher cumulants. For $i$2 particles per bin, SDE and MDE converge, but for $i$3 the SDE systematically overestimates spatial variance (Balter et al., 2010).

4. Multinomial Diffusion in Generative Modeling

The MDE formalism has been adapted for machine learning in the modeling of one-hot categorical data $i$4 through a discrete-time forward–reverse process (Hoogeboom et al., 2021).

• Forward (noising) process: At each timestep $i$5, the data is mixed with uniform noise (probability $i$6), giving the categorical transition

$i$7

This generates a Markov chain over the simplex, with cumulative signal retention $i$8.

• Marginalization: The closed-form marginal after $i$9 steps is

$t$0

• Reverse (generative) process: To sample from the target data distribution, a parameterized network $t$1 (producing $t$2 on the simplex) learns to approximate the true posterior

$t$3

with loss function

$t$4

No score matching or stochastic approximation is required; all quantities admit closed-form (Hoogeboom et al., 2021).

This discrete-time MDE generalizes the Gaussian denoising diffusion models—the dynamical equation on the simplex plays the role of a finite-difference Fokker–Planck equation in the categorical case.

5. Validation and Regimes of Applicability

Numerical comparison of the MDE, SDE, and direct particle-tracking (overdamped Langevin) show:

• All recover correct mean as $t$5
• For large $t$6, variances match across methods
• For small $t$7 ($t$8 particle/bin), MDE matches true variance, but SDE systematically overestimates variance
• SDE breakdown becomes pronounced near $t$9 particle/bin or less

MDE thus provides a valid stochastic description in regimes inaccessible to the SDE—a critical feature for low-density, finite-population, and reaction–diffusion settings (Balter et al., 2010).

6. Computational and Modeling Implications

The MDE is a $N_0 = \sum_i N_i^t$0-synchronous, grid-based, particle-conserving model suitable for:

• Capturing all cumulants of the stochastic diffusion process at the Eulerian level
• Multiscale modeling frameworks—naturally interfacing with finite difference solvers
• Efficient simulation of reaction–diffusion systems without stochastic time-step handling
• Operating between the exact but inefficient Multivariate Master Equation (asynchronous, event-based) and the SDE (efficient, Gaussian-approximate for large $N_0 = \sum_i N_i^t$1)

A key operational constraint is the Courant–Friedrichs–Lewy (CFL) condition: $N_0 = \sum_i N_i^t$2 to avoid negative populations. The limit $N_0 = \sum_i N_i^t$3 is required for the SDE approximation, while the MDE remains exact for all $N_0 = \sum_i N_i^t$4.

In reactive systems, especially with bimolecular reactions, the MDE offers computational efficiency ($N_0 = \sum_i N_i^t$5 scaling) and accuracy at moderate-to-low densities compared to particle-tracking ($N_0 = \sum_i N_i^t$6) (Balter et al., 2010).

7. Broader Impact and Analytical Paradigms

The MDE framework, as originally formulated, underpins the exact treatment of stochastic fluctuations at discrete particle scales, bridging the gap between particle-based and macroscopic continuum models. Its adaptation for categorical diffusion in machine learning enables direct, tractable training and generation of categorical data, extending denoising diffusion approaches beyond continuous domains (Hoogeboom et al., 2021). This intersection points toward unified stochastic models for both physical and data-generative systems, providing consistent ways to incorporate discreteness, mass conservation, and closed-form optimization for efficient inference and synthesis.