Information Spreading in Diffusion Models from Effective Field Theory

Abstract: We study score-matching diffusion models with a convolutional architecture. We argue that the inductive bias of locality means that the machinery of effective field theory from physics can be usefully applied to describe the denoising dynamics. We apply this formalism first to a simple toy example which permits an analytical description, and thereafter to MNIST, and show that in both cases, the mutual information between two points grows in a manner predicted by a simple effective field theory of Brownian motion.

• The paper shows that locality and translational equivariance reduce convolutional diffusion models near the pure-noise endpoint to a heat equation, with nonlinear terms suppressed as the signal vanishes.
• The paper predicts that spatial mutual information follows diffusive scaling through the variable |x−y|²/ᾱt, a result supported by curve collapse in a ±1 grid model and MNIST experiments.
• The paper identifies a smooth transition from diffusion to late-time patch snapping, showing that effective kernel size controls the saturated correlation length while the data enters through a small set of effective coefficients.

Overview

This paper, by Neogi and Iqbal (2608.14308), applies the machinery of effective field theory (EFT) to score-matching diffusion models with convolutional architectures. The central claim is that locality and translational equivariance of the network suffice to constrain the reverse-process dynamics near the pure-noise endpoint to a simple heat equation, whose predictions for the growth of spatial mutual information are verified empirically in a $\pm 1$ grid toy model and on MNIST. The work builds on the Equivariant Local Score (ELS) machine of Kamb and Ganguli, which provides an exact microscopic description of the learned score function for ConvNets, and simplifies it into a small number of universal terms.

Effective field theory of the reverse process

The reverse process is taken in its deterministic probability-flow ODE form,

$$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$

where $\pi = \nabla_x \log p_t$ is the score function. The authors expand the right-hand side in powers of the field and its spatial derivatives. Translational equivariance forbids explicit $x$-dependence of coefficients; locality justifies truncation. Imposing rotational equivariance (${\bf v}_i = 0$) and a $\mathbb{Z}_2$ symmetry $\phi \to -\phi$, and introducing a scaling limit with field dimension $\tilde{\phi} \to \lambda^{-d/2}\tilde{\phi}$ — justified both by Gaussian measure invariance and by the central limit theorem applied to box-averaged fields — only two terms survive in $d=2$: a Laplacian term and a $\tilde{\phi}^3$ term.

The decisive further step uses the microscopic ELS: every nonlinearity $$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$0 carries a factor $$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$1, so all nonlinear terms vanish as $$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$2. The dynamics near the noise endpoint therefore reduces exactly to the diffusion equation

$$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$3

The training data enters only through a few Wilsonian coefficients such as $$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$4. This is the paper's main theoretical result: the complicated patch-sum structure of the ELS collapses to Brownian motion at long distances and early reverse times.

Mutual information growth and diffusive scaling

Assuming Gaussianity — self-consistent since the initial condition is i.i.d. Gaussian noise and the evolution is linear — the mutual information between pixels is $$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$5, where $$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$6 is the correlation coefficient. Solving the diffusion equation for the connected two-point function gives

$$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$7

so correlations depend on space and time only through the self-similar variable $$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$8: curves at different times must collapse when plotted against this variable. This is a sharp, falsifiable prediction, and it also implies that the noise schedule plays the role of physical time while the schedule's detailed shape does not affect the spatial correlations built up.

Experiments

Two-sample model. For a dataset consisting of homogeneous $$\frac{\partial\phi(t)}{\partial t} = \frac{\partial_t \bar{\alpha}_t}{\bar{\alpha}_t}\left[\phi(t)+\pi(\phi(t),t)\right],$$9 and $\pi = \nabla_x \log p_t$0 grids, the ELS admits a closed-form evolution equation involving a $\pi = \nabla_x \log p_t$1 of the local patch sum. Both the analytic ELS machine (with a $\pi = \nabla_x \log p_t$2 kernel) and a trained $\pi = \nabla_x \log p_t$3-equivariant ConvNet exhibit curve collapse against $\pi = \nabla_x \log p_t$4, confirming diffusive scaling; very small separations ($\pi = \nabla_x \log p_t$5 of a few pixels) are excluded because the trained model deviates from the pure ELS there.

Regime transition and correlation length. Feeding a linear interpolation test grid into the ELS score, the authors define the end of the diffusive regime via a factor-$\pi = \nabla_x \log p_t$6 enhancement of the spatial gradient of the score, yielding $\pi = \nabla_x \log p_t$7. A Taylor expansion of the patch sum gives $\pi = \nabla_x \log p_t$8, so the saturated correlation length scales as $\pi = \nabla_x \log p_t$9 — linearly in the effective kernel size. Empirically, fitting the Gaussian slope gives $x$0, consistent with $x$1 to integer precision despite the kernel size drifting over the reverse process. Notably, the transition is smooth: the authors explicitly state there is no discontinuity and that the behavior does not correspond to a conventional phase transition.

MNIST. Without the toy symmetries, additional linear terms appear, but a field redefinition absorbs them; the connected two-point function still obeys a diffusion equation in reparametrized time $x$2, and mutual information remains controlled solely by the Laplacian coefficient. Curve collapse against the self-similar variable is again observed, though imperfectly at large distances, which the authors attribute to interactions of diffusive modes with boundary conditions (circular padding was used).

Snapping regime

Near the endpoint $x$3, expanding the ELS in $x$4 yields

$x$5

an exponential relaxation of each patch toward its nearest training patch $x$6, corresponding to the "memorisation" regime of Biroli et al. This complements the diffusive regime: spatial mutual information saturates at the value set by the architecture, after which patches decouple and snap individually.

Limitations and open questions

Several caveats are conceded directly. The theory assumes finite limits of the EFT coefficients as $x$7 and treats them as constant; the marginal $x$8 term would generate logarithmic corrections invisible at finite experimental resolution. The regime-transition criterion depends on the arbitrary factor $x$9, so no sharp transition time exists. For general datasets, constructing interpolating test functions analytically is difficult because interpolated patches may lie closer to a third training patch. Large-distance collapse on MNIST degrades and is boundary-condition sensitive. The "fuzziness" of region boundaries lacks an analytical treatment. Finally, whether an EFT description extends to non-convolutional architectures — e.g., transformers, where locality may be inherited from data statistics rather than architecture — is left open.

Conclusion

The paper establishes that locality and equivariance reduce the early reverse dynamics of convolutional diffusion models to a heat equation, with the noise schedule acting as time and the dataset entering only through a few coefficients. Diffusive collapse of mutual-information curves is confirmed quantitatively in both an analytically tractable ELS model and trained networks on MNIST, and the late-time snapping dynamics is captured by a simple exponential relaxation equation. The framework provides a systematic inference method for the effective kernel size during generation and connects the dynamical regimes identified in prior statistical-mechanical analyses to concrete EFT equations.