Stochastic Perturbation Regularization (SPR)

• SPR is the use of stochastic, irregular perturbations to regularize singular, ill-posed, or poorly convergent systems across various domains.
• It introduces effects like diffusion, viscosity, and chaotic averaging that restore well-posedness and strengthen convergence in pathwise, Fourier-analytic, and neural network settings.
• SPR spans multiple formulations—from nonlinear PDEs and stochastic heat equations to deep learning optimizations—with regime-dependent thresholds ensuring optimal regularization.

Stochastic Perturbation Regularization (SPR) denotes the use of stochastic, rough, or sufficiently irregular perturbations to regularize an otherwise singular, ill-posed, or poorly convergent evolution. In the literature represented here, SPR appears in several mathematically distinct but structurally related forms: path-by-path regularization for fully nonlinear PDE, multiplicative Brownian regularization for nonlinear transport equations, pathwise regularization of the stochastic heat equation through irregular perturbation, stochastic regularization of time averages in integrable Hamiltonian systems, and stochastic or chaos-induced regularization in deep learning and optimization (Gassiat et al., 2016, Wei et al., 2017, Catellier et al., 2021, Bernardi et al., 2012, Sun et al., 2018, Lim et al., 2022). Across these settings, the perturbation modifies the effective dynamics by introducing diffusion, irregular averaging, or a viscosity-like term, and this can restore well-posedness, improve convergence in strong norms, or bias optimization toward flatter minima.

1. Canonical formulations of SPR

A common feature of SPR is that the perturbation is not merely additive randomness in an ambient state space; it is built into the evolution law in a way that changes regularity properties of the equation or algorithm. Representative formulations appearing in the literature include the stochastic Hamilton–Jacobi equation

$$du + \frac{1}{2}|Du|^2 \circ d\xi_t = F(x,u,Du,D^2u)\,dt,$$

the stochastic nonlinear transport equation

$$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$

the multiplicative stochastic heat equation with an irregular path perturbation

$$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$

and the stochastic averaging model for integrable Hamiltonian systems

$d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$

In machine learning, the same principle is expressed through stochastic ResNet updates

$$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$

and through multiscale perturbed gradient descent (MPGD),

$$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$

These formulations are drawn from distinct research programs, but each treats perturbation as a regularizing operator rather than as exogenous noise (Gassiat et al., 2016, Wei et al., 2017, Catellier et al., 2021, Bernardi et al., 2012, Sun et al., 2018, Lim et al., 2022).

The regularized object also varies by domain. In PDE and SPDE, the target is typically existence, uniqueness, or derivative bounds. In Hamiltonian averaging, the target is convergence to the time average in stronger norms. In deep learning and optimization, the target is improvement of generalization capability and an implicit preference for flatter minima. This suggests that SPR is best understood as a family of mechanisms rather than a single theorem.

Setting	Perturbation form	Regularized outcome
Fully nonlinear PDE	Nonlinear stochastic perturbation $d\xi_t$	Path-by-path $L^\infty$ bounds for $D^2u$
Nonlinear transport	Multiplicative Stratonovich Brownian noise	Existence, uniqueness, $BV$ and Hölder regularity
mSHE	Sufficiently irregular continuous path $$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$0	Wellposedness with distributional coefficients
Integrable Hamiltonian flow	Additive white noise in angles	Convergence in Sobolev norm in a special vanishing limit
ResNet / GD	Dropout or chaotic perturbations	Artificial viscosity, Hessian penalization, improved generalization

2. Pathwise regularization in nonlinear PDE

For fully nonlinear PDE of Hamilton–Jacobi type, SPR is formulated pathwise rather than in law. The central estimate is

$$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$1

where $$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$2 and $$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$3 are maximal continuous solutions to reflected SDEs. The bounds are expressed as solutions to reflected SDE and are shown to be optimal. The resulting effect is specifically two-sided control of the second derivative, whereas in deterministic problems only one-sided estimates are usually possible. In the deterministic case, the reflected SDE becomes an ODE, and the process $$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$4 can and typically does hit zero in finite time and stays there, corresponding to blow-up in the solution’s second derivative; the stochastic term “pushes” $$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$5 away from $$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$6, preventing permanent singularity pathwise (Gassiat et al., 2016).

The canonical model

$$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$7

shows how a Stratonovich perturbation can maintain global bounds on $$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$8. In the one-dimensional case with $$\partial_t p(t,x) + b(x)\cdot \nabla f(p) + \partial_{x_i}p(t,x)\circ dB_i(t)=0,$$9, one has

$$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$0

For the stochastic $$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$1-Laplace equation in one dimension with $$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$2, the reflected SDE takes the form

$$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$3

and there is a critical value of the noise intensity: only if $$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$4 the process avoids being absorbed at $$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$5, which is stated to be sharp (Gassiat et al., 2016).

A related but distinct manifestation occurs in nonlinear transport equations. For

$$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$6

existence and uniqueness of stochastic entropy solutions are obtained under substantially weaker regularity assumptions on $$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$7 than in the deterministic problem. If $$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$8, $$du_t = Au_t\,dt + b(u_t)\,dt + g(u_t)\,d\zeta_t + dw_t,$$9 for some $d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$0, $d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$1, $d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$2, and $d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$3, then there exists a stochastic entropy solution; if in addition $d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$4, the entropy solution is unique. If $d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$5, then the unique entropy solution satisfies

$d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$6

and, for almost all $d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$7,

$d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$8

The paper also constructs counterexamples showing that the deterministic counterpart admits no $d\theta = g(I)\,dt + \sqrt{2\nu}\,dW_t.$9 solution in the relevant class. The main mechanisms are the kinetic formulation, stochastic BGK approximation, commutator estimates, and stochastic flows (Wei et al., 2017).

3. Irregular perturbation and singular stochastic heat equations

In the stochastic heat equation with multiplicative spatial noise, SPR takes the form of a perturbation by a sufficiently irregular continuous path. The equation

$$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$0

is studied under the possibility that the drift and diffusion coefficients are generalized functions or distributions. The key claim is that a perturbation by a sufficiently irregular continuous path establishes wellposedness of such equations, even when the coefficients are distributions. This is explicitly framed as pathwise regularization by noise in an infinite-dimensional setting (Catellier et al., 2021).

The central analytical device is the averaged field

$$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$1

This operation “smears out” the irregularity of $$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$2 along the trajectory of $$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$3. If $$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$4 is irregular enough, then for singular $$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$5, including distributions, $$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$6 becomes a regular function. The equation is then recast through a nonlinear Young–Volterra integral. For the shifted solution $$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$7, the mild formulation is

$$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$8

The regularizing property of $$X_{k+1}=X_k+\mathcal{F}(X_k,w_k)\odot \frac{\gamma_k}{p},$$9 is what makes the Volterra-type nonlinear Young integration well defined (Catellier et al., 2021).

The abstract existence and uniqueness theorem states, in paraphrased form, that if $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$0 is a sufficiently irregular path and the averaged field $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$1 is regular enough in suitable Besov/Hölder spaces, then the multiplicative stochastic heat equation admits a unique pathwise solution even with $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$2 and $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$3 distributions and with rough initial data and noise. A stability statement is also proved: if smooth $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$4 approximate a singular $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$5, then solutions to the mollified equations converge to the solution of the singular-coefficient equation as long as $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$6 converges to $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$7 (Catellier et al., 2021).

The concrete example in Section 5 is fractional Lévy stable motion. It is defined as a moving average of a symmetric, $$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$8-stable Lévy process, and its trajectories are described as almost surely highly irregular. For appropriate parameters and initial data, the averaged field

$$x_{k+1}^{(m)} = x_k^{(m)} - \frac{1}{m}\nabla \hat{\mathcal{R}}(x_k^{(m)},S_n) - \frac{\mu}{m^{1/\alpha_1}} v_1(y_k^{(1)}) \odot x_k^{(m)} + \frac{\sigma}{m^{1/\alpha_2}} v_2(y_k^{(2)}).$$9

is regular, and the main theorem then yields well-posedness for rough $d\xi_t$0. The summary states in particular that for $d\xi_t$1, spatial white noise $d\xi_t$2, $d\xi_t$3 can be a distribution if $d\xi_t$4 is small enough, and that the solution is global if initial conditions and $d\xi_t$5 are globally bounded (Catellier et al., 2021).

4. Resonance suppression and convergence to time averages

In integrable Hamiltonian systems, SPR is used to regularize the convergence of time averages in the presence of resonances and small divisors. For a smooth observable $d\xi_t$6, the deterministic finite-time average

$d\xi_t$7

converges poorly in strong norms. The stochastic perturbation adds white noise to the angle variables,

$d\xi_t$8

leading to the stochastic time average

$d\xi_t$9

The deterministic and stochastic averages are then compared in the uniform Fourier norm, the action-averaged $L^\infty$0 Fourier norm, and a Sobolev norm (Bernardi et al., 2012).

For generic $L^\infty$1, the deterministic sequence $L^\infty$2 or its exponentially damped version $L^\infty$3 does not converge to the true time average in the uniform Fourier norm or in the Sobolev norm. Convergence does hold in the weaker action-averaged $L^\infty$4 norm. By contrast, the regularized average $L^\infty$5 converges in all three norms, including the Sobolev norm, provided that $L^\infty$6 and $L^\infty$7. This is the special vanishing limit identified in the paper (Bernardi et al., 2012).

The mechanism is Fourier-analytic. Deterministic averaging produces denominators of the form

$L^\infty$8

where small divisors cause singular behavior near resonances. The stochastic perturbation replaces this by

$L^\infty$9

which adds a viscosity term and avoids division by exactly zero. The summary describes this as smoothing out resonances and controlling small divisors, and it explicitly connects the procedure to viscosity regularization in PDE and to weak KAM theory (Bernardi et al., 2012).

5. Optimization, deep learning, and implicit regularization

In deep learning, SPR is analyzed through stochastic training procedures that can be approximated by SDE. For residual networks, the update

$D^2u$0

can be rewritten as a deterministic residual term plus a noise term. In the limit of small step size, this approximates the SDE

$D^2u$1

Using Itô’s formula, the expected output $D^2u$2 satisfies the backward Kolmogorov equation

$D^2u$3

The second-order term is interpreted as an artificial viscosity. In the formulation given, this artificial viscosity smooths the loss landscape, reduces the number of sharp minima, and lowers energy barriers between minima, allowing SGD to find broader, more generalizable solutions (Sun et al., 2018).

The architectural example studied is Bernoulli dropout inserted after the last convolutional layer of every residual block. The reported experiments on CIFAR-10 state that dropout-empowered ResNets with appropriate $D^2u$4 obtain lower test error than plain ResNets, that there is a sweet spot for $D^2u$5, and that both too low and too high noise levels harm performance. The same discussion states that dropout narrows the gap between training and test error, while excessive regularization can oversmooth the landscape and harm training accuracy (Sun et al., 2018).

A distinct line of work replaces genuinely random perturbations by deterministic chaotic perturbations. MPGD augments gradient descent by chaotic observables generated by the Thaler map,

$D^2u$6

Under appropriate assumptions, as the step-size decreases, the MPGD recursion converges weakly to an SDE driven by a heavy-tailed Lévy-stable process in the Marcus sense,

$D^2u$7

The work further derives a generalization bound for the limiting SDE and analyzes the implicit regularization effect brought by the dynamical regularization (Lim et al., 2022).

In the weak perturbation regime, MPGD is shown to be equivalent to optimizing a regularized loss,

$D^2u$8

The dominant term for large $D^2u$9 is the trace of the Hessian, so MPGD penalizes regions with large Hessian and biases the dynamics toward flatter minima. The experimental settings listed include a synthetic “Widening Valley” loss, Airfoil Self-Noise regression with a shallow neural network, CIFAR-10 classification with ResNet-18, and electrocardiogram classification (Lim et al., 2022).

6. Mechanisms, distinctions, and limitations

The literature does not present SPR as a single universal smoothing theorem. Instead, the regularizing mechanism depends on the structural role of the perturbation. In fully nonlinear PDE, the perturbation creates reflected-SDE control of semiconvexity and semiconcavity bounds. In stochastic transport, multiplicative Stratonovich Brownian noise preserves the hyperbolic structure while enabling entropy and $BV$0 estimates. In the stochastic heat equation, irregular perturbation acts through the averaged field $BV$1, converting distributional coefficients into a pathwise meaningful nonlinear Young–Volterra problem. In Hamiltonian averaging, the perturbation regularizes Fourier denominators. In deep networks, noise injection appears as an artificial viscosity in a backward Kolmogorov equation. In MPGD, chaotic perturbations produce a heavy-tailed limiting process and an explicit Hessian-penalizing term (Gassiat et al., 2016, Wei et al., 2017, Catellier et al., 2021, Bernardi et al., 2012, Sun et al., 2018, Lim et al., 2022).

Several distinctions are explicit in these works. First, SPR need not be probabilistic in the narrow sense. The perturbation may be a sufficiently irregular deterministic path, as in the regularization of the stochastic heat equation, or a deterministic chaotic signal, as in MPGD (Catellier et al., 2021, Lim et al., 2022). Second, the strongest results are often pathwise or sample-by-sample rather than distributional. This is emphasized for the Hamilton–Jacobi derivative bounds and for the well-posedness theory of the singular stochastic heat equation (Gassiat et al., 2016, Catellier et al., 2021). Third, not every perturbation regularizes in the same way. For nonlinear transport, the summary explicitly contrasts the multiplicative Brownian perturbation with additive or certain multiplicative noises. For stochastic ResNets, excessive regularization can oversmooth the landscape. For stochastic Hamilton–Jacobi and stochastic $BV$2-Laplace, there are sharp thresholds and optimal bounds, not merely qualitative improvements (Wei et al., 2017, Sun et al., 2018, Gassiat et al., 2016).

A plausible implication is that SPR is best regarded as a structural design principle: perturbations regularize when they interact with the governing nonlinearity in a way that modifies the effective equation, the averaging operator, or the optimization landscape. The provided works support this interpretation across SPDE, nonlinear PDE, dynamical systems, and machine learning, while also showing that the effect is regime-dependent, model-dependent, and often quantitatively constrained by reflected-SDE behavior, Besov/Hölder regularity, vanishing-parameter limits, or perturbation intensity thresholds (Gassiat et al., 2016, Catellier et al., 2021, Bernardi et al., 2012).