---
title: Regularization by Regular Noise
url: https://www.emergentmind.com/topics/regularization-by-regular-noise
type: topic
---

# Regularization by Regular Noise

Searching arXiv for recent and foundational papers on “regularization by regular noise” and adjacent uses of the term.
Regularization by regular noise denotes a family of phenomena in which a deliberately specified noise mechanism improves stability, well-posedness, or generalization, even when the perturbation is itself highly structured or comparatively smooth. In the mathematical analysis of differential equations, the term is used for ill-posed ODEs, SDEs, and SPDEs whose behavior becomes well-posed after perturbation by fractional Brownian–type signals, transport noise, or time-modulated Hamilton–Jacobi noise [2009.08418; 2311.00764; 2107.00190; 1609.07074]. In machine learning and inverse problems, the same phrase describes training or estimation schemes in which controlled noise injection induces an implicit or explicit regularizer, for example through uniform label noise, Gaussian input perturbations, bootstrap perturbations, or operator-adapted Gaussian-noise analysis [2012.11073; 1104.3250; 1410.8275; 1907.08982; 2305.11055]. Across these settings, the common structure is that a “regular” noise model is not treated as incidental corruption but as a mechanism that reshapes optimization, averaging, or effective dynamics.

## 1. Terminological scope and conceptual core

The phrase has at least two technical usages. In stochastic analysis and PDE theory, “regularization by noise” means that the presence of a stochastic perturbation improves well-posedness, existence, uniqueness, or regularity of an equation that is otherwise ill-posed or singular [2311.00764; 1609.07074]. In statistical learning, it refers to noise injection procedures that act as regularizers by smoothing the effective objective, discouraging overfitting, or stabilizing a low-rank or conditional-density estimator [1104.3250; 1410.8275; 1907.08982].

Within the differential-equation literature, a distinctive subtheme concerns perturbations that are smoother than the drift or smoother than Brownian motion. "Regularisation by regular noise" [2009.08418] studies the equation
\[
X_t = \int_0^t b(X_r)\,dr + B_t^H,
\]
with \(b \in C^\alpha\) and fractional Brownian–type noise \(B^H\) for non-integer \(H>1\), and shows strong uniqueness under the familiar condition
\[
\alpha > 1 - \frac{1}{2H}.
\]
This suggests that the regularizing effect is not reducible to pathwise roughness alone [2009.08418]. A related numerical note studies the same regime for \(H \in (1,\infty)\setminus\mathbb{Z}\) and calls it “regularization by regular noise,” proving strong Euler–Maruyama convergence with rate \(n^{-1}\) and identifying a non-trivial first-order error limit under \(b\in\mathcal C^1\) [2510.27225].

Within machine learning, “regular noise” usually means a controlled perturbation with a fixed law, such as Gaussian input noise, uniform label corruption, or a bootstrap noise model. In "Regularization in network optimization via trimmed stochastic gradient descent with noisy label" [2012.11073], the regular noise is uniform label noise applied every iteration. In "Adding noise to the input of a model trained with a regularized objective" [1104.3250], the noise is isotropic Gaussian input noise. In "Bootstrap-Based Regularization for Low-Rank Matrix Estimation" [1410.8275], the noise model is encoded through a Lévy bootstrap that turns a specified data-generating perturbation into a quadratic penalty.

A plausible implication is that the phrase is best understood operationally rather than ontologically: the noise is “regular” when its law or path structure is specified well enough that one can identify the induced regularization mechanism.

## 2. Differential equations: well-posedness from smooth or structured perturbations

A central result is that smooth random perturbations can restore well-posedness for singular equations. "Regularisation by regular noise" [2009.08418] proves that perturbing ill-posed differential equations with “potentially very smooth random processes” can restore well-posedness and recovers the condition \(\alpha>1-\frac{1}{2H}\) for all non-integer \(H>1\). The same source emphasizes that the perturbation may be “potentially much more regular than the drift component of the solution” [2009.08418].

The 2025 numerical continuation of this theory studies
\[
\mathrm{d}X_t = b(X_t)\,\mathrm{d}t + \mathrm{d}B_t^H,\qquad X_0=x_0\in\mathbb{R}^d,
\]
with \(H\in(1,\infty)\setminus\mathbb{Z}\) and \(b\in \mathcal C^\alpha\), \(\alpha\in(1-\frac1{2H},1]\), and proves that the Euler–Maruyama approximation \(X^n\) converges strongly to \(X\) with rate \(n^{-1}\) [2510.27225]. Under \(b\in \mathcal C^1\), it further shows that \(n(X-X^n)\) converges to a non-trivial limit, confirming that the rate \(n^{-1}\) is optimal for that scheme [2510.27225].

A distinct but related direction is "Pathwise regularization by noise for semilinear SPDEs driven by a multiplicative cylindrical Brownian motion" [2311.00764]. There the SPDE is
\[
du_t = \Delta u_t\,dt + \sigma(u_t - w_t^H)\,dW_t,
\]
with a singular diffusion coefficient \(\sigma\), a cylindrical Wiener process \(W\) on \(L^2_x\), and a one-dimensional fractional Brownian motion \(w^H\) inserted inside the diffusion argument. The paper proves weak existence under the assumption \(\Sigma^2\in L^p(\mathbb{R})\) and parameter restrictions linking \(p\), \(H\), and \(\gamma_0\), obtaining solutions in
\[
L^m(\Omega; C^{\gamma_0}([0,T];L^2_x)) \cap L^m(\Omega;L^\infty([0,T];H^{\gamma_0}_x))
\]
for some \(m\ge2\) [2311.00764]. The regularization mechanism is explicitly pathwise and relies on local time, maximal regularity, and Volterra-sewing techniques [2311.00764].

The same conceptual theme appears in scalar conservation laws and stochastic Hamilton–Jacobi equations. "Path-by-path regularization by noise for scalar conservation laws" [1708.00823] proves that if the driving path is \((\rho,\gamma)\)-irregular, then bounded quasi-solutions gain spatial Sobolev regularity \(W^{\lambda,1}\), and for fractional Brownian motion obtains
\[
u\in L^1([0,T];W^{\lambda,1}(\mathbb{T})),\qquad \forall \lambda<\frac{1}{1+2H},
\]
in the 1D Burgers case for \(H\le \frac12\) [1708.00823]. "Regularization by noise for stochastic Hamilton-Jacobi equations" [1609.07074] proves path-by-path \(L^\infty\) bounds on second derivatives, expressed through reflected SDEs for curvature parameters \(L^\pm\), and states these bounds are optimal [1609.07074].

These results share a technical pattern. The perturbation enters an equation with low drift regularity or singular nonlinear structure; the noise induces averaging, smoothing through conditional laws, or effective curvature control; and the outcome is formulated as strong well-posedness, weak existence, pathwise uniqueness, or explicit derivative bounds [2009.08418; 2311.00764; 1708.00823; 1609.07074].

## 3. Transport, kinetic, and fluid-dynamical variants

Not all regularization-by-noise mechanisms rely on additive perturbations. "Regularization by transport noise for 3D MHD equations" [2107.00190] studies the vorticity form of the incompressible 3D MHD system on \(\mathbb T^3\), perturbed by multiplicative transport noise. In Stratonovich form, the stochastic vorticity equation contains
\[
C_\nu\sum_{k,a}\theta_k\,\Pi(\sigma_{k,a}\cdot\nabla\Phi)\circ dW_{k,a}(t),
\]
and the associated Itô correction \(\mathcal S_\theta(\Phi)\) converges in a scaling limit to \(\nu\Delta\Phi\) [2107.00190]. The main theorem states that for every radius \(K>0\) and small \(\varepsilon>0\), there exist large \(\nu>0\) and \(\theta\in\ell^2\) such that the stochastic 3D MHD equations admit a pathwise unique global solution for all initial data in the \(H\)-ball \(B_H(K)\) with probability greater than \(1-\varepsilon\) [2107.00190].

This transport-noise mechanism differs from the additive fractional-noise setting. The noise is energy-conserving in Stratonovich form, yet its Itô correction creates effective diffusion. A plausible implication is that “regular” transport noise regularizes by producing an averaged parabolic drift rather than by adding direct rough forcing [2107.00190].

The Burgers-type SPDE literature contains a further variant. "Regularization by noise and stochastic Burgers equations" [1208.6551] studies
\[
\partial_t u = - A^\theta u + \partial_x u^2 + A^{\theta/2} \xi
\]
with \(\theta>1/2\), space-time white noise \(\xi\), and white-noise initial data [1208.6551]. The paper introduces stationary weak solutions in a controlled-process framework, shows that the noise provides a regularizing effect allowing existence and suitable estimates when \(\theta>1/2\), and proves pathwise uniqueness when \(\theta>5/4\) [1208.6551]. Here the regularization acts not by pointwise smoothing of trajectories but by making the time-integrated nonlinear drift \(\int_0^t \partial_x u_s^2\,ds\) well-defined in a weak stationary sense [1208.6551].

The kinetic-SDE direction is represented in the supplied material by "Strong regularization by noise for kinetic SDEs" [2207.09726], but the available text explicitly states that exact theorems, hypotheses, and notation were not accessible. The only concrete claims that can be retained are those in the arXiv stub: the paper proves strong well-posedness for a system of stochastic differential equations driven by a degenerate diffusion satisfying a weak-type Hörmander condition, under Hölder regularity assumptions on the drift coefficient, and interprets this as a regularization-by-noise phenomenon [2207.09726].

## 4. Neural-network training: explicit noise injection as implicit regularization

In supervised deep learning, the phrase describes procedures where a prescribed noise source modifies optimization in a regularizing direction. "Regularization in network optimization via trimmed stochastic gradient descent with noisy label" [2012.11073] formulates standard supervised classification with empirical risk
\[
F(w) = \frac{1}{n}\sum_{i=1}^n \ell(h_w(x_i), y_i)
\]
and SGD update
\[
w^{t+1} = w^{t} - \eta^t \left( \frac{1}{B} \sum_{i \in \beta^t} \nabla f_i(w^{t}) \right).
\]
The label noise is defined by
\[
\hat{y}_i \coloneqq G(y_i; P) = \begin{cases} y_i & \text{if } \chi_i(P) = 1, \\ \Theta(L) & \text{if } \chi_i(P) = 0, \end{cases}
\]
where \(\chi_i(P)\) is Bernoulli with \(\Pr(\chi_i=1)=P\) and \(\Theta(L)\) is uniform over labels [2012.11073]. The noisy-label objective becomes a mixture of the true label loss and a uniform-label loss,
\[
P\, \ell(h_w(x_i), y_i) + (1-P)\,\frac{1}{L}\sum_{k=1}^L \ell(h_w(x_i), k),
\]
which the paper interprets as implicit regularization [2012.11073].

The same paper argues that label noise alone creates extremely high-loss outliers and misleading gradients, and therefore proposes Label-Noised Trim-SGD, which trims high-loss and low-loss examples based on their rank within each mini-batch [2012.11073]. The trimming rule is
\[
\hat{\beta} \coloneqq H(\beta; Q)
= \left\{ j \;\middle|\; \frac{Q}{2} \cdot B < \mathrm{order}(j) < (1 - \frac{Q}{2}) \cdot B \right\},
\]
with remaining batch size \(\hat B=(1-Q)B\), and the update averages gradients over \(\hat\beta^t\) [2012.11073]. The paper reports that on MNIST, Fashion-MNIST, and EMNIST-Letters, the method attains lower mean and minimum test loss than SGD, RMSprop, Adam, Entropy-SGD, and Accelerated-SGD for most dataset-model combinations, while also noting that on CIFAR-10 it was inferior to baseline SGD [2012.11073].

A more classical derivation appears in "Adding noise to the input of a model trained with a regularized objective" [1104.3250]. There, noisy inputs are modeled by \(z=z_i+\varepsilon\) with \(\varepsilon\sim \mathcal N(0,\sigma^2 I)\), and the noisy objective is
\[
\mathcal{C}^{\text{noisy}}(\theta)
= \frac{1}{n}\sum_{i=1}^n \mathbb{E}_{\varepsilon}\big[\mathcal{L}(z_i + \varepsilon,\theta)\big].
\]
A second-order Taylor expansion yields
\[
\mathcal{C}^{\text{noisy}}(\theta)
\approx \mathcal{C}^{\text{clean}}(\theta) + \frac{\sigma^2}{2n}\sum_{i=1}^n \mathrm{tr}\big(H_{\mathcal{L}(z_i,\theta)}\big),
\]
which identifies additive Gaussian input noise with a trace-Hessian penalty on the loss [1104.3250]. For mean-squared error, the induced regularization decomposes into Jacobian and Hessian terms of the mapping \(F_\theta\), and if one adds noise inside a Jacobian-regularized objective, the resulting approximation contains a Hessian norm penalty with coefficient \(2\lambda\sigma^2\) [1104.3250].

The conditional-density literature uses a related mechanism. "Noise Regularization for Conditional Density Estimation" [1907.08982] studies neural conditional density models trained by conditional maximum likelihood and perturbs both \(x\) and \(y\) during training. The paper shows that this corresponds to a smoothness regularization on \(\log \hat f_\theta(y\mid x)\), proves asymptotic consistency when the noise level decays according to \(h_n\to0\) and \(n h_n^d\to\infty\), and reports that the method “significantly and consistently outperforms other regularization methods across seven data sets and three CDE models” [1907.08982].

The supplied material also includes "Effect of Various Regularizers on Model Complexities of Neural Networks in Presence of Input Noise" [1901.11458], but the detailed block is truncated after the shallow-network heading. The concrete accessible claims are therefore limited to the abstract: \(L_2\) regularization leads to a simpler hypothesis class and better generalization followed by DARC1, Jacobian regularization works well for shallow architectures with high level of input noises, spectral normalization attains highest test set accuracies, dropout alone does not perform well in presence of input noise, and deeper architectures are robust to input noise as opposed to their shallow counterparts [1901.11458].

## 5. Low-rank estimation and inverse problems: converting noise models into penalties

In matrix estimation and inverse problems, “regularization by regular noise” often means that a specified noise law determines the regularizer itself. "Bootstrap-Based Regularization for Low-Rank Matrix Estimation" [1410.8275] observes a noisy matrix \(X\in\mathbb R^{n\times p}\) with \(\mathbb E[X]=\mu\) and seeks a low-rank estimate of \(\mu\). The classical rank-\(k\) autoencoder is written as
\[
\hat\mu_k = X B_k,\quad
B_k = \arg\min_{B}\{\|X - X B\|_F^2 : \operatorname{rank}(B) \le k\}.
\]
The paper defines a stable autoencoder by replacing \(X\) with a bootstrap perturbation \(\tilde X\sim\tilde{\mathcal L}(X)\),
\[
\hat B_k = \arg\min_{B} \left\{ \mathbb{E}_{\tilde X\sim \tilde{\mathcal L}(X)} \|X - \tilde X B\|_F^2 : \operatorname{rank}(B)\le k \right\},
\]
and shows that this is equivalent to
\[
\hat B_k = \arg\min_B \left\{\|X - X B\|_F^2 + \|S^{1/2} B\|_F^2 : \operatorname{rank}(B)\le k\right\},
\]
where \(S\) is diagonal with entries derived from \(\operatorname{Var}(\tilde X_{ij})\) [1410.8275]. In the isotropic Gaussian case, \(S=\lambda I\) and the procedure reduces to a singular-value shrinker
\[
\psi(d) = \frac{d}{1+\lambda/d^2},
\]
while for non-isotropic noise such as Poisson bootstrap, the estimator is no longer simple singular-value shrinkage and also rotates singular vectors [1410.8275]. This suggests that the regularizer is not chosen separately from the noise model; it is the noise model, analytically transformed.

The inverse-problem version of this principle appears in "Small noise analysis for Tikhonov and RKHS regularizations" [2305.11055]. In the linear inverse model
\[
y_k = L_k \theta^\* + \sigma W_k,\qquad k=1,\dots,n,
\]
the averaged operator is
\[
L_G = \frac1n \sum_{k=1}^n L_k^* L_k,
\]
and the Gaussian noise in parameter space satisfies
\[
\theta_y = L_G \theta^\* + \sigma \eta,\quad \eta\sim\mathcal N(0,L_G)
\]
[2305.11055]. The paper defines the fractional RKHS scale
\[
H_s := \mathcal D(L_G^{s/2}),\qquad \|\theta\|_{H_s}^2 = \|L_G^{s/2}\theta\|_{L^2}^2,
\]
and studies the regularized estimator
\[
\hat{\theta}_{\lambda,s}
= \arg\min_{\theta\in H_s} \Bigl( \mathcal E(\theta) + \lambda \|\theta\|_{H_s}^2 \Bigr).
\]
The analysis shows that conventional \(L^2\)-regularization can be unstable in the small-noise limit, whereas adaptive fractional RKHS regularizers aligned with \(L_G\) can achieve optimal convergence rates, with the striking conclusion that over-smoothing consistently yields optimal convergence rates, although the optimal hyper-parameter may decay too fast to be selected in practice [2305.11055].

A plausible implication is that in ill-posed estimation, “regular noise” does not merely contaminate observations; it determines the correct geometry of the penalty through its covariance structure.

## 6. Quantum and cross-domain extensions

The same regularization logic has recently been imported into quantum machine learning. "Method for noise-induced regularization in quantum neural networks" [2410.19921] studies variational quantum circuits with standard noise channels inserted after each gate on all qubits. The paper considers amplitude damping, phase damping, and depolarizing channels with strength parameter \(\gamma\), trains a 4-qubit QNN on a medical regression task derived from the diabetes dataset, and reports that tuning \(\gamma\) yields up to an 8% improvement in mean squared error relative to the noiseless model [2410.19921]. The empirical signature is classical: training MSE increases monotonically with \(\gamma\), while validation MSE exhibits a minimum at nonzero \(\gamma\) [2410.19921].

This quantum example is structurally parallel to the classical Gaussian-noise results [1104.3250; 1907.08982], even though the perturbation acts through Kraus channels instead of additive Euclidean noise. In both cases, the perturbation is controlled, stationary, and treated as part of the model class rather than as an external nuisance.

Across all the supplied work, several recurring mechanisms appear:

| Setting | Noise mechanism | Regularization effect |
|---|---|---|
| Singular ODE/SDE with \(B^H\) | Fractional Brownian–type additive perturbation | Strong well-posedness under \(\alpha>1-\frac{1}{2H}\) [2009.08418] |
| Semilinear SPDE | Cylindrical Brownian motion with \(w^H\) inside diffusion | Weak existence via pathwise local-time smoothing [2311.00764] |
| Stochastic Hamilton–Jacobi | Time signal in \(\frac12|Du|^2\circ d\xi\) | Pathwise Hessian bounds via reflected SDEs [1609.07074] |
| Network optimization | Uniform label noise plus trimming | Improved generalization on several grayscale benchmarks [2012.11073] |
| Neural objectives | Gaussian input noise | Trace-Hessian, Jacobian, and Hessian penalties [1104.3250] |
| Low-rank matrix estimation | Bootstrap perturbation \(\tilde X\) | Variance-weighted quadratic penalty [1410.8275] |
| Linear inverse problems | Gaussian noise with covariance \(L_G\) | Fractional RKHS penalties aligned with \(L_G\) [2305.11055] |
| Quantum neural networks | Amplitude/phase damping, depolarizing channels | Validation-error minimum at nonzero noise [2410.19921] |

The most common misconception is that regularization by noise requires highly irregular sample paths. The material on fractional Brownian type noise with \(H>1\), transport noise, and Hamilton–Jacobi perturbations shows that the relevant feature may instead be conditional Gaussian smoothing, local-time structure, reflected-curvature dynamics, or Itô correction under scaling [2009.08418; 2311.00764; 2107.00190; 1609.07074]. A second misconception is that noise-based regularization is inherently heuristic in machine learning. The Gaussian-input, conditional-density, bootstrap, and fractional-RKHS analyses all derive explicit objective-level penalties or asymptotic risk characterizations from the perturbation law itself [1104.3250; 1907.08982; 1410.8275; 2305.11055].

Taken together, these works support a precise interpretation: regularization by regular noise is a design principle in which the perturbation distribution, path structure, or channel model is chosen so that the induced averaging, trimming, spectral damping, or curvature control compensates for ill-posedness, overconfidence, or operator instability. This suggests a unifying view across stochastic analysis and statistical learning: the right noise is not merely randomness added to a system, but a structured operator acting on the effective hypothesis class or dynamics.

Source: https://www.emergentmind.com/topics/regularization-by-regular-noise