Regularization by Regular Noise

• Regularization by regular noise is a framework where structured, controlled noise (e.g., fractional Brownian, Gaussian, or bootstrap noise) is used to restore well-posedness and improve model stability.
• It leverages specific noise mechanisms to induce smoothing, averaging, and effective curvature control in applications ranging from stochastic differential equations to neural network training.
• Empirical and theoretical results confirm that this approach can optimize learning algorithms, enhance low-rank estimation, and stabilize quantum neural models.

Searching arXiv for recent and foundational papers on “regularization by regular noise” and adjacent uses of the term. Regularization by regular noise denotes a family of phenomena in which a deliberately specified noise mechanism improves stability, well-posedness, or generalization, even when the perturbation is itself highly structured or comparatively smooth. In the mathematical analysis of differential equations, the term is used for ill-posed ODEs, SDEs, and SPDEs whose behavior becomes well-posed after perturbation by fractional Brownian–type signals, transport noise, or time-modulated Hamilton–Jacobi noise (Gerencsér, 2020, Bechtold et al., 2023, Luo, 2021, Gassiat et al., 2016). In machine learning and inverse problems, the same phrase describes training or estimation schemes in which controlled noise injection induces an implicit or explicit regularizer, for example through uniform label noise, Gaussian input perturbations, bootstrap perturbations, or operator-adapted Gaussian-noise analysis [(Nakamura et al., 2020); (Rifai et al., 2011); (Josse et al., 2014); (Rothfuss et al., 2019); (Lang et al., 2023)]. Across these settings, the common structure is that a “regular” noise model is not treated as incidental corruption but as a mechanism that reshapes optimization, averaging, or effective dynamics.

1. Terminological scope and conceptual core

The phrase has at least two technical usages. In stochastic analysis and PDE theory, “regularization by noise” means that the presence of a stochastic perturbation improves well-posedness, existence, uniqueness, or regularity of an equation that is otherwise ill-posed or singular (Bechtold et al., 2023, Gassiat et al., 2016). In statistical learning, it refers to noise injection procedures that act as regularizers by smoothing the effective objective, discouraging overfitting, or stabilizing a low-rank or conditional-density estimator [(Rifai et al., 2011); (Josse et al., 2014); (Rothfuss et al., 2019)].

Within the differential-equation literature, a distinctive subtheme concerns perturbations that are smoother than the drift or smoother than Brownian motion. "Regularisation by regular noise" (Gerencsér, 2020) studies the equation

$X_t = \int_0^t b(X_r)\,dr + B_t^H,$

with $b \in C^\alpha$ and fractional Brownian–type noise $B^H$ for non-integer $H>1$, and shows strong uniqueness under the familiar condition

$\alpha > 1 - \frac{1}{2H}.$

This suggests that the regularizing effect is not reducible to pathwise roughness alone (Gerencsér, 2020). A related numerical note studies the same regime for $H \in (1,\infty)\setminus\mathbb{Z}$ and calls it “regularization by regular noise,” proving strong Euler–Maruyama convergence with rate $n^{-1}$ and identifying a non-trivial first-order error limit under $b\in\mathcal C^1$ (Song et al., 31 Oct 2025).

Within machine learning, “regular noise” usually means a controlled perturbation with a fixed law, such as Gaussian input noise, uniform label corruption, or a bootstrap noise model. In "Regularization in network optimization via trimmed stochastic gradient descent with noisy label" (Nakamura et al., 2020), the regular noise is uniform label noise applied every iteration. In "Adding noise to the input of a model trained with a regularized objective" (Rifai et al., 2011), the noise is isotropic Gaussian input noise. In "Bootstrap-Based Regularization for Low-Rank Matrix Estimation" (Josse et al., 2014), the noise model is encoded through a Lévy bootstrap that turns a specified data-generating perturbation into a quadratic penalty.

A plausible implication is that the phrase is best understood operationally rather than ontologically: the noise is “regular” when its law or path structure is specified well enough that one can identify the induced regularization mechanism.

2. Differential equations: well-posedness from smooth or structured perturbations

A central result is that smooth random perturbations can restore well-posedness for singular equations. "Regularisation by regular noise" (Gerencsér, 2020) proves that perturbing ill-posed differential equations with “potentially very smooth random processes” can restore well-posedness and recovers the condition $\alpha>1-\frac{1}{2H}$ for all non-integer $H>1$. The same source emphasizes that the perturbation may be “potentially much more regular than the drift component of the solution” (Gerencsér, 2020).

The 2025 numerical continuation of this theory studies

$b \in C^\alpha$0

with $b \in C^\alpha$1 and $b \in C^\alpha$2, $b \in C^\alpha$3, and proves that the Euler–Maruyama approximation $b \in C^\alpha$4 converges strongly to $b \in C^\alpha$5 with rate $b \in C^\alpha$6 (Song et al., 31 Oct 2025). Under $b \in C^\alpha$7, it further shows that $b \in C^\alpha$8 converges to a non-trivial limit, confirming that the rate $b \in C^\alpha$9 is optimal for that scheme (Song et al., 31 Oct 2025).

A distinct but related direction is "Pathwise regularization by noise for semilinear SPDEs driven by a multiplicative cylindrical Brownian motion" (Bechtold et al., 2023). There the SPDE is

$B^H$0

with a singular diffusion coefficient $B^H$1, a cylindrical Wiener process $B^H$2 on $B^H$3, and a one-dimensional fractional Brownian motion $B^H$4 inserted inside the diffusion argument. The paper proves weak existence under the assumption $B^H$5 and parameter restrictions linking $B^H$6, $B^H$7, and $B^H$8, obtaining solutions in

$B^H$9

for some $H>1$0 (Bechtold et al., 2023). The regularization mechanism is explicitly pathwise and relies on local time, maximal regularity, and Volterra-sewing techniques (Bechtold et al., 2023).

The same conceptual theme appears in scalar conservation laws and stochastic Hamilton–Jacobi equations. "Path-by-path regularization by noise for scalar conservation laws" (Chouk et al., 2017) proves that if the driving path is $H>1$1-irregular, then bounded quasi-solutions gain spatial Sobolev regularity $H>1$2, and for fractional Brownian motion obtains

$H>1$3

in the 1D Burgers case for $H>1$4 (Chouk et al., 2017). "Regularization by noise for stochastic Hamilton-Jacobi equations" (Gassiat et al., 2016) proves path-by-path $H>1$5 bounds on second derivatives, expressed through reflected SDEs for curvature parameters $H>1$6, and states these bounds are optimal (Gassiat et al., 2016).

These results share a technical pattern. The perturbation enters an equation with low drift regularity or singular nonlinear structure; the noise induces averaging, smoothing through conditional laws, or effective curvature control; and the outcome is formulated as strong well-posedness, weak existence, pathwise uniqueness, or explicit derivative bounds (Gerencsér, 2020, Bechtold et al., 2023, Chouk et al., 2017, Gassiat et al., 2016).

3. Transport, kinetic, and fluid-dynamical variants

Not all regularization-by-noise mechanisms rely on additive perturbations. "Regularization by transport noise for 3D MHD equations" (Luo, 2021) studies the vorticity form of the incompressible 3D MHD system on $H>1$7, perturbed by multiplicative transport noise. In Stratonovich form, the stochastic vorticity equation contains

$H>1$8

and the associated Itô correction $H>1$9 converges in a scaling limit to $\alpha > 1 - \frac{1}{2H}.$0 (Luo, 2021). The main theorem states that for every radius $\alpha > 1 - \frac{1}{2H}.$1 and small $\alpha > 1 - \frac{1}{2H}.$2, there exist large $\alpha > 1 - \frac{1}{2H}.$3 and $\alpha > 1 - \frac{1}{2H}.$4 such that the stochastic 3D MHD equations admit a pathwise unique global solution for all initial data in the $\alpha > 1 - \frac{1}{2H}.$5-ball $\alpha > 1 - \frac{1}{2H}.$6 with probability greater than $\alpha > 1 - \frac{1}{2H}.$7 (Luo, 2021).

This transport-noise mechanism differs from the additive fractional-noise setting. The noise is energy-conserving in Stratonovich form, yet its Itô correction creates effective diffusion. A plausible implication is that “regular” transport noise regularizes by producing an averaged parabolic drift rather than by adding direct rough forcing (Luo, 2021).

The Burgers-type SPDE literature contains a further variant. "Regularization by noise and stochastic Burgers equations" (Gubinelli et al., 2012) studies

$\alpha > 1 - \frac{1}{2H}.$8

with $\alpha > 1 - \frac{1}{2H}.$9, space-time white noise $H \in (1,\infty)\setminus\mathbb{Z}$0, and white-noise initial data (Gubinelli et al., 2012). The paper introduces stationary weak solutions in a controlled-process framework, shows that the noise provides a regularizing effect allowing existence and suitable estimates when $H \in (1,\infty)\setminus\mathbb{Z}$1, and proves pathwise uniqueness when $H \in (1,\infty)\setminus\mathbb{Z}$2 (Gubinelli et al., 2012). Here the regularization acts not by pointwise smoothing of trajectories but by making the time-integrated nonlinear drift $H \in (1,\infty)\setminus\mathbb{Z}$3 well-defined in a weak stationary sense (Gubinelli et al., 2012).

The kinetic-SDE direction is represented in the supplied material by "Strong regularization by noise for kinetic SDEs" (Lucertini et al., 2022), but the available text explicitly states that exact theorems, hypotheses, and notation were not accessible. The only concrete claims that can be retained are those in the arXiv stub: the paper proves strong well-posedness for a system of stochastic differential equations driven by a degenerate diffusion satisfying a weak-type Hörmander condition, under Hölder regularity assumptions on the drift coefficient, and interprets this as a regularization-by-noise phenomenon (Lucertini et al., 2022).

4. Neural-network training: explicit noise injection as implicit regularization

In supervised deep learning, the phrase describes procedures where a prescribed noise source modifies optimization in a regularizing direction. "Regularization in network optimization via trimmed stochastic gradient descent with noisy label" (Nakamura et al., 2020) formulates standard supervised classification with empirical risk

$H \in (1,\infty)\setminus\mathbb{Z}$4

and SGD update

$H \in (1,\infty)\setminus\mathbb{Z}$5

The label noise is defined by

$H \in (1,\infty)\setminus\mathbb{Z}$6

where $H \in (1,\infty)\setminus\mathbb{Z}$7 is Bernoulli with $H \in (1,\infty)\setminus\mathbb{Z}$8 and $H \in (1,\infty)\setminus\mathbb{Z}$9 is uniform over labels (Nakamura et al., 2020). The noisy-label objective becomes a mixture of the true label loss and a uniform-label loss,

$n^{-1}$0

which the paper interprets as implicit regularization (Nakamura et al., 2020).

The same paper argues that label noise alone creates extremely high-loss outliers and misleading gradients, and therefore proposes Label-Noised Trim-SGD, which trims high-loss and low-loss examples based on their rank within each mini-batch (Nakamura et al., 2020). The trimming rule is

$n^{-1}$1

with remaining batch size $n^{-1}$2, and the update averages gradients over $n^{-1}$3 (Nakamura et al., 2020). The paper reports that on MNIST, Fashion-MNIST, and EMNIST-Letters, the method attains lower mean and minimum test loss than SGD, RMSprop, Adam, Entropy-SGD, and Accelerated-SGD for most dataset-model combinations, while also noting that on CIFAR-10 it was inferior to baseline SGD (Nakamura et al., 2020).

A more classical derivation appears in "Adding noise to the input of a model trained with a regularized objective" (Rifai et al., 2011). There, noisy inputs are modeled by $n^{-1}$4 with $n^{-1}$5, and the noisy objective is

$n^{-1}$6

A second-order Taylor expansion yields

$n^{-1}$7

which identifies additive Gaussian input noise with a trace-Hessian penalty on the loss (Rifai et al., 2011). For mean-squared error, the induced regularization decomposes into Jacobian and Hessian terms of the mapping $n^{-1}$8, and if one adds noise inside a Jacobian-regularized objective, the resulting approximation contains a Hessian norm penalty with coefficient $n^{-1}$9 (Rifai et al., 2011).

The conditional-density literature uses a related mechanism. "Noise Regularization for Conditional Density Estimation" (Rothfuss et al., 2019) studies neural conditional density models trained by conditional maximum likelihood and perturbs both $b\in\mathcal C^1$0 and $b\in\mathcal C^1$1 during training. The paper shows that this corresponds to a smoothness regularization on $b\in\mathcal C^1$2, proves asymptotic consistency when the noise level decays according to $b\in\mathcal C^1$3 and $b\in\mathcal C^1$4, and reports that the method “significantly and consistently outperforms other regularization methods across seven data sets and three CDE models” (Rothfuss et al., 2019).

The supplied material also includes "Effect of Various Regularizers on Model Complexities of Neural Networks in Presence of Input Noise" (Sharma et al., 2019), but the detailed block is truncated after the shallow-network heading. The concrete accessible claims are therefore limited to the abstract: $b\in\mathcal C^1$5 regularization leads to a simpler hypothesis class and better generalization followed by DARC1, Jacobian regularization works well for shallow architectures with high level of input noises, spectral normalization attains highest test set accuracies, dropout alone does not perform well in presence of input noise, and deeper architectures are robust to input noise as opposed to their shallow counterparts (Sharma et al., 2019).

5. Low-rank estimation and inverse problems: converting noise models into penalties

In matrix estimation and inverse problems, “regularization by regular noise” often means that a specified noise law determines the regularizer itself. "Bootstrap-Based Regularization for Low-Rank Matrix Estimation" (Josse et al., 2014) observes a noisy matrix $b\in\mathcal C^1$6 with $b\in\mathcal C^1$7 and seeks a low-rank estimate of $b\in\mathcal C^1$8. The classical rank-$b\in\mathcal C^1$9 autoencoder is written as

$\alpha>1-\frac{1}{2H}$0

The paper defines a stable autoencoder by replacing $\alpha>1-\frac{1}{2H}$1 with a bootstrap perturbation $\alpha>1-\frac{1}{2H}$2,

$\alpha>1-\frac{1}{2H}$3

and shows that this is equivalent to

$\alpha>1-\frac{1}{2H}$4

where $\alpha>1-\frac{1}{2H}$5 is diagonal with entries derived from $\alpha>1-\frac{1}{2H}$6 (Josse et al., 2014). In the isotropic Gaussian case, $\alpha>1-\frac{1}{2H}$7 and the procedure reduces to a singular-value shrinker

$\alpha>1-\frac{1}{2H}$8

while for non-isotropic noise such as Poisson bootstrap, the estimator is no longer simple singular-value shrinkage and also rotates singular vectors (Josse et al., 2014). This suggests that the regularizer is not chosen separately from the noise model; it is the noise model, analytically transformed.

The inverse-problem version of this principle appears in "Small noise analysis for Tikhonov and RKHS regularizations" (Lang et al., 2023). In the linear inverse model

$\alpha>1-\frac{1}{2H}$9

the averaged operator is

$H>1$0

and the Gaussian noise in parameter space satisfies

$H>1$1

(Lang et al., 2023). The paper defines the fractional RKHS scale

$H>1$2

and studies the regularized estimator

$H>1$3

The analysis shows that conventional $H>1$4-regularization can be unstable in the small-noise limit, whereas adaptive fractional RKHS regularizers aligned with $H>1$5 can achieve optimal convergence rates, with the striking conclusion that over-smoothing consistently yields optimal convergence rates, although the optimal hyper-parameter may decay too fast to be selected in practice (Lang et al., 2023).

A plausible implication is that in ill-posed estimation, “regular noise” does not merely contaminate observations; it determines the correct geometry of the penalty through its covariance structure.

6. Quantum and cross-domain extensions

The same regularization logic has recently been imported into quantum machine learning. "Method for noise-induced regularization in quantum neural networks" (Somogyi et al., 2024) studies variational quantum circuits with standard noise channels inserted after each gate on all qubits. The paper considers amplitude damping, phase damping, and depolarizing channels with strength parameter $H>1$6, trains a 4-qubit QNN on a medical regression task derived from the diabetes dataset, and reports that tuning $H>1$7 yields up to an 8% improvement in mean squared error relative to the noiseless model (Somogyi et al., 2024). The empirical signature is classical: training MSE increases monotonically with $H>1$8, while validation MSE exhibits a minimum at nonzero $H>1$9 (Somogyi et al., 2024).

This quantum example is structurally parallel to the classical Gaussian-noise results [(Rifai et al., 2011); (Rothfuss et al., 2019)], even though the perturbation acts through Kraus channels instead of additive Euclidean noise. In both cases, the perturbation is controlled, stationary, and treated as part of the model class rather than as an external nuisance.

Across all the supplied work, several recurring mechanisms appear:

Setting	Noise mechanism	Regularization effect
Singular ODE/SDE with $b \in C^\alpha$00	Fractional Brownian–type additive perturbation	Strong well-posedness under $b \in C^\alpha$01 (Gerencsér, 2020)
Semilinear SPDE	Cylindrical Brownian motion with $b \in C^\alpha$02 inside diffusion	Weak existence via pathwise local-time smoothing (Bechtold et al., 2023)
Stochastic Hamilton–Jacobi	Time signal in $b \in C^\alpha$03	Pathwise Hessian bounds via reflected SDEs (Gassiat et al., 2016)
Network optimization	Uniform label noise plus trimming	Improved generalization on several grayscale benchmarks (Nakamura et al., 2020)
Neural objectives	Gaussian input noise	Trace-Hessian, Jacobian, and Hessian penalties (Rifai et al., 2011)
Low-rank matrix estimation	Bootstrap perturbation $b \in C^\alpha$04	Variance-weighted quadratic penalty (Josse et al., 2014)
Linear inverse problems	Gaussian noise with covariance $b \in C^\alpha$05	Fractional RKHS penalties aligned with $b \in C^\alpha$06 (Lang et al., 2023)
Quantum neural networks	Amplitude/phase damping, depolarizing channels	Validation-error minimum at nonzero noise (Somogyi et al., 2024)

The most common misconception is that regularization by noise requires highly irregular sample paths. The material on fractional Brownian type noise with $b \in C^\alpha$07, transport noise, and Hamilton–Jacobi perturbations shows that the relevant feature may instead be conditional Gaussian smoothing, local-time structure, reflected-curvature dynamics, or Itô correction under scaling (Gerencsér, 2020, Bechtold et al., 2023, Luo, 2021, Gassiat et al., 2016). A second misconception is that noise-based regularization is inherently heuristic in machine learning. The Gaussian-input, conditional-density, bootstrap, and fractional-RKHS analyses all derive explicit objective-level penalties or asymptotic risk characterizations from the perturbation law itself [(Rifai et al., 2011); (Rothfuss et al., 2019); (Josse et al., 2014); (Lang et al., 2023)].

Taken together, these works support a precise interpretation: regularization by regular noise is a design principle in which the perturbation distribution, path structure, or channel model is chosen so that the induced averaging, trimming, spectral damping, or curvature control compensates for ill-posedness, overconfidence, or operator instability. This suggests a unifying view across stochastic analysis and statistical learning: the right noise is not merely randomness added to a system, but a structured operator acting on the effective hypothesis class or dynamics.