Ridge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularization

Abstract: We connect stochastic resetting from non-equilibrium statistical physics with ridge regularization in statistical learning. For linear gradient flow, resetting to the origin at rate $r$ produces stationary mean $(X^\top X+rI)^{-1}X^\top y$, exactly the ridge estimator with penalty $λ=r$. This uses the known Laplace-transform relationship between ridge regression and exponential-time averaging of gradient flow, with the exponential time now interpreted as the stationary age associated with Poisson resetting. We then extend this identity to general renewal reset laws: the exponential reset time distribution is the unique renewal law whose stationary mean reproduces scalar ridge in every eigendirection as an exact filter identity for every positive curvature, while non-exponential renewal laws generate alternative spectral filters. At the fluctuation level, we study a separate additive Ornstein-Uhlenbeck extension with constant diffusion, interpreted as a stylized SGD approximation. In this setting, the equality holds only at the level of the mean, since the reset process has a nonzero stationary covariance from accumulated OU noise and reset-timing variance, whereas deterministic ridge is a fixed estimator with the same center. Stylized experiments compare the deterministic renewal-induced filters directly and illustrate when filters induced by non-exponential reset-time laws can differ predictively from ridge. The results for the stationary mean and the induced spectral filters are established for continuous-time gradient flow with isotropic resetting on quadratic objectives; the covariance and risk formulas additionally assume additive noise with state-independent covariance.

• The paper demonstrates that Poisson resetting of gradient flow yields exactly the ridge estimator for quadratic objectives.
• It introduces a spectral regularization framework where different renewal resetting laws produce distinct shrinkage filters, with only the exponential law matching ridge regression exactly.
• Empirical evaluations in high-dimensional settings reveal that while stochastic resets add extra variance, non-exponential filters can selectively improve prediction performance.

Ridge Regression via Poisson Resetting and Renewal: A Spectral Regularization Framework

Introduction and Context

The paper "Ridge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularization" (2605.30059) advances the conceptual and technical connection between spectral regularization in linear models and stochastic resetting procedures rooted in non-equilibrium statistical physics. It formalizes how the stationary mean of continuous-time gradient flow for least squares, periodically interrupted via Poisson resetting, recovers exactly the ridge estimator with penalty $\lambda = r$, where $r$ is the reset rate. This identification holds at the level of the mean for quadratic objectives. The work extends this framework to general renewal resetting schemes, analyzes the induced spectral regularization filters, and quantifies the fluctuation-driven variance and total estimation risk arising from stochastic resets and gradient noise. Theoretical results are complemented with high-dimensional empirical validation in structured covariance models.

Poisson Resetting and the Ridge Estimator

The core analytic claim is that if unregularized gradient flow is intermittently reset to the origin at times drawn from a Poisson process with rate $r$, the stationary mean of this process, $m_\infty = (X^\top X + rI)^{-1} X^\top y$, coincides exactly with the ridge regression estimator for penalty $\lambda = r$. This equivalence follows from the Laplace-transform relationship between the resolvent and exponential-time averaging along the gradient flow path. In the spectral domain, this produces classical ridge shrinkage in each eigendirection: modes with small curvature (weak identification) are heavily shrunk, those with large curvature are nearly preserved.

Figure 1: Gradient-flow trajectories in a two-parameter linear regression under Poisson resetting, illustrating ensemble mean tracking the ridge solution.

Figure 1 demonstrates how the ensemble mean of sample paths converges to the ridge solution, whereas individual sample paths fluctuate due to stochastic resets. The effect is strongest in weakly identified directions, in accordance with the spectral interpretation.

Renewal Resetting and Admissible Spectral Filters

The reset-to-ridge correspondence is unique to Poisson (exponential) resetting. More generally, if reset intervals are drawn from an arbitrary renewal law $S$, the stationary mean is a spectral regularizer with a law-dependent filter $g_S(\mu)$ at curvature $\mu$. Only the exponential distribution yields exact ridge shrinkage $g(\mu) = \mu/(\mu + \lambda)$ for all $\mu > 0$. Deterministic periodic resets and Gamma/Erlang reset laws interpolate between Poisson and periodic limiting cases, producing sharper or smoother shrinkage profiles that can retain or suppress different portions of the spectrum.

Figure 2: Retained fraction under different renewal filters versus ridge for two eigenmodes, showing that only the exponential law matches ridge for all $r$0.

Figure 2 highlights the mismatch between renewal-induced shrinkage and ridge: only Poisson resetting matches the ridge filter for every eigendirection.

Fluctuation Costs and Full Risk Decomposition

While Poisson resetting replicates the ridge mean, the stochastic resetting process yields nonzero stationary covariance due to two distinct sources: (1) accumulation of gradient (or SGD) noise, modeled by an Ornstein-Uhlenbeck process, and (2) additional variance arising from the randomness of reset timing. The total parameter risk splits into: the risk of the mean estimator (ridge risk), an extra cost from accumulated SGD/stochastic noise, and a 'reset timing' variance term that is absent in deterministic ridge.

Figure 3: Decomposition of total risk into mean estimator, SGD variance, and timing variance terms as the reset-time law is varied; plots show when non-Poisson laws outperform Poisson.

Empirically and analytically, at matched filters, stochastic resetting cannot outperform deterministic ridge due to these extra variance terms. Any predictive improvement must arise from the alteration of the mean filter itself, i.e., the choice of spectral shrinkage rather than from stochastic implementation.

Renewal Law Design and Spectral Risk Tradeoffs

The paper generalizes the risk/covariance formulas to arbitrary renewal laws, explicitly characterizing biases, noise accumulation, and timing variance in the stationary state for any spectral filter induced by the renewal law. Deterministic periodic resetting suppresses weak modes more aggressively than Poisson, at the cost of increased bias in strong modes, and this mechanism can translate into either improved or worsened prediction risk, depending on the alignment of signal and noise in the spectrum.

Figure 4: Validation-tuned empirical performance of periodic and Erlang-family filters on spiked covariance models; left: filter curves versus spectrum; right: test MSE gains over ridge.

Figure 5: Block-covariance experiments showing filter-induced prediction gains from renewal-induced shrinkage compared to ridge in the presence of correlated nuisance structure.

Sharp differences from ridge are often beneficial when predictive signal is confined to a few leading directions and the bulk of the spectrum is noise-dominated; this occurs in both spiked and structured block-covariance settings.

Practical and Theoretical Implications

Key claims and findings:

• Poisson resetting is the unique renewal-based process whose stationary mean exactly equals scalar ridge shrinkage in every eigendirection.
• Resetting with other renewal laws yields a structured family of admissible spectral filters, all of which can be written as Laplace transforms over equilibrium age distributions.
• The added variance from stochastic resetting (SGD noise and timing) is strictly non-negative; hence, at fixed filter, ridge weakly dominates stochastic implementations in risk.
• Empirical experiments show that non-exponential filters (e.g., periodic/Erlang) can lead to nontrivial test set MSE gains (1–4% over ridge in certain high-dimensional, noise-dominated regimes), but are still outperformed by sharp cutoff (PCR/TSVD) filters.

Broader implications:

• Realizing non-ridge spectral shrinkage dynamically via renewal resetting suggests new algorithmic variants for iterative regularization, including deterministic partial resets or non-Poisson stochastic resets.
• Renewal theory constrains the class of admissible filters if one insists on a dynamical, physically interpretable implementation; only completely monotone filters corresponding to Laplace transforms of distributions can arise.
• Connection to Bayesian regularization: Poisson resetting identically matches the Bayesian posterior mean for an isotropic prior, but this is a dynamical, not a probabilistic, equivalence.

Future research opportunities include optimization of renewal laws for spectral risk minimization, extensions to state-dependent or anisotropic resetting, analysis under non-constant gradient noise (as in practical SGD), discrete-time variants, and systematic empirical study across more complex model classes.

Conclusion

This paper tightens the analytic correspondence between Poisson resetting of gradient flow and ridge regression, embedding the latter within a general renewal regularization framework. The spectral structure of renewal-induced shrinkage is explicit, with only exponential laws yielding true ridge regularization in all directions. While stochastic resetting always incurs additional variance relative to deterministic ridge at a matched filter, non-exponential resetting can nonetheless implement new, selective regularizers that improve test-set performance in high-dimensional regimes where noise accumulates in low-curvature eigenspaces. Physically interpretable, renewal-based regularization offers a principled path to new forms of iterative, spectral regularization, motivating both new theoretical exploration and algorithmic innovation.