Smooth Shrinkage Methods

• Smooth shrinkage is a family of regularization strategies that continuously attenuate coefficients or singular values, avoiding abrupt thresholding.
• It spans applications including wavelet denoising, matrix correction, Bayesian posterior shrinkage, and neural-network initialization with structured adaptation.
• Key insights include improved risk properties, enhanced scale adaptivity, and well-behaved transitional effects that overcome limitations of traditional hard thresholding.

Searching arXiv for papers on “smooth shrinkage” and closely related usages to ground the article in current literature. Smooth shrinkage denotes a family of estimation and regularization ideas in which coefficients, singular values, functions, or parameter vectors are attenuated continuously or structurally rather than handled only by abrupt keep-or-kill rules. In the literature, however, the term is not uniform. In wavelet denoising it often refers to continuous thresholding laws or smooth attenuation factors; in matrix denoising it can mean a smooth-above-threshold correction of empirical singular values; in Bayesian work it can describe continuous posterior shrinkage that asymptotically mimics thresholding; in functional regression it can mean shrinkage of a smooth effect toward a predefined low-dimensional subspace; and in some neural-network work the word “shrinkage” is used in a nonstandard way for SVD-based normalization at initialization (Alt et al., 2019, Gavish et al., 2014, Song, 2020, Wiemann et al., 2021, Cheng et al., 12 Apr 2025).

1. Terminological range and recurrent structural properties

Across the cited literature, “smooth shrinkage” does not identify a single operator class. It instead names several related design principles: continuity across a threshold region, attenuation with weak tail bias, structural pooling toward a target subspace, or smooth dependence of an estimator on a shrinkage path or on posterior scale parameters. A further complication is that some papers use “shrinker” or “shrinkage” in adjacent but semantically distinct ways, especially in geometric analysis and neural-network initialization (Vimalajeewa et al., 17 Jun 2026, Kulkarni et al., 16 Jan 2026, Mramor, 2021, Cheng et al., 12 Apr 2025).

Setting	Object being shrunk	Meaning of “smooth”
Wavelet denoising	Coefficients or coupled subband vectors	Continuous attenuation, no jump or kink, or a smooth transition band
Matrix denoising	Empirical singular values	Smooth-above-threshold nonlinear correction
Bayesian or structured estimation	Coordinates, group means, functional deviations	Continuous posterior shrinkage or partial pooling
Nonstandard usage	Layer transformations or geometric hypersurfaces	SVD-based normalization, or self-similar shrinking under flow

Several properties recur, but not universally. In the SCOPE framework, the shrinkage rule

$$\delta(x;\lambda,k)=x\,|F^*(\lambda x)|^k,\qquad F^*(x)=2F(x)-1,$$

is proved odd, sign-preserving, contractive, monotone increasing, continuous, and asymptotically unbiased in the relative sense $\delta(x;\lambda,k)/x\to 1$ as $|x|\to\infty$ (Vimalajeewa et al., 17 Jun 2026). By contrast, Smooth SCAD keeps exact thresholding below $\lambda$ and identity above $a\lambda$, while smoothing only the transition region by a raised cosine (Kulkarni et al., 16 Jan 2026). In matrix denoising, the Frobenius-optimal singular-value rule is zero below the bulk edge and smooth for $y>1+\sqrt{\beta}$, so “smoothness” attaches only to the retained regime, not to the entire map (Gavish et al., 2014). The literature therefore distinguishes continuity, differentiability, exact sparsity, asymptotic unbiasedness, and low-bias transition behavior rather than collapsing them into a single definition.

2. Wavelet and coefficient-domain formulations

Wavelet denoising is the most direct setting in which smooth shrinkage is used as a technical term. A prominent example is the adaptive wavelet rule derived from the Forward-and-Backward diffusivity,

$$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$

embedded in the coupled Haar shrinkage of Mrázek and Weickert,

$$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$

Here smoothness means a globally smooth response with no abrupt threshold boundary, together with rotationally coupled treatment of the directional channels. The final generic parameterization

$$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$

makes the rule adaptive to both scale and noise level. The paper reports that, at $\sigma=20$, hard shrinkage with coupled non-decimating Haar gives $\delta(x;\lambda,k)/x\to 1$0 dB, the smooth non-amplifying variant with $\delta(x;\lambda,k)/x\to 1$1 gives $\delta(x;\lambda,k)/x\to 1$2 dB, allowing amplification raises this to $\delta(x;\lambda,k)/x\to 1$3 dB, and making the rule scale-adaptive gives $\delta(x;\lambda,k)/x\to 1$4 dB; the generic two-parameter model is only about $\delta(x;\lambda,k)/x\to 1$5 dB worse than a fully trained scale/noise-specific model, while reducing the parameter count from $\delta(x;\lambda,k)/x\to 1$6 to $\delta(x;\lambda,k)/x\to 1$7 (Alt et al., 2019). In this formulation, smoothness alone is not the dominant source of the gain; amplification and especially scale adaptivity are.

A more abstract coefficientwise construction is SCOPE shrinkage,

$\delta(x;\lambda,k)/x\to 1$8

where $\delta(x;\lambda,k)/x\to 1$9 is a continuous symmetric unimodal distribution function and $|x|\to\infty$0. The centered CDF generates an odd attenuation profile with strong suppression near zero and asymptotically weak tail bias. The logistic prototype,

$|x|\to\infty$1

is globally smooth; the uniform prototype is continuous and monotone but only piecewise smooth; the Cauchy prototype yields more persistent tail shrinkage. The framework also admits an even penalty representation and, for suitable subclasses, an exact MAP interpretation under proper symmetric unimodal priors (Vimalajeewa et al., 17 Jun 2026).

Smooth SCAD regularizes a different classical object. It retains the SCAD architecture of exact thresholding for $|x|\to\infty$2 and identity for $|x|\to\infty$3, but replaces the piecewise linear transition by

$|x|\to\infty$4

The resulting rule is continuous on $|x|\to\infty$5, continuously differentiable for $|x|\to\infty$6, and falls within the continuous-thresholding class for which Stein’s unbiased risk estimate is valid. On the Donoho–Johnstone signals at $|x|\to\infty$7 and $|x|\to\infty$8, the reported AMSEs show smooth SCAD consistently below classical SCAD, hard, and soft thresholding; for example, on Blocks the values are $|x|\to\infty$9 for hard, $\lambda$0 for soft, $\lambda$1 for SCAD, and $\lambda$2 for smooth SCAD (Kulkarni et al., 16 Jan 2026).

MLShrink is best understood as a contrast case. It uses two thresholds,

$\lambda$3

sets coefficients below $\lambda$4 to zero, keeps those above $\lambda$5, and classifies the undecided band using local wavelet-domain features. The paper states explicitly that MLShrink is not a new continuous shrinkage family but a support-selection rule. Its excess risk relative to an oracle two-threshold keep/discard rule is governed by classification errors on the undecided band (Vimalajeewa et al., 17 Jun 2026). This clarifies a recurring fault line in the literature: some methods smooth the attenuation function itself, while others replace the transition region by learned decisions.

3. Spectral and higher-order shrinkage

In low-rank matrix denoising, smooth shrinkage appears as nonlinear correction of empirical singular values rather than coefficient thresholding. If

$\lambda$6

the estimator

$\lambda$7

shrinks only singular values. For Frobenius loss in the spiked model, the asymptotically optimal shrinker is

$\lambda$8

and in the square case $\lambda$9,

$a\lambda$0

The paper emphasizes that this is smooth for $a\lambda$1, zero below threshold, and asymptotically equal to the identity for large $a\lambda$2. In the square model $a\lambda$3, the corresponding asymptotic MSE guarantee is $a\lambda$4, compared with $a\lambda$5 for optimally tuned hard thresholding and $a\lambda$6 for optimally tuned soft thresholding (Gavish et al., 2014).

A different spectral use of shrinkage appears in nonconvex Hessian regularization. The local $a\lambda$7 Hessian at pixel $a\lambda$8 has singular values $a\lambda$9 and $y>1+\sqrt{\beta}$0, and the scalar $y>1+\sqrt{\beta}$1-shrinkage operator is

$y>1+\sqrt{\beta}$2

Applied spectrally, this yields a nonconvex second-order regularizer that is continuous and piecewise smooth, but not globally differentiable because of the threshold kink. The induced penalty $y>1+\sqrt{\beta}$3 is defined implicitly through this proximal map, is even and continuous, differentiable on $y>1+\sqrt{\beta}$4, generally nondifferentiable at $y>1+\sqrt{\beta}$5, coercive for $y>1+\sqrt{\beta}$6, and strictly concave on the positive half-line. The paper proves restricted proximal regularity of the local spectral penalty and convergence of its ADMM algorithm to stationary points. In undersampled MRI with $y>1+\sqrt{\beta}$7, the reported SSIM on image 2, mask 2 is $y>1+\sqrt{\beta}$8 for the proposed method versus $y>1+\sqrt{\beta}$9 for TV-1, $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$0 for TV-2, and $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$1 for HS-1 (Ghulyani et al., 2023).

These two examples show that spectral smooth shrinkage is not limited to scalar threshold curves. It may refer either to smooth nonlinear singular-value correction in random matrix models or to continuous-but-nonsmooth proximal shrinkage applied to local Hessian spectra.

4. Bayesian, empirical-Bayes, and structured partial pooling

Bayesian work often uses “smooth shrinkage” to mean continuous posterior contraction without discrete model selection. In sparse normal means,

$$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$2

the paper on sharp minimaxity studies one-group priors with polynomial tails of order $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$3. The posterior mean remains continuous in $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$4, but for $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$5 sufficiently close to $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$6 and suitably calibrated global scale $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$7, the effective shrinkage behaves asymptotically like hard thresholding at approximately $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$8. The main theoretical statement is that the posterior can attain sharp minimax contraction, with asymptotic constant arbitrarily close to $$g(s^2)= 2\exp\!\left(-\frac{s^2}{\lambda_1^2}\right)-\exp\!\left(-\frac{s^2}{\lambda_2^2}\right),\qquad \lambda_2\ge \lambda_1,$$9, when $$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$0 and $$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$1 is chosen appropriately; a transformed Beta prior on $$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$2 yields adaptation to unknown sparsity (Song, 2020). This is continuous shrinkage in form, threshold-like in asymptotic effect.

The SMASH framework places the same idea in multiscale denoising. After a wavelet or Poisson multiscale transform, the coefficients satisfy a heteroskedastic normal-means approximation

$$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$3

and the empirical-Bayes prior is a normal scale mixture

$$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$4

The posterior mean

$$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$5

produces continuous coefficientwise shrinkage that adapts across scales and to unequal standard errors. In the original signal domain, this is smoothing; in the transform domain, it is adaptive shrinkage toward zero without a fixed threshold rule (Xing et al., 2016).

Structured partial pooling gives a further generalization. Pairwise cross-smoothing for categorical regressors solves a penalized least-squares problem with pairwise targets and yields

$$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$6

This is smooth shrinkage in the sense of continuous borrowing across groups rather than thresholding or exact fusion. Under the paper’s asymptotic risk criterion, the feasible PCS estimator uniformly dominates OLS when the number of groups $$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$7 (Heiler et al., 2019). In treatment choice, a related idea shrinks subgroup CATE estimates toward the average score,

$$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$8

with the shrinkage factor selected by minimizing an upper bound on maximum regret (Ishihara et al., 2022).

For panel-data fixed effects and multi-source transfer, smoothness enters through covariance-aware shrinkage paths. In the panel model,

$$S\!\left(\begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}\right) = \bigl(1-g(s^2)\bigr) \begin{pmatrix}w_x\ w_y\ w_{xy}\end{pmatrix}, \qquad s^2=w_x^2+w_y^2+2w_{xy}^2.$$9

so off-diagonal structure in $$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$0 induces timewise borrowing across a unit’s trajectory (Kwon, 2023). In multi-source transfer, the covariance-aware path

$$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$1

varies continuously in the shrinkage parameter and admits an explicit risk-improving interval for the reparameterized size $$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$2; the paper then extends this construction to smooth $$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$3-estimation via local quadratic expansion (Jing et al., 29 Jun 2026). In both cases, smooth shrinkage means continuous movement along a structured shrinkage path rather than a threshold map.

5. Functional, neural, and geometric reinterpretations

Functional regression gives a particularly literal form of smooth shrinkage. A spline-based effect

$$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$4

is decomposed relative to a predefined subspace $$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$5 with projection $$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$6, and the prior precision is

$$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$7

The first term shrinks the deviation from the chosen parametric subspace, while the second is a P-spline-type smoothness penalty. The construction uses one scale parameter per spline, not one per coefficient, and aims to shrink the function toward constants, linears, quadratics, or trigonometric subspaces while preventing highly oscillating overfit (Wiemann et al., 2021). Here “smooth shrinkage” is exact: the target is a smooth functional effect, the deviation is shrunk, and roughness is penalized simultaneously.

A markedly different usage appears in neural-network initialization. SINL defines layer maps by $$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$8, estimates inter-layer bridges

$$\lambda_1(\ell,\sigma)=\frac{5.4\,\sigma}{\ell^2},\qquad \lambda_2(\ell,\sigma)=\frac{8.9\,\sigma}{\ell^2}$$9

computes

$\sigma=20$0

and updates boundary weights by orthogonal factors. For odd depth it replaces the median map by

$\sigma=20$1

discarding singular values. The paper states that this “shrinkage initialization” supports “smooth learning,” but also makes clear that the method does not use soft-thresholding, proximal shrinkage, or an explicit shrinkage penalty. “Shrinkage” here refers instead to SVD-based orthogonal rotation and normalization of inter-layer transformations, and “smooth learning” is only informally defined through stable activations, stable objective descent, and visually similar weight transformations across epochs (Cheng et al., 12 Apr 2025).

A still more distant semantic branch is geometric analysis. In mean curvature flow, a self-shrinker satisfies

$\sigma=20$2

or equivalently $\sigma=20$3 up to sign convention. Papers on smooth asymptotically conical self-shrinkers in $\sigma=20$4 and on generating shrinkers by mean curvature flow use “shrinker” to denote self-similarly shrinking hypersurfaces, not shrinkage estimators (Mramor, 2021, Hoffman et al., 27 Feb 2025). This is a homonym rather than a contribution to statistical shrinkage.

6. Conceptual fault lines and open directions

Several fault lines recur across the literature. First, smoothness does not always mean differentiability. SCOPE includes fully smooth and piecewise smooth subclasses; Smooth SCAD is continuous and continuously differentiable away from $\sigma=20$5; Hessian $\sigma=20$6-shrinkage is continuous but nonsmooth at the threshold; MLShrink is not a continuous shrinkage family at all (Vimalajeewa et al., 17 Jun 2026, Kulkarni et al., 16 Jan 2026, Ghulyani et al., 2023, Vimalajeewa et al., 17 Jun 2026). Second, smoothness does not by itself explain empirical improvement. In the FAB-based wavelet rule, the ablation study indicates that scale adaptivity and coefficient amplification contribute more than smoothness alone (Alt et al., 2019). Third, shrinkage need not mean thresholding. It may mean continuous posterior contraction, covariance-aware pooling, deviation shrinkage toward a function class, or SVD-based normalization of network maps (Song, 2020, Kwon, 2023, Wiemann et al., 2021, Cheng et al., 12 Apr 2025).

Common misconceptions follow directly from these fault lines. One is that smooth shrinkage is always a proximal map of a convex penalty; several of the cited constructions are nonconvex, only implicitly penalized, or have no proximal interpretation at all (Ghulyani et al., 2023, Alt et al., 2019). Another is that smooth shrinkage necessarily avoids exact sparsity; Smooth SCAD preserves exact zeros below threshold, and some Bayesian constructions remain continuous while asymptotically mimicking hard thresholding (Kulkarni et al., 16 Jan 2026, Song, 2020). A third is that the word “shrinkage” always carries its standard sparse-estimation meaning; SINL is an explicit counterexample (Cheng et al., 12 Apr 2025).

The open directions named in these papers are domain-specific. SCOPE points to fuller data-driven calibration, especially level-dependent and adaptive tuning, and to redundant-transform, multivariate, inverse-problem, and high-dimensional generalizations (Vimalajeewa et al., 17 Jun 2026). The functional-subspace prior suggests extension beyond Gaussian regression to exponential-family or distributional settings (Wiemann et al., 2021). SINL leaves its claimed universality largely untested beyond three-layer fully connected networks (Cheng et al., 12 Apr 2025). The geometric literature treats higher-dimensional extensions of the medium-entropy self-shrinker classification as open and harder (Mramor, 2021). Taken together, these directions indicate that “smooth shrinkage” is not a settled method class but a broad and still expanding collection of regularization strategies centered on replacing crude selection by more structured attenuation.