Shrinkage Incidence Ratios

• Shrinkage incidence ratios are defined as the proportion of risk retained when using shrinkage estimators instead of conventional MLEs, highlighting risk reduction.
• They arise from incorporating regularization or Bayesian priors in models like Poisson, binomial, and multivariate normal, offering a clear metric for estimator performance.
• These ratios inform practical decisions in penalized regression and mixture models by quantifying risk attenuation and empirical improvements in mean-squared error.

A shrinkage incidence ratio quantifies the proportion of risk or uncertainty that remains after applying shrinkage estimators to incidence or rate parameters, relative to conventional maximum likelihood estimators (MLEs). Such ratios arise when regularization or Bayesian prior information is incorporated into parameter estimation for models of rates or counts (e.g., Poisson or binomial-modelled incidences, multivariate normal means), particularly in high-dimensional or small-sample settings. The shrinkage incidence ratio rigorously encapsulates the degree of risk attenuation—and thus “incidence of shrinkage”—obtained by moving from unregularized to shrinkage-based estimation of rate-statistics or regression parameters.

1. Risk Ratios for Shrinkage Estimators: Formal Definition

Let $R(\delta)$ denote the quadratic risk of an estimator $\delta$ for a parameter vector $\theta$, under a standard quadratic loss $E_\theta[\|\delta - \theta\|^2]$. For a shrinkage estimator (such as a Bayes, empirical Bayes, or penalized estimator), the shrinkage incidence ratio is defined as

$r(\delta) = \frac{R(\delta)}{R(X)}$

where $X$ is the MLE or conventional estimator. This ratio measures the proportion of the MLE’s expected loss that persists when adopting a shrinkage rule. Values strictly below 1 indicate dominance (risk reduction) of the shrinkage estimator over the MLE under quadratic loss. This ratio is central to the minimaxity analysis of shrinkage estimators in both classical and Bayesian contexts (Hamdaoui et al., 2020).

2. Shrinkage Incidence Ratios for Multivariate Normal Means

In the canonical multivariate normal case $X \sim N_p(\theta, \sigma^2 I_p)$, with prior $\theta \sim N_p(\nu, \tau^2 I_p)$, explicit shrinkage incidence ratios arise from the analysis of two classes of estimators:

• Modified Bayes Estimator (known $\tau^2$):

$$\delta_B^* = (1 - b_B(S^2))(X - \nu) + \nu,\qquad b_B(S^2) = \frac{S^2}{S^2 + n \tau^2}$$

• Empirical Modified Bayes Estimator (unknown $\tau^2$):

$$\delta_{EB}^* = (1 - b_{EB}(S^2, X))(X - \nu) + \nu, \qquad b_{EB}(S^2, X) = \frac{(p-2)}{n+2}\cdot \frac{S^2}{\|X - \nu\|^2}$$

where $S^2$ estimates $\sigma^2$ and $n$ is degrees of freedom. The exact risk ratios are:

• For the modified Bayes estimator,

$$r(\delta_B^*) = 1 + n(n+2)(1+\lambda) E_{u \sim \chi^2_n}\Bigl[\frac{1}{(u + n\lambda)^2}\Bigr] - 2n E_{u \sim \chi^2_n}\Bigl[\frac{1}{u + n\lambda}\Bigr]$$

where $\lambda = \tau^2 / \sigma^2$.

• For the empirical Bayes version,

$$r(\delta_{EB}^*) = 1 - \frac{p-2}{p} \cdot \frac{n}{n+2} \cdot \frac{1}{1+\lambda}$$

In both cases, for $p \geq 3$ and $n \geq 5$ the risk ratios are strictly less than 1, indicating dominance over the MLE and minimaxity (Hamdaoui et al., 2020).

3. Asymptotic Limits and Interpretation

As $p, n \to \infty$ (without coupling constraints),

$$\lim_{p,n \to \infty} r(\delta_B^*) = \lim_{p,n \to \infty} r(\delta_{EB}^*) = \frac{\lambda}{1+\lambda}$$

This constant, $\lambda/(1+\lambda) = \tau^2/(\sigma^2+\tau^2)$, represents the shrinkage-incidence ratio in the high-dimensional (or large-sample) Bayesian setting and quantifies the ultimate risk-reduction achievable through shrinkage. As $\tau^2 \ll \sigma^2$ (strong prior, concentrated beliefs), $r(\delta) \to 0$, indicating substantial risk reduction. As $\tau^2 \gg \sigma^2$ (diffuse, non-informative prior), $r(\delta) \to 1$, and the shrinkage offers little advantage over the MLE (Hamdaoui et al., 2020).

The shrinkage-incidence ratio thus exactly summarizes the “incidence of shrinkage” in terms of retained risk proportion and provides a dimension-independent theoretical target for regularization performance in large-scale estimation problems.

4. Shrinkage Incidence Ratios for Rates and Incidence Statistics

In Poisson or binomial models for incidence statistics—e.g., $Y_i \sim \mathrm{Poisson}(n_i \theta_i)$ or $Y_i \sim \mathrm{Binomial}(n_i, \theta_i)$—Bayesian and empirical Bayes estimators again give rise to shrinkage of rate or incidence-ratio estimates:

$$\hat{\theta}_i^\mathrm{shrink} = w_i \hat{\theta}_i^\mathrm{MLE} + (1-w_i) \bar{\theta}$$

where $w_i = n_i/(n_i+\alpha+\beta)$ (for the binomial–beta case) and $\bar{\theta}$ is the pooled average (prior mean). Here, for small $n_i$ (low exposure), $w_i \ll 1$, inducing substantial shrinkage toward the global mean. The aggregate global mean-squared error (risk) of shrinkage estimates is empirically and theoretically bounded strictly below that of the MLE:

• MLE risk: $\simeq 0.00066$
• Shrinkage risk: $\simeq 0.00054$
• James–Stein risk (for comparison): $\simeq 0.00059$

Hence, under squared-error loss, shrinkage estimators “dominate” the MLE for aggregate risk (Holsbø et al., 2018).

5. Shrinkage Incidence Ratios in Penalized Regression and Mixture Models

Shrinkage incidence ratios also manifest in penalized regression settings, especially with collinear predictors or complex mixture structures:

• Ridge and Liu-type shrinkage in finite mixtures of Poisson regressions with “experts” significantly decrease mean-squared errors of incidence-rate ratio (IRR) estimates.
• In settings with severe collinearity (design correlations $0.85$–$0.95$), risk reductions are as follows:

Estimator	$\sqrt{\mathrm{MSE}}$ (components)
MLE	$\approx 2.31$
Ridge	$\approx 0.70$
Liu-type	$\approx 0.16$

Similarly, in real-data applications (e.g., UCI Cleveland-clinic heart-disease data), Liu-type shrinkage leads to median $\sqrt{\mathrm{MSE}}$ reductions from MLE ($0.22$) to ridge ($0.15$) and further to Liu-type ($0.12$) (Ghanem et al., 2023).

Shrinkage IRRs, computed as $\exp(\hat{\beta}_{k,\mathrm{shrink}})$, reflect this enhanced stability and risk attenuation, though at the cost of small bias. These phenomena exemplify shrinkage-incidence ratios beyond explicit closed-form expressions.

6. Practical Considerations and Empirical Bayes Implementation

Estimation and application of shrinkage incidence ratios in practice involve:

• Explicit estimation of prior hyperparameters (via empirical Bayes moments, e.g., for binomial–beta: solving $\alpha/(\alpha+\beta) = \bar\theta$, $$\alpha\beta/\{(\alpha+\beta)^2(\alpha+\beta+1)\} = S^2$$ using the pooled MLEs).
• Use of posterior means as shrinkage estimators for group-level or site-level rates.
• Risk assessments via simulation or analytic bounds, routinely reporting the shrinkage-incidence ratio to convey expected efficiency gains.

For regression problems, ridge-tuning parameters are heuristically set as $$\lambda_j \approx p/\|\hat{\beta}_{j,\mathrm{ML}}\|^2$$; Liu-type parameters add an extra bias-correction $d_j$, minimizing approximate mean-squared error (Ghanem et al., 2023). Reporting of IRRs, intervals, and risk comparisons in terms of shrinkage-incidence ratios conveys the reproducibility and reliability benefits over non-shrunk estimators.

7. Interpretation and Limitations

The shrinkage incidence ratio is an interpretable, dimensionless summary of the risk-reduction achieved by shrinkage, finding broad applicability in high-dimensional estimation, hierarchical Bayes modelling, and regularized regression. Its limiting value, notably $\lambda/(1+\lambda)$ in Gaussian Bayesian contexts, encapsulates the interplay between signal ($\tau^2$) and noise ($\sigma^2$) variances and determines the ultimate benefit achievable by shrinkage irrespective of ambient dimension. While explicit closed-form ratios are mainly available for Gaussian and exponential family models under quadratic loss, the general principle extends to a wide spectrum of estimation and statistical learning settings where overfitting and parameter variance are controlled by regularization (Hamdaoui et al., 2020, Holsbø et al., 2018, Ghanem et al., 2023).