Confidence Interval Clipping via SCAD

• The confidence interval-based clipping method uses the SCAD estimator to create intervals that balance variable selection and estimation in sparse linear models.
• It employs a thresholding mechanism that clips small signals while preserving the near-unbiased estimation of large coefficients in an orthonormal design.
• The approach achieves oracle properties asymptotically and ensures conservative coverage, though reducing the expected interval length uniformly is inherently challenging.

A confidence interval-based clipping method refers to constructing confidence intervals (CIs) for regression coefficients where both the center and radius of the interval depend on a clipped or thresholded estimator, specifically the smoothly clipped absolute deviation (SCAD) estimator, rather than the classical least squares estimator. This approach is motivated by variable selection scenarios in sparse linear models, balancing the goals of selection, estimation, and valid post-selection inference. The method is particularly relevant for regression models with orthonormal design matrices and aims to extend the oracle and shrinkage properties of SCAD to associated inferential intervals (Farchione et al., 2012).

1. SCAD Penalty and Its Derivatives

The SCAD penalty $p_\lambda(\theta)$, proposed by Fan and Li (2001), is a nonconcave penalty used in penalized regression to encourage sparsity while alleviating bias for large coefficients. It is parameterized by %%%%1%%%% (tuning parameter) and %%%%2%%%% (typically $a=3.7$):

$$p_\lambda(\theta) = \begin{cases} \lambda |\theta|, & |\theta| \leq \lambda, \ -\frac{\theta^2 - 2 a \lambda |\theta| + \lambda^2}{2(a-1)}, & \lambda < |\theta| \leq a \lambda, \ \frac{(a+1)\lambda^2}{2}, & |\theta| > a \lambda. \end{cases}$$

Its derivative $p'_\lambda(\theta)$ is

$$p'_\lambda(\theta) = \begin{cases} \lambda\,\mathrm{sign}(\theta), & |\theta| \leq \lambda, \ \frac{a\lambda - |\theta|}{a-1}\,\mathrm{sign}(\theta), & \lambda < |\theta| \leq a\lambda, \ 0, & |\theta| > a\lambda. \end{cases}$$

Alternatively, for $t \geq 0$,

$$p'_\lambda(t) = \lambda I(t \leq \lambda) + \frac{(a\lambda - t)_+}{a-1} I(t > \lambda),$$

with $$p_\lambda(\theta) = \int_0^{|\theta|} p'_\lambda(t)\,dt$$.

SCAD is designed to perform thresholding (clipping small signals to zero) for variable selection, with less shrinkage for large signals, thereby maintaining model selection consistency and the oracle property (Farchione et al., 2012).

2. Construction of the SCAD Estimator in Orthonormal Designs

Under the standard Gaussian linear regression model $Y = X\beta + \varepsilon$, $\varepsilon \sim N(0, \sigma^2 I_n)$, with orthonormal design $X^T X = I_p$, consider estimating a specific component $\beta_i$. The least-squares estimator is $\hat\beta_i$, and $\hat\Sigma$ is the unbiased estimator for $\sigma$.

The SCAD estimator $\tilde\beta_i$ is the minimizer of $\tfrac12(\hat\beta_i - b)^2 + p_\lambda(|b|)$, where $\lambda = \hat\Sigma \eta$, $a = 3.7$. The explicit solution is

$$\tilde\beta_i = \begin{cases} \operatorname{sign}(\hat\beta_i) (|\hat\beta_i|-\lambda)_+, & |\hat\beta_i| \leq 2\lambda, \ \frac{(a-1)\hat\beta_i - \operatorname{sign}(\hat\beta_i) a\lambda}{a-2}, & 2\lambda < |\hat\beta_i| \leq a\lambda, \ \hat\beta_i, & |\hat\beta_i| > a\lambda. \end{cases}$$

This estimator combines hard thresholding for small signals and near-unbiased estimation for large coefficients. For $|\hat\beta_i| > a\lambda$, it coincides with the least-squares estimator (Farchione et al., 2012).

3. Confidence Interval Construction and Clipping Mechanism

The classical $(1-\alpha)$ confidence interval for $\beta_i$ is $$[\hat\beta_i - t(m)\hat\Sigma,\, \hat\beta_i + t(m)\hat\Sigma]$$, where $t(m)$ is the $1-\alpha/2$ quantile of the $t_m$ distribution, with $m = n-p$.

The confidence interval centred on $\tilde\beta_i$ adopts the form:

$$J(s) = [\tilde\beta_i - \hat\Sigma s(|\hat\beta_i|/\hat\Sigma),\ \tilde\beta_i + \hat\Sigma s(|\hat\beta_i|/\hat\Sigma)],$$

where $s: (0, \infty) \to (0, \infty)$ is continuous with $s(x) = t(m)$ for $x \geq k$, $k = a\eta$.

Thus, the lower and upper endpoints are

$$L(\hat\beta_i, \hat\Sigma) = \tilde\beta_i - \hat\Sigma s(|\hat\beta_i|/\hat\Sigma), \quad U(\hat\beta_i, \hat\Sigma) = \tilde\beta_i + \hat\Sigma s(|\hat\beta_i|/\hat\Sigma).$$

For $|\hat\beta_i| > k\hat\Sigma$, the interval $J(s)$ reduces to the classical interval, because both the center and width revert to the least-squares solution, enforcing "clipping" at large signal-to-noise ratios (Farchione et al., 2012).

4. Finite-Sample and Asymptotic Properties

4.1 Coverage Probability

Let $\theta = \beta_i/\sigma$ and $W = \hat\Sigma/\sigma$. Define the function $h(x)$:

$$h(x) = \begin{cases} \mathrm{sign}(x) (|x|-\eta)_+, & |x| \leq 2\eta, \ \frac{(a-1)x - \mathrm{sign}(x) a\eta}{a-2}, & 2\eta < |x| \leq a\eta, \ x, & |x| > a\eta. \end{cases}$$

The coverage probability of $J(s)$ is

$$\mathrm{CP}(\theta) = \int_{-k}^{k} \int_{0}^{\infty} \mathbf{1}\left(h(x)-s(|x|) \leq \theta/w \leq h(x)+s(|x|)\right) \phi(w x - \theta) w f_W(w) dw dx + 1-\alpha - \int_0^\infty b(w;m,k,\theta) f_W(w) dw,$$

where $f_W$ is the density of $W$, and $$b(w;m,k,\theta) = \Phi(\min(t(m)w,kw-\theta)) - \Phi(\max(-t(m)w,-kw-\theta))$$. Key coverage properties are:

• $\mathrm{CP}(\theta)$ is an even function of $\theta$.
• For any $s$ with $s(x) = t(m)$ for $x \geq k$, $\mathrm{CP}(\theta) \to 1 - \alpha$ as $|\theta| \to \infty$.
• Numerically, $\mathrm{CP}(\theta)$ strictly exceeds $1-\alpha$ for small $|\theta|$, so $s(\cdot)$ cannot be reduced in that region without violating nominal coverage (Farchione et al., 2012).

4.2 Oracle Properties and Asymptotics

In the asymptotic regime where $\lambda_n \to 0$ and $\sqrt{n}\lambda_n \to \infty$, the SCAD estimator achieves the oracle property: it sets truly zero coefficients to exactly zero with probability tending to one, and for nonzero coefficients, $$\sqrt{n}(\tilde\beta_j - \beta_j) \to N(0, \sigma^2)$$. Since $J(s)$ converges to the standard $t$-interval when $|\hat\beta_i|/\hat\Sigma > k$, its coverage and central tendency inherit these oracle characteristics for large $|\beta_i|$.

4.3 Interval Length under Sparsity

The scaled expected length is

$$e(\theta; s) = \frac{E_\theta[\text{length of }J(s)]}{E[\text{length of }I]} = 1 + \frac{1}{t(m) E(W)} \int_{-k}^k (s(|x|) - t(m)) \int_0^\infty \phi(wx - \theta) w^2 f_W(w) dw dx.$$

Desirable properties include $e(\theta; s) \to 1$ as $|\theta| \to \infty$. However, minimizing $e(0; s) < 1$ while maintaining $\mathrm{CP}(\theta) \geq 1-\alpha$ across all $\theta$ has been shown to be unattainable without incurring unacceptably large lengths elsewhere, as established by Farchione and Kabaila (2008) and general admissibility arguments (Kabaila 2011) (Farchione et al., 2012).

5. Numerical Illustration and Optimization

Empirical evaluations consider $\alpha=0.05$ with $m = 200$ (large) or $m = 3$ (small degrees of freedom), and $\eta = 0.5, 1, 2$. The function $s(\cdot)$ is represented by a natural cubic spline on $[0, k]$ with $q = 4, 5, 6$ knots, enforcing $s(k) = t(m)$. Optimization is performed:

• Minimize $e(0; s)$,
• Subject to $s(x) > 0$ for $x \in [0, k]$, and $\mathrm{CP}(\theta) \geq 1 - \alpha$ for all $\theta \geq 0$.

Key findings:

• $e(0; s^*) > 1$ for every $(m, \eta, q)$; the interval cannot be shorter than the standard interval when $\beta_i = 0$.
• $\sup_\theta e(\theta; s^*)$ is also $>1$, often substantially so when $\eta$ or the variance of $1/W$ is large.
• As $|\theta|$ increases, $e(\theta; s^*)$ converges to $1$ from above, and $\mathrm{CP}(\theta)$ exceeds $0.95$ for small $|\theta|$, then returns to $0.95$ as $|\theta| \to \infty$.

These results establish a strict barrier: no $s(\cdot)$ achieves both uniform coverage and materially reduced expected length at $\theta=0$ (Farchione et al., 2012).

6. Comparisons and Admissibility Results

Classical intervals, by contrast, do not adapt to sparsity structure but avoid the inflammation of interval length observed in $J(s)$ when enforcing minimal coverage. Admissibility results (Kabaila 2011) establish that intervals uniformly shorter than the usual confidence interval while preserving nominal coverage are infeasible, highlighting the trade-off intrinsic to CI construction in sparse regression (Farchione et al., 2012).

7. References and Historical Context

• Fan, J. and Li, R. (2001), "Variable selection via nonconcave penalized likelihood and its oracle properties" (JASA 96, 1348–1360)
• Farchione, D. and Kabaila, P. (2008), "Confidence intervals for the normal mean utilizing prior information" (Stat. Prob. Letters 78, 1094–1100)
• Kabaila, P. (2011), "Admissibility of the usual confidence interval for the normal mean" (Stat. Prob. Letters 81, 352–359)
• Confidence interval properties and their admissibility in the context of shrinkage, thresholding, and model selection are fundamentally shaped by the impossibility results for simultaneously achieving shorter expected length and nominal coverage (Farchione et al., 2012).

Key properties and limitations of the confidence interval-based clipping method in the SCAD framework are thus determined by fundamental statistical trade-offs. This framework provides essential insight for the post-selection inference literature, clarifying the inextricable link between shrinkage, coverage, and conservatism in high-dimensional regression.