Half-Thresholding Rule in Sparse Recovery

• Half-thresholding is a nonlinear, closed-form operator that balances bias and stability, serving as a key tool in ℓ1/2 regularization for sparse recovery.
• It underpins iterative schemes like adaptively iterative thresholding and proximal gradient methods to efficiently detect support and converge in underdetermined systems.
• Compared to hard and soft thresholding, it offers an intermediate trade-off by aggressively promoting sparsity while maintaining robustness in high-dimensional applications.

The half-thresholding rule is a nonlinear scalar mapping and associated iterative thresholding scheme, fundamental to sparse regularization via nonconvex $\ell_{1/2}$-type penalties. It admits a closed-form, non-monotone thresholding operator, and is central to both adaptively iterative thresholding in underdetermined systems and fixed-parameter $\ell_{1/2}$-regularized optimization. The half-thresholding operator is distinguished by its discontinuity at the threshold and its strong sparsity-promoting properties, offering a balance between the bias of soft-thresholding and the instability of hard-thresholding. Rigorous convergence guarantees, complexity estimates, and comparative analyses support its advantages in high-dimensional sparse recovery applications (Zeng et al., 2013, Zeng et al., 2013, Zhang et al., 2014).

1. The Half-Thresholding Operator: Definition and Construction

The half-thresholding operator, denoted here as $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$, is defined for a scalar $u$ and threshold $\tau > 0$ by

$$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$

as developed for adaptively iterative thresholding (AIT) (Zeng et al., 2013) and equivalently for proximal solutions to an $\ell_{1/2}$-regularized quadratic subproblem (Zeng et al., 2013, Zhang et al., 2014). Applied to vectors $v\in\mathbb{R}^N$, the operator acts componentwise.

Key properties:

• $h_{\tau,1/2}$ is odd, strictly nondecreasing on $[0, \infty)$, and admits explicit lower and upper bounds: $\ell_{1/2}$0 for all $\ell_{1/2}$1.
• The map is discontinuous at the threshold $\ell_{1/2}$2, where the operator jumps from $\ell_{1/2}$3 to a strictly positive value, similar to hard-thresholding but unlike the continuous soft-thresholding transition (Zhang et al., 2014).
• The closed-form arises from solving a depressed cubic in the subproblem $\ell_{1/2}$4 using Cardano’s formula.

Alternative parameterization appears in the fixed-step iterative thresholding setting for $\ell_{1/2}$5 regularization, where $\ell_{1/2}$6, with $\ell_{1/2}$7 the regularization parameter and $\ell_{1/2}$8 the step size (Zeng et al., 2013).

2. Application in Iterative Schemes: AIT and Proximal Algorithms

The half-thresholding rule underpins two principal classes of algorithms:

(A) Adaptively Iterative Thresholding (AIT): For an underdetermined linear system $\ell_{1/2}$9, where $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$0, the AIT with half-thresholding seeks a $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$1-sparse solution by:

1. Initializing $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$2.
2. Iteratively: (a) Compute $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$3 (Landweber step). (b) Set $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$4 to the $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$5-st largest entry in $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$6. (c) Update $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$7. (d) Stop when a chosen criterion is met (Zeng et al., 2013).

(B) Proximal Gradient Method for $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$8 Regularization: For

$h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$9

the iterative scheme applies: $u$0 with $u$1 the closed-form half-thresholding operator, threshold $u$2, and $u$3 (Zeng et al., 2013, Zhang et al., 2014).

Both methods achieve per-iteration complexity $u$4, with scalar thresholding dominating the update after the two matrix-vector multiplications (Zeng et al., 2013, Zeng et al., 2013).

Algorithmic Setting	Threshold Update	Support Size Control	Reference
AIT (sparse recovery)	Adaptive by sparsity $u$5	Exactly $u$6	(Zeng et al., 2013)
$u$7 Proximal	Fixed or cross-validated	Data-driven	(Zeng et al., 2013)

3. Theoretical Guarantees and Convergence Analysis

Comprehensive convergence results are established under explicit measurement matrix coherence or restricted isometry assumptions:

• AIT with half-thresholding:

The algorithm recovers the true support of $u$8 in finitely many steps provided the coherence $u$9 obeys $\tau > 0$0, with $\tau > 0$1 the true sparsity (Zeng et al., 2013). Support identification is guaranteed in at most $\tau > 0$2 steps (constant dependent on $\tau > 0$3, and dynamic range). Once support is identified, the iterates $\tau > 0$4 converge exponentially fast to $\tau > 0$5:

$\tau > 0$6

where $\tau > 0$7, $\tau > 0$8 (Zeng et al., 2013).

• Iterative half-thresholding for $\tau > 0$9 regularization:

Under $$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$0, the sequence $$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$1 converges to a stationary point $$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$2. Local minimality is ensured for sufficiently small $$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$3 or well-conditioned $$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$4, and eventual linear convergence rate ($$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$5, $$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$6, for large $$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$7) is achieved (Zeng et al., 2013).

• Continuity at the threshold: Half-thresholding is discontinuous at the threshold, inducing more aggressive sparsity than the continuous soft-thresholding map of the $$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$8 case (Zhang et al., 2014).

4. Comparison with Hard and Soft Thresholding Schemes

Half-thresholding occupies an intermediate position between hard ($$h_{\tau,1/2}(u) = \begin{cases} \displaystyle \frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \ 0 & |u| \le \tau \end{cases}$$9) and soft ($\ell_{1/2}$0) thresholding:

• Coherence constraints:
  • Hard: $\ell_{1/2}$1
  • Half: $\ell_{1/2}$2
  • Soft: $\ell_{1/2}$3
  • The half-thresholding rule requires a slightly more restrictive coherence bound than hard-thresholding, but is less restrictive than soft (Zeng et al., 2013).
• Practical implications:
  • Hard-thresholding is unbiased but becomes unstable near the coherence limit.
  • Soft-thresholding introduces bias for large coefficients but is robust.
  • Half-thresholding achieves a tradeoff, with reduced bias compared to soft and enhanced stability compared to hard.
• Empirical iteration counts (example: $\ell_{1/2}$4, $\ell_{1/2}$5, dynamic range 10):
  • Hard: $\ell_{1/2}$620 iterations
  • Half: $\ell_{1/2}$725 iterations
  • Soft: $\ell_{1/2}$842 iterations
  • This demonstrates that half-thresholding achieves intermediate support detection speed and iterative complexity (Zeng et al., 2013).

Method	Coherence Bound	Empirical Iterations	Feature at Threshold
Hard	$\ell_{1/2}$9	$v\in\mathbb{R}^N$020	Discontinuous, unbiased
Half	$v\in\mathbb{R}^N$1	$v\in\mathbb{R}^N$225	Discontinuous, nonconvex
Soft	$v\in\mathbb{R}^N$3	$v\in\mathbb{R}^N$442	Continuous, biased

5. Parameter Selection, Implementation, and Computational Complexity

Parameter selection is scenario-dependent:

• AIT: The threshold is adaptively set to the $v\in\mathbb{R}^N$5-st largest magnitude, enforcing exact $v\in\mathbb{R}^N$6-sparsity for each iterate (Zeng et al., 2013).
• Proximal half-thresholding: The threshold $v\in\mathbb{R}^N$7 equals $v\in\mathbb{R}^N$8 (with corresponding critical $v\in\mathbb{R}^N$9), and $h_{\tau,1/2}$0 can be set by sparsity, cross-validation, or inspecting the sorted entries of the pre-thresholded iterate (Zhang et al., 2014).

Implementation is efficient:

• Each iteration requires two matrix-vector multiplications and $h_{\tau,1/2}$1 scalar thresholdings.
• Per-iteration complexity: $h_{\tau,1/2}$2, advantageous over IRLS and IRL1 ($h_{\tau,1/2}$3 due to matrix inversion/LP solves) for large $h_{\tau,1/2}$4 (Zeng et al., 2013).

The proximity operator is not continuous at threshold, and higher selectivity of sparse components is achieved compared to $h_{\tau,1/2}$5, particularly relevant for high dynamic-range signals or measurement matrices with moderate coherence (Zhang et al., 2014).

6. Numerical Performance and Empirical Comparisons

Extensive simulation studies confirm:

• For small $h_{\tau,1/2}$6, IRLS can be marginally faster due to efficient small-scale least squares (Zeng et al., 2013).
• For larger $h_{\tau,1/2}$7, half-thresholding substantially outperforms IRLS and IRL1.
• Recovery accuracy (MSE) of half-thresholding matches or improves upon traditional alternatives, providing high-precision sparse signal reconstruction (Zeng et al., 2013).

The discontinuous threshold mechanism imbues half-thresholding with robustness to noise and measurement coherence, consistently achieving support recovery with fewer iterations than soft-thresholding and greater stability than hard-thresholding (Zeng et al., 2013, Zhang et al., 2014).

7. Extensions and Connections

The closed-form half-thresholding operator exists due to the solvability of the scalar cubic arising from the $h_{\tau,1/2}$8 regularizer; analogous formulas in the $h_{\tau,1/2}$9 setting only appear for $[0, \infty)$0 and $[0, \infty)$1 (Zhang et al., 2014). Generalizations to transformed $[0, \infty)$2 (TL1) penalties have been developed, retaining robust sparsity promotion with explicit thresholding maps, and exhibit performance advantages in compressed sensing beyond half-thresholding (Zhang et al., 2014). The nonconvex nature of half-thresholding enhances sparsity beyond convex $[0, \infty)$3, motivating its adoption in fields including signal processing, statistical estimation, and high-dimensional machine learning.