---
title: Half-Thresholding Rule in Sparse Recovery
url: https://www.emergentmind.com/topics/half-thresholding-rule
type: topic
---

# Half-Thresholding Rule in Sparse Recovery

The half-thresholding rule is a nonlinear scalar mapping and associated iterative thresholding scheme, fundamental to sparse regularization via nonconvex $\ell_{1/2}$-type penalties. It admits a closed-form, non-monotone thresholding operator, and is central to both adaptively iterative thresholding in underdetermined systems and fixed-parameter $\ell_{1/2}$-regularized optimization. The half-thresholding operator is distinguished by its discontinuity at the threshold and its strong sparsity-promoting properties, offering a balance between the bias of soft-thresholding and the instability of hard-thresholding. Rigorous convergence guarantees, complexity estimates, and comparative analyses support its advantages in high-dimensional sparse recovery applications [1310.3954][1311.0156][1412.5240].

## 1. The Half-Thresholding Operator: Definition and Construction

The half-thresholding operator, denoted here as $h_{\tau,1/2} : \mathbb{R} \rightarrow \mathbb{R}$, is defined for a scalar $u$ and threshold $\tau > 0$ by
\[
h_{\tau,1/2}(u) = 
\begin{cases}
\displaystyle
\frac{2}{3} u \left(1 + \cos\left(\frac{2\pi}{3} - \frac{2}{3} \arccos\left( \frac{\sqrt{2}}{2} \left(\frac{\tau}{|u|}\right)^{3/2} \right) \right) \right) & |u| > \tau \\
0 & |u| \le \tau
\end{cases}
\]
as developed for adaptively iterative thresholding (AIT) [1310.3954] and equivalently for proximal solutions to an $\ell_{1/2}$-regularized quadratic subproblem [1311.0156][1412.5240]. Applied to vectors $v\in\mathbb{R}^N$, the operator acts componentwise. 

Key properties:
- $h_{\tau,1/2}$ is odd, strictly nondecreasing on $[0, \infty)$, and admits explicit lower and upper bounds: $u - \frac{\tau}{3} \le h_{\tau,1/2}(u) \le u$ for all $u \ge \tau$.
- The map is discontinuous at the threshold $|u|=\tau$, where the operator jumps from $0$ to a strictly positive value, similar to hard-thresholding but unlike the continuous soft-thresholding transition [1412.5240].
- The closed-form arises from solving a depressed cubic in the subproblem $\min_z \frac12 (z-u)^2 + \tau |z|^{1/2}$ using Cardano’s formula.

Alternative parameterization appears in the fixed-step iterative thresholding setting for $\ell_{1/2}$ regularization, where $\tau = \lambda \mu$, with $\lambda$ the regularization parameter and $\mu$ the step size [1311.0156].

## 2. Application in Iterative Schemes: AIT and Proximal Algorithms

The half-thresholding rule underpins two principal classes of algorithms:

**(A) Adaptively Iterative Thresholding (AIT):** For an underdetermined linear system $y = A x$, where $A \in \mathbb{R}^{M \times N}$, the AIT with half-thresholding seeks a $k$-sparse solution by:
1. Initializing $x^{(0)} = 0$.
2. Iteratively:  
    (a) Compute $z^{(t+1)} = x^{(t)} + A^T(y - A x^{(t)})$ (Landweber step).  
    (b) Set $\tau^{(t+1)}$ to the $(k+1)$-st largest entry in $|z^{(t+1)}|$.  
    (c) Update $x^{(t+1)} = H^{(1/2)}_{\tau^{(t+1)}}(z^{(t+1)})$.  
    (d) Stop when a chosen criterion is met [1310.3954].

**(B) Proximal Gradient Method for $\ell_{1/2}$ Regularization:** For
\[
\min_{x \in \mathbb{R}^N} \frac12 \|A x - y\|_2^2 + \lambda \sum_{i=1}^N |x_i|^{1/2}
\]
the iterative scheme applies:
\[
x^{k+1} = T_{\lambda, \mu}\left( x^k - \mu A^T (A x^k - y) \right)
\]
with $T_{\lambda, \mu}$ the closed-form half-thresholding operator, threshold $\tau = \frac32 (\lambda \mu)^{2/3}$, and $0 < \mu < \|A\|_2^{-2}$ [1311.0156][1412.5240].

Both methods achieve per-iteration complexity $O(m N)$, with scalar thresholding dominating the update after the two matrix-vector multiplications [1310.3954][1311.0156].

| Algorithmic Setting     | Threshold Update          | Support Size Control | Reference      |
|------------------------|--------------------------|---------------------|---------------|
| AIT (sparse recovery)  | Adaptive by sparsity $k$ | Exactly $k$         | [1310.3954]   |
| $\ell_{1/2}$ Proximal  | Fixed or cross-validated | Data-driven         | [1311.0156]   |

## 3. Theoretical Guarantees and Convergence Analysis

Comprehensive convergence results are established under explicit measurement matrix coherence or restricted isometry assumptions:

- **AIT with half-thresholding:**  
  The algorithm recovers the true support of $x^*$ in finitely many steps provided the coherence $\mu = \max_{i \neq j} |\langle A_i, A_j \rangle|$ obeys $\mu < \frac{3}{10 k^*}$, with $k^*$ the true sparsity [1310.3954]. Support identification is guaranteed in at most $T^*_{k^*}$ steps (constant dependent on $k^*, \mu$, and dynamic range).  
  Once support is identified, the iterates $x^{(t)}$ converge exponentially fast to $x^*$: 
  \[
  \| x^{(t)} - x^* \|_\infty \le \frac{3 + c}{2} \min_{i \in \mathrm{supp}(x^*)} |x_i^*| \rho^{t-t^*+1}
  \]
  where $c=1/3$, $\rho = \frac{4}{3} k \mu < 1/2$ [1310.3954].

- **Iterative half-thresholding for $\ell_{1/2}$ regularization:**  
  Under $0 < \mu < \|A\|_2^{-2}$, the sequence $\{x^k\}$ converges to a stationary point $x^*$. Local minimality is ensured for sufficiently small $\lambda$ or well-conditioned $A_I$, and eventual linear convergence rate ($\|x^{k+1}-x^*\|_2 \leq p \|x^k-x^*\|_2$, $p<1$, for large $k$) is achieved [1311.0156].

- **Continuity at the threshold:** Half-thresholding is discontinuous at the threshold, inducing more aggressive sparsity than the continuous soft-thresholding map of the $\ell_1$ case [1412.5240]. 

## 4. Comparison with Hard and Soft Thresholding Schemes

Half-thresholding occupies an intermediate position between hard ($\ell_0$) and soft ($\ell_1$) thresholding:

- **Coherence constraints:**  
    - Hard: $\mu < 1/(3k^*)$
    - Half: $\mu < 3/(10k^*) \approx 1/(3.33k^*)$
    - Soft: $\mu < 1/(4k^*)$  
  The half-thresholding rule requires a slightly more restrictive coherence bound than hard-thresholding, but is less restrictive than soft [1310.3954].

- **Practical implications:**  
    - Hard-thresholding is unbiased but becomes unstable near the coherence limit.
    - Soft-thresholding introduces bias for large coefficients but is robust.
    - Half-thresholding achieves a tradeoff, with reduced bias compared to soft and enhanced stability compared to hard.

- **Empirical iteration counts** (example: $k^* = 9$, $\mu = 1/40$, dynamic range 10):  
    - Hard: $\sim$20 iterations  
    - Half: $\sim$25 iterations  
    - Soft: $\sim$42 iterations  
  This demonstrates that half-thresholding achieves intermediate support detection speed and iterative complexity [1310.3954].

| Method           | Coherence Bound       | Empirical Iterations | Feature at Threshold     |
|------------------|----------------------|---------------------|--------------------------|
| Hard             | $1/(3k^*)$           | $\sim$20            | Discontinuous, unbiased  |
| Half             | $3/(10k^*)$          | $\sim$25            | Discontinuous, nonconvex |
| Soft             | $1/(4k^*)$           | $\sim$42            | Continuous, biased       |

## 5. Parameter Selection, Implementation, and Computational Complexity

Parameter selection is scenario-dependent:

- **AIT:** The threshold is adaptively set to the $(k+1)$-st largest magnitude, enforcing exact $k$-sparsity for each iterate [1310.3954].
- **Proximal half-thresholding:** The threshold $\tau$ equals $\lambda\mu$ (with corresponding critical $t_{1/2} \sim \tau^{2/3}$), and $\lambda$ can be set by sparsity, cross-validation, or inspecting the sorted entries of the pre-thresholded iterate [1412.5240].

Implementation is efficient:
- Each iteration requires two matrix-vector multiplications and $O(N)$ scalar thresholdings.
- Per-iteration complexity: $O(mN)$, advantageous over IRLS and IRL1 ($O(m N^2)$ due to matrix inversion/LP solves) for large $N$ [1311.0156].

The proximity operator is not continuous at threshold, and higher selectivity of sparse components is achieved compared to $\ell_1$, particularly relevant for high dynamic-range signals or measurement matrices with moderate coherence [1412.5240].

## 6. Numerical Performance and Empirical Comparisons

Extensive simulation studies confirm:
- For small $N$, IRLS can be marginally faster due to efficient small-scale least squares [1311.0156].
- For larger $N$, half-thresholding substantially outperforms IRLS and IRL1.
- Recovery accuracy (MSE) of half-thresholding matches or improves upon traditional alternatives, providing high-precision sparse signal reconstruction [1311.0156].

The discontinuous threshold mechanism imbues half-thresholding with robustness to noise and measurement coherence, consistently achieving support recovery with fewer iterations than soft-thresholding and greater stability than hard-thresholding [1310.3954][1412.5240].

## 7. Extensions and Connections

The closed-form half-thresholding operator exists due to the solvability of the scalar cubic arising from the $\ell_{1/2}$ regularizer; analogous formulas in the $0 < p < 1$ setting only appear for $p = 1/2$ and $p = 2/3$ [1412.5240]. Generalizations to transformed $\ell_1$ (TL1) penalties have been developed, retaining robust sparsity promotion with explicit thresholding maps, and exhibit performance advantages in compressed sensing beyond half-thresholding [1412.5240]. The nonconvex nature of half-thresholding enhances sparsity beyond convex $\ell_1$, motivating its adoption in fields including signal processing, statistical estimation, and high-dimensional machine learning.

Source: https://www.emergentmind.com/topics/half-thresholding-rule