Signal Lasso with Non-Convex Penalties

• Signal Lasso with non-convex penalties is a framework that replaces the traditional ℓ1 penalty with alternatives like ℓ0, MCP, and SCAD to enhance sparsity recovery and reduce amplitude bias.
• The method employs various penalties to align with specific signal priors, improving support recovery in applications ranging from compressed sensing to network reconstruction.
• Efficient optimization strategies such as ADMM, MADMM, and proximal splitting are used to solve these non-convex models while providing rigorous error bounds and recovery guarantees.

Signal lasso with non-convex penalties denotes a family of sparse estimation methods that retain the least-squares-plus-regularization structure of the Lasso while replacing, modifying, or augmenting the convex $\ell_1$ penalty by non-convex penalties such as $\ell_0$, MCP, SCAD, sorted $\ell_1$, trimmed penalties, $\ell_1/\ell_2$ ratios, and binary-target penalties. Across the literature, the term covers both classical sparse signal recovery and a more specialized network-reconstruction setting in which coefficients are shrunk toward the binary states $0$ and $1$. The unifying objective is to reduce the amplitude bias of $\ell_1$, strengthen sparsity promotion, or encode structural priors that convex Lasso does not represent directly (He et al., 2018).

1. Core formulations and the role of non-convex regularization

A standard starting point is the sparse recovery objective

$\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$

with observed stimulus $\mathbf{s}$, dictionary $\bm{\Phi}$, sparse code $\ell_0$0, and sparsity-inducing regularizer $\ell_0$1. In the classical spiking sparse recovery setting, the corresponding SNN algorithm solves the nonnegative or constrained LASSO,

$\ell_0$2

whereas the non-convex extension replaces $\ell_0$3 by $\ell_0$4 in order to avoid underestimation of large coefficients (Zhang et al., 2020).

In saturated-measurement compressive sensing, the measurements are split into unsaturated data $\ell_0$5 and saturated data $\ell_0$6, with sign vector $\ell_0$7. The recovery model proposed for mixed one-bit compressive sensing with linear loss is

$\ell_0$8

Here the quadratic term fits unsaturated measurements, while the linear term replaces hard sign-consistency constraints or hinge losses and makes non-convex penalties practical within the same framework (He et al., 2018).

A distinct but related usage appears in network reconstruction. There the coefficient vector is assumed to satisfy the signal property $\ell_0$9, and signal lasso penalizes deviation from the two target states rather than shrinkage only toward zero. The proposed non-convex criteria include

$\ell_1$0

and

$\ell_1$1

which implement two-direction shrinkage toward $\ell_1$2 and $\ell_1$3 (Shi et al., 30 Aug 2025).

The common motivation is consistent across these settings. The $\ell_1$4 norm is computationally attractive but tends to underestimate amplitudes, whereas non-convex penalties punish large coefficients less heavily or align directly with structured alphabets and binary states. This motivates signal-lasso-type models in which the data-fit term remains familiar but the regularizer is altered to better match the target signal class (Fosson, 2018).

2. Penalty families and induced signal models

The literature does not use a single non-convex penalty; instead it employs a range of penalties, each tied to a specific signal prior or geometric effect.

Penalty family	Representative form	Typical role
$\ell_1$5	$\ell_1$6	Direct sparsity promotion
MCP	$\ell_1$7 for $\ell_1$8; $\ell_1$9 otherwise	Reduced shrinkage bias
SCAD	Piecewise $\ell_1/\ell_2$0, quadratic transition, constant tail	$\ell_1/\ell_2$1 near zero, flat for large coefficients
sorted $\ell_1/\ell_2$2	$\ell_1/\ell_2$3	Ordered shrinkage
Trimmed Lasso	$\ell_1/\ell_2$4	Exact control over selected model size
Ratio penalty	$\ell_1/\ell_2$5	Scale-invariant sparse geometry
Binary signal penalties	$\ell_1/\ell_2$6 or $\ell_1/\ell_2$7	Shrinkage toward $\ell_1/\ell_2$8

Within sparse recovery from saturated measurements, the regularizer $\ell_1/\ell_2$9 can be $0$0, $0$1, MCP, or sorted $0$2. The paper explicitly emphasizes that linear loss makes these non-convex penalties much easier to incorporate than hinge-loss or hard-constraint formulations (He et al., 2018).

SCAD and MCP are the canonical folded-concave penalties in compressed sensing and generalized linear modeling. For SCAD,

$0$3

and for MCP,

$0$4

Both are exactly $0$5-like near the origin and nearly flat for large coefficients, thereby reducing bias relative to Lasso (Sakata et al., 2019).

Other models tailor the penalty to structural priors. The trimmed Lasso penalizes only the smallest $0$6 coefficients,

$0$7

which yields direct control over the number of selected variables (Bertsimas et al., 2017). The graph-signal formulation uses the non-convex, non-smooth ratio

$0$8

which is zero-homogeneous and is meant to force the solution toward the sparse part of the $0$9-sphere (Bresson et al., 2015). In fused-lasso signal approximation, the non-convex penalty is introduced through $1$0 in both the signal and first-difference terms,

$1$1

so that sparsity and piecewise constancy are both regularized by non-convex functions (Parekh et al., 2015).

For finite-valued sparse signals, MCP is aligned with the discrete alphabet. In the ternary case $1$2, the proposed objective is

$1$3

where the negative quadratic term counteracts shrinkage and pushes components toward $1$4 rather than merely toward small nonzero values (Fosson, 2018).

3. Optimization architectures

The algorithmic value of signal lasso with non-convex penalties depends heavily on how the regularizer interacts with the loss. In the saturated-measurement problem, introducing an auxiliary variable $1$5 and enforcing $1$6 yields an ADMM splitting in which the $1$7-subproblem is quadratic: $1$8 Its closed-form solution is

$1$9

This removes the inner iterative solver required by constrained or hinge-loss formulations and is the main computational advantage of the linear-loss model (He et al., 2018).

For finite-valued sparse recovery, the MCP-Lasso formulation is solved by MADMM. In the ternary case, the split problem

$\ell_1$0

leads to projected linear solves and soft-thresholding updates. The algorithm is designed for a non-convex objective but converges to the set of stationary points, and the paper also proposes MADMM-R, a reshuffling version that restarts when the algorithm fails to land on the discrete alphabet in the noise-free case (Fosson, 2018).

A different strategy appears in convex non-convex fused lasso. There the non-convex penalty is decomposed as $\ell_1$1, and because $\ell_1$2 is concave, it is majorized by its tangent. Each MM step reduces to a standard $\ell_1$3 fused lasso problem with modified data

$\ell_1$4

followed by

$\ell_1$5

No matrix inverses are required, and the taut-string algorithm can be used for the TV subproblem (Parekh et al., 2015).

For graph-sparse recovery, the objective

$\ell_1$6

is handled by a proximal forward-backward splitting scheme with an inner accelerated primal-dual loop. The outer iteration computes a forward step using the subgradient of $\ell_1$7, while the backward step solves a proximal problem for $\ell_1$8 plus the quadratic fidelity term (Bresson et al., 2015).

Spiking implementations replace explicit proximal iterations by dynamics. In adaptive spiking sparse recovery, the adaptive leakage current is

$\ell_1$9

with firing rate

$\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$0

Because $\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$1 decreases with $\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$2, large coefficients receive weaker shrinkage. The recovered sparse coefficients are the firing rates of the integrate-and-fire system (Zhang et al., 2020).

Non-convexity also drives solver design in broader penalized estimation. The trimmed Lasso admits DC, ADMM, and mixed-integer optimization formulations; the mixed-integer version gives certified global optimality but at higher computational cost, whereas DC and ADMM are heuristic or local methods (Bertsimas et al., 2017). The package yaglm implements local linear approximation for folded-concave penalties such as SCAD and MCP by replacing the non-convex penalty with a weighted convex surrogate at each iteration, thereby reducing the problem to a sequence of adaptive sparse estimation subproblems (Carmichael et al., 2021). In the binary network-reconstruction setting, the non-convex penalties are optimized by coordinate descent with explicit one-dimensional update rules for both the product and minimum penalties (Shi et al., 30 Aug 2025).

4. Recovery guarantees, convexity control, and asymptotic theory

The theoretical landscape splits into two broad regimes. One regime uses non-convex penalties while constraining parameters so that the total objective remains convex; the other accepts genuine non-convex optimization and proves stationarity, error bounds, or statistical phase-transition results.

For saturated measurements with linear loss, the paper proves an estimation error bound under Gaussian sensing and mild noise assumptions. A key lemma establishes alignment between the sign information and the true signal,

$\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$3

With high probability, a deviation bound of the form

$\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$4

holds, leading to explicit $\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$5-error bounds for both $\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$6 and $\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$7 regularization (He et al., 2018).

For adaptive spiking sparse recovery, the main theorem states that the average soma current $\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$8 and the firing rate $\underset{\mathbf{a}\in \mathbb{R}^N }{\operatorname{min}\ \frac{1}{2}\|\mathbf{s}-\bm{\Phi} \mathbf{a}\|_{2}^{2}+\lambda C(\mathbf{a}),\quad \lambda>0,$9 are globally asymptotically convergent, and that $\mathbf{s}$0 converges to a critical point of the non-convex objective. The proof is organized through boundedness of the dynamics, convergence of an auxiliary ODE system, and asymptotic equivalence between the spiking system and that auxiliary system (Zhang et al., 2020).

Finite-valued MCP-Lasso obtains a stronger form of recovery in the noise-free discrete setting. Under robust null space properties with

$\mathbf{s}$1

the true ternary signal is the unique global minimizer of the MCP-Lasso functional over $\mathbf{s}$2. In the noisy model $\mathbf{s}$3, the paper also derives robustness bounds in $\mathbf{s}$4 norm (Fosson, 2018).

Convexity-preserving non-convex regularization is formalized very explicitly in fused-lasso and MSC formulations. For convex non-convex fused lasso, the total objective remains strictly convex if

$\mathbf{s}$5

For maximally sparse convex optimization, the full least-squares-plus-penalty objective is strictly convex when the penalty parameters satisfy

$\mathbf{s}$6

where $\mathbf{s}$7 is chosen so that $\mathbf{s}$8. The non-convexity is thus limited by the curvature of the quadratic data-fit term [(Parekh et al., 2015); (Selesnick et al., 2013)].

The trimmed Lasso sits between continuous regularization and explicit subset selection. For each fixed $\mathbf{s}$9, there exists $\bm{\Phi}$0 such that, for all $\bm{\Phi}$1, the trimmed-Lasso-regularized least squares problem coincides with the constrained best-subset problem with $\bm{\Phi}$2. This gives an exact sparsity-control result that ordinary Lasso does not provide (Bertsimas et al., 2017).

Replica and AMP analysis provide a different type of guarantee for SCAD and MCP in compressed sensing. Under a Bernoulli–Gaussian sparse prior and i.i.d. Gaussian sensing, the success solution is

$\bm{\Phi}$3

The analytically derived threshold for perfect reconstruction improves on $\bm{\Phi}$4, and for small non-convexity parameters the recovery limit can approach $\bm{\Phi}$5. At the same time, plain AMP may fail to realize that improvement because the basin of attraction to the perfect-reconstruction fixed point shrinks, which motivates continuation in the non-convexity parameter $\bm{\Phi}$6 as a control strategy (Sakata et al., 2019).

5. Empirical behavior across problem classes

Empirical results consistently report that non-convex penalties improve amplitude recovery or support recovery relative to plain $\bm{\Phi}$7, but the nature of the gain depends on the signal class.

In saturated compressive sensing, the linear-loss method with $\bm{\Phi}$8 performs similarly to M1bit-CSC in SNR and angular error when $\bm{\Phi}$9, $\ell_0$00, $\ell_0$01, and the saturation ratio varies up to $\ell_0$02. When non-convex penalties are activated, sorted $\ell_0$03 and MCP significantly outperform the comparison methods in SNR and AE, whereas the $\ell_0$04 penalty is less stable because optimization can get trapped in poor local minima. Runtime reductions are especially large: at $\ell_0$05, $\ell_0$06, reported average times are $\ell_0$07 s for RDCS, $\ell_0$08 s for M1bit-CSC, $\ell_0$09 s for Alg. 1–L1, $\ell_0$10 s for Alg. 1–MCP, $\ell_0$11 s for Alg. 1–L0, and $\ell_0$12 s for Alg. 1–sL1 (He et al., 2018).

In spiking sparse recovery, the first experiment with a 3-neuron SNN and exponential penalty reports NMSE values of $\ell_0$13 dB for SSR and $\ell_0$14 dB for A-SSR. Across the larger experiments on $\ell_0$15 in both noise-free and noisy settings, all A-SSR variants outperform SSR in accuracy, the exponential penalty performs best among the reported non-convex choices, A-SSR is more robust to noise, and convergence is faster than SSR (Zhang et al., 2020).

Finite-valued MCP-Lasso is evaluated on randomly generated data and a localization problem. The reported findings are higher estimation accuracy, fewer measurements needed for the same recovery rate, and faster convergence than Lasso/BP-type methods. In a tested noise-free ternary regime, MADMM-R achieves exact recovery with about $\ell_0$16, and in multiple-target localization MADMM yields slightly lower localization error while converging in significantly fewer iterations than Lasso, especially when the number of sensors is small (Fosson, 2018).

For piecewise-constant denoising, convex non-convex fused lasso yields lower RMSE than standard $\ell_0$17 fused lasso and modified fused lasso in the reported pulse-detection experiments. In the ECG R-wave example with

$\ell_0$18

the non-convex method detects R-waves with larger amplitudes and better visibility, whereas the $\ell_0$19 method underestimates amplitudes and the Pan-Tompkins detector produces several false positives under strong AWGN (Parekh et al., 2015).

Graph-sparse recovery with the $\ell_0$20 penalty also shows consistent improvements. On LFR, MNIST, and 20NEWS graphs, the reported recovery errors are $\ell_0$21 vs $\ell_0$22, $\ell_0$23 vs $\ell_0$24, and $\ell_0$25 vs $\ell_0$26 for standard versus proposed graph Lasso. In the inpainting setting the corresponding results are $\ell_0$27 vs $\ell_0$28, $\ell_0$29 vs $\ell_0$30, and $\ell_0$31 vs $\ell_0$32. The method is reported to be $\ell_0$33–$\ell_0$34 times slower than standard Lasso but computationally practical (Bresson et al., 2015).

In network topology inference, the non-convex signal-lasso variants SL$\ell_0$35 and SL$\ell_0$36 are reported to outperform convex SigL and often ASigL in low or moderate noise and in denser networks, while ASigL is often more robust under high noise variance. The CPU table shows SL$\ell_0$37 and SL$\ell_0$38 to be dramatically faster than ASigL and SigL, and in the real behavioral experiment SL$\ell_0$39 performs best in the ring-network treatment while both non-convex methods achieve nearly zero UCR (Shi et al., 30 Aug 2025).

6. Conceptual distinctions, limitations, and practical interpretation

A central distinction in this area is between non-convex penalties used inside a globally convex objective and non-convex penalties used in genuinely non-convex optimization. Convex non-convex fused lasso and maximally sparse convex optimization belong to the first class: they attempt to obtain the bias-reduction benefits of concave penalties while preserving strict convexity of the total cost [(Parekh et al., 2015); (Selesnick et al., 2013)]. SCAD/MCP compressed sensing, trimmed Lasso, graph $\ell_0$40, spiking sparse recovery, and binary signal lasso belong to the second class: they accept local minima or stationary-point solutions in exchange for stronger sparsity geometry (Sakata et al., 2019, Bertsimas et al., 2017).

A recurrent misconception is that “more non-convex” is always better. The literature is more qualified. In saturated compressive sensing, the $\ell_0$41 penalty is reported to be less stable because optimization can get trapped in poor local minima (He et al., 2018). In SCAD/MCP compressed sensing, stronger non-convexity can improve the analytic recovery threshold while simultaneously shrinking the basin of attraction of the perfect-reconstruction fixed point, so plain AMP may diverge or miss the favorable solution (Sakata et al., 2019). In network reconstruction, ASigL can outperform the proposed non-convex penalties at high noise, and SL$\ell_0$42 is less robust than SL$\ell_0$43 when zero columns appear in the design matrix (Shi et al., 30 Aug 2025).

Another important distinction is between separable and non-separable penalties. MCP, SCAD, log, and related folded-concave penalties are coordinate-wise separable; trimmed Lasso is explicitly non-separable because it couples coordinates through sorting. The trimmed-Lasso paper further shows that trimmed penalties subsume existing coordinate-wise separable penalties in a strict-containment sense in general and places trimmed Lasso in a complementary relation with SLOPE/OWL through robust statistics and robust optimization viewpoints (Bertsimas et al., 2017).

Software support reflects this diversity. yaglm exposes adaptive and non-convex penalties including SCAD and MCP, supports structured variants such as group, fused, generalized, and singular-value penalties, and uses LLA, FISTA, ADMM, or user-supplied solvers. It also provides cross-validation, information criteria such as AIC, BIC, EBIC, HBIC, and ERIC, and degrees-of-freedom-based criteria, which is directly relevant because prediction-oriented tuning and support-recovery-oriented tuning need not coincide (Carmichael et al., 2021).

Taken together, these results suggest that “signal lasso with non-convex penalties” is not a single method but a design principle. The principle is to retain the statistical and computational scaffold of Lasso-like estimation while replacing the $\ell_0$44 penalty by a regularizer whose geometry better matches the target signal: saturated but signed measurements, finite-valued alphabets, sparse graph Fourier coefficients, sparse piecewise-constant signals, approximate $\ell_0$45-sparsity, or binary adjacency variables. The practical efficacy of the approach therefore depends less on non-convexity in the abstract than on how precisely the penalty encodes the signal prior and how well the optimization scheme exploits that structure.