Generic bound on active-set size for generalized LASSO with arbitrary sparsifying transform

Determine a generic upper bound on the cardinality of the active set |I^c| = |{j : (W x̂)_j ≠ 0}| for minimizers x̂ of the generalized LASSO J_α(x; y) = ||A x − y||_2^2 + α ||W x||_1 in finite dimensions, when W ∈ R^{p×n} is an arbitrary (not necessarily invertible) linear operator. The bound should be expressed as a function of the regularization parameter α and the data y and hold uniformly over all y, thereby extending the known O(1/α) estimate available when W = I or W is invertible.

Background

In the analysis of the reconstruction map defined by the generalized LASSO J_α(x; y) = ||A x − y||2 + α||W x||1, the authors establish Hausdorff Lipschitz continuity. On each affine region determined by the sign pattern of W x̂, the Lipschitz constant can be improved from 1/σ_min(A) to 1/σ_min(A P{ker(W_{I,:})}), where I denotes indices with (W x̂)_I = 0. Obtaining this improvement requires a priori control of the active-set size |Ic|.

For W = I (standard LASSO) or invertible W, the KKT conditions yield |Ic| ≲ 1/α. However, for general (possibly non-invertible) W, the authors state that they do not know a generic bound on |Ic| as a function of α and y. Establishing such a bound would enable sharper stability estimates than the naive 1/σmin(A) bound by allowing the use of restricted singular values σ_min(A P{ker(W_{I,:})}).

References

If W = I (namely for LASSO reconstruction) or if W is invertible, this directly leads to an estimate on |Ic| that scales as 1 / α. However, the situation is more challenging in our general setting, and we are not aware of a generic bound.

— A remark on an error analysis for classical and learned Tikhonov regularization schemes  (2604.00759 - Behrens et al., 1 Apr 2026) in Section 'Learned regularization terms' (paragraph beginning “Now, we look into estimates for κ”)