JSRL: Joint Sparsity-Ranked LASSO

• JSRL is a regularization framework that leverages convex surrogates for both low-rank and joint-sparse constraints to enhance multi-dimensional estimation.
• It employs proximal splitting algorithms under a joint Restricted Isometry Property to ensure efficient and stable recovery of sparse and low-rank structures.
• The method is widely applicable in compressed sensing, multi-task learning, feature selection, and neuroimaging, with extensions to Bayesian and nonconvex models.

The Joint Sparsity-Ranked LASSO (JSRL) is a regularization framework designed to simultaneously promote low-rank and joint-sparse structures in multi-dimensional estimation problems. The method combines convex surrogates for rank (nuclear norm) and joint sparsity (mixed $\ell_{2,1}$ norm), enabling efficient and theoretically robust estimation of matrices that exhibit both structural constraints. JSRL is foundational in modern compressed sensing, multi-task learning, feature selection, and high-dimensional neuroimaging, with strong connections to group LASSO, sparse principal component analysis, and information-theoretic model selection.

1. Mathematical Formulation and Interpretative Framework

JSRL seeks estimators for matrix-valued variables $X \in \mathbb{R}^{n_1 \times n_2}$ that are both low-rank and joint-sparse, typically in the context of multichannel measurements. The canonical convex formulation replaces the nonconvex rank and support penalties by their respective convex relaxations: $$\begin{aligned} \text{minimize} \quad & \|X\|_{2,1} + \lambda \|X\|_* \ \text{subject to} \quad & \|y - \mathcal{A}(X)\|_2 \leq \epsilon \end{aligned}$$ where $\|X\|_{2,1} = \sum_i \|X_{i,\cdot}\|_2$ enforces joint row sparsity and $\|X\|_*$ is the nuclear (trace) norm promoting low rank. Here, $y$ are compressed (possibly noisy) linear measurements, $\mathcal{A}$ is a linear operator, and $\lambda > 0$ balances the two regularization effects.

This formulation generalizes ranked LASSO approaches, as it penalizes both groupwise support and rank simultaneously. In the context of feature selection or principal component regression, the concept has been extended to settings where penalties are ranked (or weighted non-uniformly) by prior beliefs or empirical features, allowing group- and index-wise evidence requirements to be explicitly encoded (Peterson et al., 2021, Rieck et al., 9 Sep 2025).

2. Restricted Isometry Property for Simultaneous Structure

The stability and recoverability of JSRL largely depend on the measurement operator $\mathcal{A}$ satisfying a joint Restricted Isometry Property (RIP), formulated for matrices with simultaneous rank and joint-sparsity constraints: $$(1 - \delta_{r, k}) \|X\|_F^2 \leq \|\mathcal{A}(X)\|_2^2 \leq (1 + \delta_{r, k}) \|X\|_F^2$$ for all $X$ of rank at most $r$ and with at most $k$ nonzero rows. This “joint” RIP ensures that $\mathcal{A}$ nearly preserves the Euclidean geometry of the set of feasible solutions. For suitable random measurement schemes (subgaussian ensembles), one can derive near-optimal sampling bounds, guaranteeing that the number of measurements required for stable recovery scales with the intrinsic degrees of freedom, i.e., $m \gtrsim k \log(n_1 / k) + k r + n_2 r$ (Golbabaee et al., 2012, Foucart, 2022).

The RIP adapts classical theory from single-signal compressed sensing and group LASSO to simultaneous low-rank and group-sparse situations, generalizing guarantees to high-dimensional matrix recovery.

3. Proximal Algorithms and Split Convex Optimization

Because JSRL entails minimization of a convex but nonsmooth objective (nuclear norm and mixed norm), standard gradient descent is not applicable. Proximal splitting algorithms—most notably Parallel Proximal Algorithms (PPXA)—are implemented to solve the problem. The three central proximity operators are:

• Mixed $\ell_{2,1}$ norm: row-wise soft thresholding,

$$(\text{prox}_{\alpha,\,\|\cdot\|_{2,1}}(X))_{i,\cdot} = \left(1 - \frac{\alpha}{\|X_{i,\cdot}\|_2}\right)_+ X_{i,\cdot}$$

• Nuclear norm: singular value thresholding,

$$\text{prox}_{\alpha,\,\|\cdot\|_*}(X) = U \,\operatorname{diag}((\sigma_i - \alpha)_+) V^\top$$

where $X = U \Sigma V^\top$.

• Data fidelity constraint: Euclidean projection,

$$\text{prox}_{\text{indicator},\,\mathcal{C}}(X) = \text{Proj}_{\{X: \|y - \mathcal{A}(X)\|_2 \leq \epsilon\}}(X)$$

At each iteration, these operators are computed in parallel, their outputs averaged, and the solution updated. This enables efficient non-smooth optimization on large-scale problems (Golbabaee et al., 2012, Fan et al., 2014).

4. Phase Transitions and Empirical Performance

Phase transition experiments reveal sharp boundaries between successful and unsuccessful recovery regimes as a function of the subsampling ratio $m / (n_1 n_2)$ and row sparsity ratio $k / n_1$, visualizing the interaction between rank and sparsity constraints. Within the recovery zone, near-perfect reconstruction is achieved; outside, errors are large. The required measurement count is dramatically reduced when both low-rank and joint sparsity are exploited, compared to using either constraint alone (Golbabaee et al., 2012).

Empirical evidence from synthetic and application-oriented data (sensor networks, sparse principal component analysis) confirms the theoretical bound and demonstrates substantial performance improvements over plain $\ell_{2,1}$ minimization, especially in high-dimensional, multichannel scenarios.

5. Extensions: Bayesian, Nonconvex, and Hierarchical Models

Recent work has extended JSRL via:

• Nonconvex joint sparsity: Truncated or weighted models use adaptive binary weights on joint-sparse structures, with multi-stage convex relaxation schemes for support identification (e.g., ISDJS), and thresholding heuristics to overcome the limitations of convex shrinkage for uniform-magnitude signals (Fan et al., 2014).
• Rank-aware regularization: Orthogonally weighted $\ell_{2,1}$ (ow$\ell_{2,1}$) regularizers normalize magnitude information, making the penalty directly sensitive to solution rank. The nonconvex metric attains the exact row sparsity for full-rank solutions, outperforming rank-blind methods in feature selection and dictionary learning (Petrosyan et al., 2023).
• Bayesian joint spike-and-slab and hierarchical sparsity priors: Flexible mixture and hierarchical models allow adaptive penalty selection per edge (or group), enabling bias reduction and interpretable selection in Gaussian graphical models and multi-coil MRI reconstruction (Li et al., 2018, Glaubitz et al., 2023). In hierarchical Bayesian settings, joint-sparsity-promoting hyper-priors with gamma distributions on variance-control parameters provide efficient edge structure inference in multi-measurement settings.

6. Applications in Multi-task Learning, Feature Selection, and Neuroimaging

JSRL underpins methodologies across multi-task regression—where random effects models link joint sparsity to shared covariance estimation (Balasubramanian et al., 2013)—and model selection, interaction, and polynomial term ranking, where penalty scaling ensures appropriate evidence thresholds (Peterson et al., 2021, Peterson et al., 2022). In time series, ranked penalties enable efficient seasonality selection and exogenous variable incorporation, implemented via scalable packages (fastTS) (Peterson et al., 2022).

The hybrid neuroimaging approach combines principal component ranking and voxel-level penalties to produce interpretable brain maps and improved classification accuracy. Information parity scaling aligns Fisher information between PCs and voxel predictors, and the ranking exponent is tuned by cross-validation to maximize signal recovery in multi-voxel pattern analysis (Rieck et al., 9 Sep 2025).

7. Theoretical and Algorithmic Guarantees

Analysis of LASSO-type minimizers under generalized restricted isometry reveals that, for appropriately chosen regularization and measurement ensembles, JSRL minimizers have sparsity comparable to the true signal—even with moderate observation noise (Foucart, 2022). Extensions to adaptive, group, and hierarchical prioritization are justified by these theoretical results.

The strongly consistent performance across sampling regimes, rapid empirical convergence, and multi-channel generalizability position JSRL as a theoretically robust and practically effective methodology for high-dimensional statistical inference.

---

The Joint Sparsity-Ranked LASSO thus subsumes classical group/structured regularization, integrates ranked and adaptive penalty mechanisms, and enables stable and efficient recovery of signal structure in multi-output, matrix, and high-dimensional problems. Its theoretical foundation, rich algorithmic toolkit, and empirical validation across diverse application domains make it central in contemporary high-dimensional data analysis.