Group Lasso: Structured Sparsity Method

Updated 25 June 2026

Group Lasso is a convex regularization method that uses the ℓ1,2 norm to promote sparsity at the group level, making it ideal for structured variable selection.
Efficient algorithms like block coordinate descent and Newton-ABS accelerate convergence while providing theoretical guarantees for consistent group support recovery.
Extensions such as overlapping, sparse, and exclusive group Lasso address complex data structures, improving estimation accuracy over traditional Lasso in high-dimensional settings.

The group Lasso is a convex regularization method for regression and classification that enforces group-wise sparsity in coefficient vectors or matrices. It extends the classical Lasso—which promotes elementwise sparsity via the $\ell_1$ norm—by employing a mixed norm, typically the $\ell_{1,2}$ norm, to induce sparsity at the level of user-specified groups of variables. The group Lasso is foundational in high-dimensional statistics, machine learning, multi-task learning, and structured variable selection, especially when intrinsic grouping of covariates is present (e.g., categorical variables, interaction effects, gene sets, multi-channel signals).

1. Mathematical Formulation

Let $X \in \mathbb{R}^{n \times p}$ be a data matrix and $y \in \mathbb{R}^n$ the response. The $p$ predictors are partitioned into $G$ groups $\{\mathcal{I}_g\}_{g=1}^G$ of possibly varying sizes $p_g = |\mathcal{I}_g|$ . For regression coefficients $\beta \in \mathbb{R}^p$ with group-specific subvectors $\beta_g \in \mathbb{R}^{p_g}$ , the standard group Lasso problem is

$\ell_{1,2}$ 0

with penalty weights $\ell_{1,2}$ 1 (commonly $\ell_{1,2}$ 2) and global regularization parameter $\ell_{1,2}$ 3 (Friedman et al., 2010, Foygel et al., 2010, Yang et al., 2024). The $\ell_{1,2}$ 4 norm $\ell_{1,2}$ 5 promotes sparse group support: entire blocks $\ell_{1,2}$ 6 are set to zero.

Variants extend this basic formulation: the sparse group Lasso adds an $\ell_{1,2}$ 7 penalty for within-group sparsity (Friedman et al., 2010, Zhang et al., 2017); the $\ell_{1,2}$ 8 group Lasso replaces the $\ell_{1,2}$ 9-norm by $X \in \mathbb{R}^{n \times p}$ 0 for any $X \in \mathbb{R}^{n \times p}$ 1 (Vogt et al., 2012); and the latent group Lasso addresses overlapping group structures by introducing latent variables (Obozinski et al., 2011).

2. Statistical Guarantees and Consistency

Group Lasso model selection and estimation consistency rely on generalized irrepresentable conditions and restricted eigenvalue (RE) assumptions. For linear models,

Sufficient Condition: Under finite fourth moments and invertibility of $X \in \mathbb{R}^{n \times p}$ 2, the group-Lasso achieves model selection consistency (recovery of group support) if the maximal scaled norm of certain submatrices stays below 1 (C-strict). This generalizes the Lasso's irrepresentable condition to group-structured settings. (0707.3390)
Necessary Condition: A weaker inequality (C-weak) is necessary for any sequence of group Lasso estimators to achieve consistency. (0707.3390, Dedieu, 2019)

In high dimensions, group Lasso achieves $X \in \mathbb{R}^{n \times p}$ 3 estimation rates of the form

$X \in \mathbb{R}^{n \times p}$ 4

where $X \in \mathbb{R}^{n \times p}$ 5 is the number of active groups and $X \in \mathbb{R}^{n \times p}$ 6 the total number of nonzero coefficients, provided appropriate group RE conditions hold (Dedieu, 2019). When the signal is group-sparse with relatively large groups, the group Lasso outperforms the standard (elementwise) Lasso in estimation error.

Adaptive group Lasso, with data-driven group weights, consistently recovers the support under milder requirements and parallels the theoretical properties of the adaptive Lasso (0707.3390).

3. Algorithmic Methods

Algorithms for the group Lasso exploit the block-separability of the penalty:

Block Coordinate Descent (BCD): The canonical approach cyclically updates $X \in \mathbb{R}^{n \times p}$ 7 for each group, typically via exact minimization (e.g., using the single line search (SLS) or Newton-ABS root-finding) (Foygel et al., 2010, Yang et al., 2024). For each block, the subproblem reduces to projecting a shifted residual onto a ball determined by the penalty. The Newton-ABS variant further accelerates block updates by combining bracketing with quadratic convergence (Yang et al., 2024).
Screening and Warm Starts: Active set strategies and pathwise regularization allow efficient computation along a decreasing grid of $X \in \mathbb{R}^{n \times p}$ 8 values, leveraging previous solutions to initialize subsequent ones (Yang et al., 2024).
Parallelization: The DC-gLasso framework distributes data shards, solves local group Lasso problems, and aggregates supports and coefficients using majority voting and averaging, with theoretical and empirical near-linear speedups (Chen et al., 2016).
Overlapping Groups: Latent group Lasso (Obozinski et al., 2011) and its proximal variants tackle the computational challenge of overlapping groups via variable duplication and block coordinate descent or dual-based accelerated gradient methods (Liu et al., 2010).
Sparse Group Lasso: Solved by alternating between soft-thresholding and groupwise updates, often using within-group coordinate descent for the nonsmooth penalty (Friedman et al., 2010, Foygel et al., 2010).

These algorithms all converge under convexity and separability, and their per-iteration complexity is dictated by the largest group size and data dimensions.

4. Structured and Extended Variants

Group Lasso has been generalized to support a variety of structured sparsity needs:

Overlapping Group and Latent Group Lasso: For settings where features belong to multiple groups (e.g., pathways, hierarchies), the latent group Lasso imposes blockwise penalties on latent variables constrained to sum to the estimator (Obozinski et al., 2011). The unit ball thus becomes the convex hull of all group-supported $X \in \mathbb{R}^{n \times p}$ 9-balls, permitting supports equal to unions of groups.
Exclusive Group Lasso: Reverses the order of summation and norm, employing $y \in \mathbb{R}^n$ 0 as a penalty to encourage sparsity within (rather than across) groups, promoting diversity across the active features (Gregoratti et al., 2021, Sun et al., 2020).
Hierarchical Sparse Modeling: Hierarchies encoded as DAGs induce constraints that can be enforced either via group Lasso (descendant-groups) or latent overlapping group Lasso (ancestor-groups), with markedly different patterns of shrinkage and bias (Yan et al., 2015). LOG regularization resolves the depth-dependent over-penalization in deep hierarchies.
Sparse Group Lasso: Adds an $y \in \mathbb{R}^n$ 1 term so that both group-level and within-group sparsity may be present (Friedman et al., 2010, Zhang et al., 2017). The associated optimization is more complex but blockwise subproblems remain tractable.
Generalized Linear and Poisson Models: Group Lasso extends naturally to GLMs with convex negative log-likelihood loss and admits sharp oracle inequalities and error rates analogous to the Gaussian case, provided the design matrix satisfies group-wise RE conditions. For Poisson GLMs, heteroscedasticity necessitates data-driven, concentration-based group weights (Blazère et al., 2013, Ivanoff et al., 2014).

5. Practical Implementation and Computational Considerations

Block coordinate descent and its Newton-ABS acceleration guarantee global convergence for convex loss and block-separable penalties (Yang et al., 2024). Key implementation aspects:

Group weights: The default $y \in \mathbb{R}^n$ 2 for group size normalization is robust; adaptive or pathway-specific weights may further reduce bias (Yang et al., 2024, Obozinski et al., 2011).
Warm starts and path algorithms: Tracing the $y \in \mathbb{R}^n$ 3-grid with previous solutions as initialization reduces total computation, especially in high-dimensional settings.
Screening rules: Strong rules leveraging group subdifferentials preemptively discard groups unlikely to be active, improving efficiency (Yang et al., 2024).
Handling non-orthonormal groups: Exact block updates require solving for nonzero solutions in general, often via blockwise root-finding rather than closed-form thresholds (Friedman et al., 2010, Foygel et al., 2010).
Solution accuracy: Modern group Lasso solvers achieve specified KKT suboptimality, and empirical benchmarks show they are several times faster than earlier coordinate or gradient-based implementations (Yang et al., 2024).
Parallel and distributed computing: DC-gLasso and related frameworks provide nearly linear scaling in the number of workers for very large datasets, at the cost of minimal accuracy loss (Chen et al., 2016).

Empirical studies confirm these techniques' benefits: compared to standard Lasso, the group Lasso achieves lower false positive rates when the truth is group-sparse, and yields models with strong interpretability and parsimony.

6. Applications and Extensions

Categorical Data: In high-dimensional settings with categorical predictors, the group Lasso is used for factor selection (group = factor levels), but final models may be dense within surviving groups. Two-stage procedures, such as PDMR (Plain DMR), first screen via group Lasso, then use hierarchical clustering and information criteria to merge levels and achieve full parsimony (Nowakowski et al., 2022).
Covariance Estimation and Sparse PCA: Group Lasso can estimate sparse representations of high-dimensional covariance matrices by enforcing group sparsity on columns of the coefficient matrix in a basis expansion. This leads to accurate recovery of underlying structure and facilitates sparse principal component analysis (Bigot et al., 2010).
Design of Experiments: The group Lasso framework can be used to select optimal subsets of experimental runs (groups = runs), recasting the A-optimal design as a convex group-sparse estimator, which recovers orthogonal arrays under equality constraints (Tanaka et al., 2013).
Multi-task Learning: The $y \in \mathbb{R}^n$ 4 group Lasso with $y \in \mathbb{R}^n$ 5 is especially relevant in joint regularization where parameter vectors for multiple tasks are partitioned into groups. The value of $y \in \mathbb{R}^n$ 6 regulates the coupling among tasks: for $y \in \mathbb{R}^n$ 7 near 2, moderate sharing is enforced; as $y \in \mathbb{R}^n$ 8, stronger within-group coupling is promoted (Vogt et al., 2012).

7. Limitations, Comparative Performance, and Recent Developments

Principal limitations and considerations include:

Choice of grouping: Group Lasso is not robust to misspecification of group structure; performance declines if constructed groups do not correspond to the true pattern of nonzeros (0707.3390).
Overlapping groups: Naïve penalties double-count shared coordinates; latent group Lasso and dual norm constructions resolve this, albeit with increased computational and statistical complexity (Obozinski et al., 2011, Liu et al., 2010).
Bias and shrinkage: Standard group Lasso tends to over-penalize large groups. Weighting schemes and latent models can offset this effect (Obozinski et al., 2011).
Role of p-norm coupling: For multi-task settings, moderate coupling ( $y \in \mathbb{R}^n$ 9–2) yields superior prediction accuracy when sparsity patterns are shared only partially across groups/tasks (Vogt et al., 2012).
Recent advances: Block coordinate Newton-ABS methods, robust screening rules, and distributed architectures (DC-gLasso) have markedly improved the scalability and efficiency of group Lasso solvers, with leaders such as the adelie implementation (Yang et al., 2024).