Principal Components LASSO (PC-LASSO)

• PC-LASSO is a regression framework that integrates PCA and LASSO to exploit the structure of predictor correlations while enforcing sparsity.
• It incorporates methods such as projecting predictors onto leading principal components and applying a PC-guided quadratic penalty in the original space.
• The method spans unsupervised and supervised variants with tailored tuning strategies, balancing bias and variance for improved prediction and stability.

Principal Components LASSO (PC-LASSO) denotes a family of regression procedures that combine principal-component representations with $\ell_1$ regularization. In the classical usage, the term refers either to projecting the original predictors onto their leading principal components and fitting a LASSO, or to imposing penalties aligned with the principal-component directions of the raw predictors (Wang et al., 11 Feb 2026). In a narrower and now standard sense, “pcLasso” denotes the method that combines the lasso penalty with a quadratic penalty that shrinks the coefficient vector toward the leading principal components of the feature matrix (Tay et al., 2018). Across these formulations, the common objective is to exploit correlation structure in the predictors while retaining sparsity, but the family now spans unsupervised principal-component regression with $\ell_1$ selection, supervised one-stage alternatives, robust PC-space regression, and principal-component reductions in nonparametric HAL feature spaces (Kawano et al., 2014).

1. Conceptual scope and historical formulations

PC-LASSO sits at the intersection of principal component regression (PCR) and sparse regression. Classical PCR is a two-stage procedure that selects some principal components and then constructs a regression model regarding them as new explanatory variables; the principal components are obtained from only explanatory variables and not considered with the response variable (Kawano et al., 2014). The canonical PC-LASSO formulation augments this pipeline by imposing $\ell_1$ regularization on the regression coefficients in PC space: $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$ with $V_K$ obtained from PCA of $X$ (Kawano et al., 2014).

A distinct line of work uses “pcLasso” for a different but related construction: the lasso objective is supplemented by a quadratic penalty that is zero on leading PC directions and increases away from them. In the single-group setting, the method solves

$$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$

where $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$ for the SVD $X=UDV^\top$ (Tay et al., 2018). This formulation does not first regress on fixed PC scores; instead, it keeps coefficients in the original feature space and uses an anisotropic quadratic term to bias them toward leading PCs.

Later work broadened the term further. Sparse Principal Component Regression (SPCR) was introduced as a one-stage supervised alternative that jointly learns sparse principal component loadings and component regression coefficients, rather than fixing loadings by unsupervised PCA (Kawano et al., 2014). Principal component-guided sparse reduced-rank regression integrates the pcLasso quadratic guidance term into reduced-rank regression so that the coefficient matrix is biased toward principal component directions with large variance, globally or within predefined groups (Goto et al., 12 Jan 2026). In highly adaptive regression, Principal Component based Highly Adaptive Lasso (PCHAL) was described as a PC-LASSO in the HAL feature space, because the principal components are computed on the HAL design rather than on the raw covariates (Wang et al., 11 Feb 2026). This suggests that PC-LASSO is best understood as a methodological template rather than a single estimator.

2. Canonical objectives and penalty geometry

The simplest PC-LASSO acts directly on PC scores. If $X=UDV^\top$ and $\ell_1$0, then linear LASSO PCR takes the form

$\ell_1$1

and the fitted coefficients can be mapped back to the original variables through $\ell_1$2 (Rieck et al., 9 Sep 2025). In logistic settings, the corresponding objective is

$\ell_1$3

This construction yields sparsity among principal components rather than among raw variables (Rieck et al., 9 Sep 2025).

The 2018 pcLasso formulation instead operates in the original coefficient vector $\ell_1$4 but uses a PC-guided quadratic form. Its penalty decomposes as

$\ell_1$5

so the leading PC incurs no quadratic penalty and directions orthogonal to the leading PC are penalized increasingly with larger spectral gaps $\ell_1$6 (Tay et al., 2018). In grouped form, if predictors are partitioned into non-overlapping groups and each group $\ell_1$7 has SVD $\ell_1$8, then

$\ell_1$9

and the grouped objective becomes

$\ell_1$0

The grouped version encourages coefficients in each group to align with that group’s leading PCs and, together with the $\ell_1$1 term, can induce both feature sparsity and a form of group selection when internal group structure is strong (Tay et al., 2018).

This PC-guided geometry differs from isotropic ridge regularization. With $\ell_1$2, ridge yields fitted-value shrinkage factors $\ell_1$3, whereas the pcLasso quadratic term yields

$\ell_1$4

which shrinks more aggressively toward top PCs than ridge when degrees of freedom are small (Tay et al., 2018). A plausible implication is that the method interpolates between sparse regression and a soft version of low-rank PCR, without enforcing a hard truncation in PC space.

3. Estimation algorithms and tuning procedures

For fixed principal components, optimization in PC space is straightforward because the predictors are orthogonal. In the canonical PC-LASSO and LASSO-PCR settings, this orthogonality means that the $\ell_1$5 problem is especially convenient numerically. A more extreme version appears in PCHAL: because the retained PC scores are orthogonal, the LASSO solution decouples into coordinatewise soft-thresholding, eliminating iterative optimization (Wang et al., 11 Feb 2026). With squared loss, if $\ell_1$6 and $\ell_1$7, then

$\ell_1$8

with $\ell_1$9 when $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$0 (Wang et al., 11 Feb 2026).

For the original-space pcLasso of 2018, the objective is convex with a separable non-smooth $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$1 term, and the method is optimized by coordinate descent (Tay et al., 2018). Writing

$$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$2

and using the soft-thresholding operator $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$3, the coordinate update is

$$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$4

with warm starts along a decreasing $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$5 path and strong rules for screening (Tay et al., 2018). The same paper parameterizes $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$6 through a “rat” quantity controlling the shrinkage ratio between the second and first PCs, and cross-validation chooses both “rat” and $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$7 (Tay et al., 2018).

SPCR and adaptive SPCR are optimized differently because the loadings themselves are learned. The objective is convex in each parameter block, and a blockwise coordinate descent algorithm yields a monotone decrease of the objective (Kawano et al., 2014). The method alternates updates of $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$8, $$\min_{\beta_0,\ \alpha\in\mathbb{R}^K}\ \sum_{i=1}^n \bigl(y_i - \beta_0 - (V_K^T x_i)^T \alpha \bigr)^2 + \lambda \|\alpha\|_1,$$9, $V_K$0, and the orthonormal auxiliary matrix $V_K$1, with the latter obtained from the SVD of $V_K$2 (Kawano et al., 2014). By setting $V_K$3 as an upper bound and imposing an $V_K$4 penalty on $V_K$5, SPCR can “simultaneously” perform dimension reduction and regression because components with $V_K$6 are effectively dropped (Kawano et al., 2014).

Tuning strategies differ across variants. Canonical PC-LASSO and ranked-sparsity LASSO PCR use cross-validation over $V_K$7 and, where relevant, over component-weight schedules such as $V_K$8 (Rieck et al., 9 Sep 2025). PCHAL/PCHAR use V-fold cross-validation over $V_K$9, sometimes aided by scree plots or cumulative explained variance heuristics (Wang et al., 11 Feb 2026). Robust rank-based PC-LASSO calibrates the first-stage penalty $X$0 by permutation and the second-stage penalty by an HBIC criterion (Feng et al., 6 Apr 2026). This indicates that “PC-LASSO” does not imply a single tuning regime; the optimization and model-selection strategy depend heavily on whether PCs are fixed, supervised, weighted, or constructed in an expanded feature space.

4. Statistical properties and interpretations

Theoretical analysis of PC-LASSO depends strongly on the specific formulation. For pcLasso as principal component-guided sparse regression, the objective admits an augmented-design equivalence: $X$1 with

$X$2

This leads to improved eigenvalue bounds for the augmented design and associated estimation and prediction error bounds; in particular, if the signal aligns with the leading PC so that $X$3, the error term simplifies and pcLasso achieves a strictly better rate than lasso (Tay et al., 2018).

The canonical criticism of unsupervised PC-LASSO is also explicit in the literature. PCR does not use $X$4 to construct PCs and can underperform if the signal lies in low-variance directions of $X$5 (Kawano et al., 2014). This concern motivates SPCR, which couples prediction and reconstruction in a one-stage objective, and motivates ranked-sparsity LASSO PCR, which imposes component-specific weights $X$6 reflecting a prior that relevant signal may be concentrated in early PCs (Rieck et al., 9 Sep 2025). A plausible implication is that the choice between unsupervised and supervised PC-LASSO is primarily a bias-variance choice about whether variance-dominant directions are also prediction-dominant directions.

Robust rank-based PC-LASSO under measurement error adds a different theoretical layer. With contaminated predictors $X$7, the empirical principal components of $X$8 induce a contamination geometry controlled by an eigengap parameter $X$9, and the contamination term in the prediction error decreases as $$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$0 grows under appropriate conditions, manifesting a blessing-of-dimensionality phenomenon (Feng et al., 6 Apr 2026). Replacing squared loss by a Wilcoxon-type rank loss yields robustness to heavy-tailed response errors, and the main theorem shows that the second-stage fitted mean satisfies

$$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$1

thereby preserving the contamination term from least-squares PC-LASSO while improving robustness under heavy tails (Feng et al., 6 Apr 2026).

In HAL-based PC-LASSO variants, the salient theoretical point is not merely prediction but preservation of HAL-style complexity control. HAL attains essentially dimension-free rates under bounded Hardy–Krause variation, and PC-based HAL reductions preserve this geometry when leading PCs capture most of the HAL/HAR Gram structure (Wang et al., 11 Feb 2026). Closely related PC-HA theory shows that principal-component reductions of the HAL design can retain HAL’s dimension-free minimax rates up to log-factors, and can transfer plug-in efficiency and pointwise asymptotic normality under comparable complexity control (Meixide et al., 18 Mar 2026). This suggests that in nonparametric settings PC-LASSO can be interpreted not only as a sparsity device but also as a principled sufficient dimension reduction.

5. Major variants and extensions

Several recent methods can be understood as structurally distinct PC-LASSO variants.

Sparse Principal Component Regression (SPCR) and adaptive SPCR replace the two-stage unsupervised PCR-plus-lasso pipeline with a joint objective: $$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$2 Its defining feature is that the loadings $$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$3 are learned in a supervised fashion rather than fixed by PCA (Kawano et al., 2014).

Principal component-guided sparse reduced-rank regression extends the pcLasso quadratic guidance term to multivariate-response regression. With $$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$4, $$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$5, and optional predictor groups, the estimator minimizes

$$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$6

thereby accounting simultaneously for predictor correlation, group structure, and response correlation through latent factors (Goto et al., 12 Jan 2026).

Ranked-sparsity LASSO PCR and Joint Sparsity-Ranked LASSO (JSRL) modify the PC-space lasso by weighting the $$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$7 penalty according to PC index. The weighted PC-only objective is

$$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$8

with the index-based schedule

$$\min_{\beta\in\mathbb{R}^p}\;\frac{1}{2}\,\|y - X\beta\|_2^2\;+\;\lambda\,\|\beta\|_1\;+\;\frac{\theta}{2}\,\beta^\top A\,\beta,$$9

JSRL further adds voxel-level coefficients and calibrates the relative penalties by an information-parity framework (Rieck et al., 9 Sep 2025). This is still a PC-LASSO-type construction, but it differs from quadratic pcLasso because it reweights the $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$0 term directly on PC axes rather than adding a PC-guided quadratic term (Rieck et al., 9 Sep 2025).

PCHAL and PCHAR transpose the PC-LASSO idea into the HAL basis. Here the principal components are computed from the HAL/HAR Gram operator induced solely by covariates $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$1, and the resulting orthogonal score matrix $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$2 is used for $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$3- or $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$4-regularized fitting (Wang et al., 11 Feb 2026). PC-HA generalizes this principle further and distinguishes PC-HAL, PC-HAR, and PC-HAGL according to whether regularization is placed on PC coefficients directly or on the induced spline coefficients (Meixide et al., 18 Mar 2026). A plausible implication is that once the feature dictionary is expanded beyond raw covariates, “PC-LASSO” becomes a family of geometry-driven regularizers rather than a simple PCR modification.

6. Empirical behavior, applications, and limitations

Empirical evidence for PC-LASSO is heterogeneous because the family contains methods aimed at different signal structures. The original pcLasso paper reports that the method can be especially powerful if the features are pre-assigned to groups and that, in simulations, pcLasso is best when the signal lies on top PCs, is approximately tied with lasso and elastic net in “neutral court,” and tends to revert toward lasso-like behavior when signal lies on bottom PCs because cross-validation often chooses $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$5 (Tay et al., 2018). This establishes a central practical point: the method is most useful when predictor covariance is informative about the signal.

Macroeconomic forecasting experiments used a LASSO-PC modification that first preselected monthly indicators by sparse regression, rotated the selected predictors by PCA, and then applied LASSO in factor space. In pseudo-real-time experiments for gross fixed capital formation, private consumption, imports, and exports over 2005–2019, the proposed LASSO-PC modification showed further improvement in forecast accuracy relative to the sparse baselines and benchmarks (Jokubaitis et al., 2019). The paper also reported that AggregatedPC outperformed DirectPC in post-LASSO OLS experiments using the first five principal components (Jokubaitis et al., 2019). This suggests that orthogonalization can be useful not only for estimation but also for forecast aggregation in mixed-frequency settings.

In fMRI decoding, ranked-sparsity LASSO PCR and JSRL were applied to risk taking, monetary incentive, and emotion regulation tasks. Incorporating sparsity ranking into LASSO PCR produced models with enhanced classification performance, with JSRL achieving up to $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$6 improvement in cross-validated deviance $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$7 and $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$8 improvement in cross-validated AUC, and both ranked-sparsity methods performing as well as or better than standard LASSO PCR across all classification tasks (Rieck et al., 9 Sep 2025). The same study emphasized that hard-threshold PCR underperformed the ranked-sparsity variants (Rieck et al., 9 Sep 2025).

In highly adaptive regression, synthetic benchmarks showed that PCHAR tracks HAR closely across settings and that, averaged over $A := V \operatorname{diag}(d_1^2-d_j^2)V^\top$9 scenarios, PCHAR’s test MSE is approximately $X=UDV^\top$0–$X=UDV^\top$1 higher than HAR while often indistinguishable at $X=UDV^\top$2-decimal precision; PCHAL was more variable but competitive (Wang et al., 11 Feb 2026). Real-data regression on UCI-style datasets showed that PCHAR/PCHAL typically stayed close to HAR and often competitive with baselines, while avoiding severe degradations (Wang et al., 11 Feb 2026). In robust high-dimensional regression under measurement error, the rank-based PC-LASSO was competitive under Gaussian noise and substantially more stable under heavy-tailed errors, with the Scheetz dataset example showing smaller average prediction error than L1PCR and LASSO at moderate-to-high contamination levels (Feng et al., 6 Apr 2026).

Several limitations recur across the literature. Outcome-blind truncation can discard predictive components if $X=UDV^\top$3 aligns with low-variance modes, and cross-validation mitigates this but cannot guarantee optimality in finite samples (Wang et al., 11 Feb 2026). For grouped or quadratic pcLasso, mis-specified groups or weak internal correlation tend to push the method back toward lasso-like behavior (Tay et al., 2018). In robust PC-LASSO, mapping coefficients back to the original feature space can be unstable when empirical singular values are small (Feng et al., 6 Apr 2026). In HAL-based variants, storing or approximating the Gram matrix can still be heavy for extremely large $X=UDV^\top$4, motivating Nyström approximations or randomized sketching (Wang et al., 11 Feb 2026). Taken together, these results indicate that PC-LASSO is not uniformly superior to lasso or PCR; its comparative advantage is conditional on whether principal-component structure is an effective inductive bias for the task at hand.