---
title: 'uniLasso: Univariate-Guided Sparse Regression'
url: https://www.emergentmind.com/topics/univariate-guided-sparse-regression-unilasso
type: topic
---

# uniLasso: Univariate-Guided Sparse Regression

Univariate-Guided Sparse Regression, also termed "uniLasso", designates a class of statistical procedures for regression and feature selection that harness univariate effect sizes to stabilize and interpret multivariate sparse estimators. The methodology has been developed for linear, generalized linear, and survival models, with specific adaptations to high-dimensional biobank-scale -omics datasets. Its distinguishing principle is a two-stage approach: univariate regressions inform and constrain multivariate selection. This ensures sign-consistency between marginal and combined effects and preferentially retains variables with strong univariate predictive capacity. Across empirical and theoretical studies, uniLasso demonstrates improved stability, interpretable sign patterns, and sparser solutions relative to the standard Lasso, with comparable or better predictive accuracy in numerous regimes [2501.18360], [2511.22049].

## 1. Mathematical Formulation and Principle

Let $X\in\mathbb{R}^{n\times p}$ denote the design matrix, $y\in\mathbb{R}^n$ the response vector, and $\gamma_0, \gamma \in \mathbb{R}, \mathbb{R}^{p}$ the intercept and coefficients of the linear model $y \approx \gamma_0 + X \gamma$. The uniLasso comprises two sequential steps:

**Stage 1 (Univariate regression screening):**  
For each feature $j=1,\dots,p$, fit the univariate linear model $y_i \approx \beta_{0j} + \beta_j x_{ij}$ and compute the leave-one-out (LOO) prediction $\eta_j^{-i}$ for each observation.

**Stage 2 (Guided Nonnegative Lasso):**  
Solve  
$
(\hat\theta_0, \hat\theta) = \arg\min_{\theta_0\in\mathbb{R}, \;\theta\in\mathbb{R}^p}
	\left\{
		\frac{1}{n} \sum_{i=1}^n \left( y_i - \theta_0 - \sum_{j=1}^p \theta_j \eta_j^{-i} \right)^2
		+ \lambda \sum_{j=1}^p |\theta_j|
	\right\}
	\ \text{subject to} \ \theta_j \ge 0,\ \forall j
$
where $\lambda>0$ is the regularization parameter.

The final predictor for any $x$ is  
$
\hat\eta(x) = \hat\gamma_0 + \sum_{j=1}^p \hat\gamma_j x_j
$  
where  
$\hat\gamma_j = \hat\theta_j \beta_j$,  
$\hat\gamma_0 = \hat\theta_0 + \sum_j \hat\theta_j \beta_{0j}$.

Nonstandardized features preserve the scale of univariate effects. The nonnegativity constraint ensures $\text{sign}(\hat\gamma_j) = \text{sign}(\beta_j)$ or $\hat\gamma_j = 0$.

## 2. Theoretical Guarantees

Under the model $Y = \gamma_0 + \sum_{j\in S} \gamma_j X_j + \epsilon$, with $\epsilon$ mean-zero, independent of features, and feature support $S$, the population univariate slope is $\beta_j = \text{Cov}(Y,X_j) / \text{Var}(X_j)$. If each active variable satisfies $\gamma_j \cdot \beta_j > 0$ (sign-alignment), covariance of active features is nondegenerate, and variables have sub-Gaussian tails, then for appropriately chosen $\lambda$:
- All inactive coefficients $\hat{\gamma}_j$ are exactly zero.
- Each active coefficient satisfies $|\hat{\gamma}_j - \gamma_j| = O(\lambda)$.
This holds with high probability ($\ge 1 - O(pn \exp(-c n\lambda^2))$).

Contrast with standard Lasso, which requires incoherence (“irrepresentable”) conditions: strong collinearity between relevant and irrelevant features in the Gram matrix can result in spurious selections. UniLasso instead relies on small marginal correlations for inactive variables and sign-alignment for relevant effects, a weaker and more interpretable set of sufficient conditions.

Sign-alignment is preserved under the condition that for all $j, k \in S$,  
$\delta_{j,k} = \text{Cov}(X_j,X_k)/\text{Var}(X_k)$ satisfies $\delta_{j,k} \geq 0$ if $\gamma_j\gamma_k >0$ and $\leq 0$ if $\gamma_j\gamma_k <0$; thus, for each $j$, $\beta_j = \gamma_j + \sum_{k\neq j} \gamma_k \delta_{k,j}$ preserves the sign of $\gamma_j$ and $|\beta_j| \geq |\gamma_j|$ [2501.18360].

## 3. Algorithmic Procedure and Computational Aspects

**Pseudocode (linear regression):**
```python
# Input: X (n×p), y (n), λ > 0

# Stage 1: For j in 1..p
for j in range(p):
    # Fit univariate regression y ~ β_{0j} + β_j x_{ij}
    # Compute leave-one-out predictions η_j^{-i} for i in 1..n

# Stage 2:
# Z_{i,j} = η_j^{-i}
# Solve nonnegative Lasso:
#   min_{θ_0, θ ≥ 0} (1/n)‖y - θ_0 - Zθ‖² + λ‖θ‖₁

# Output: γ_j = θ_j · β_j, γ_0 = θ_0 + ∑ θ_j β_{0j}
```

Stage 1 is distributable and achieves $O(np)$ complexity (using hat matrix diagonal formulae). Stage 2 is handled by standard coordinate-descent Lasso solvers with nonnegativity constraints and no feature standardization (e.g., `glmnet` or Adelie). Parallelization and memory efficiency are achieved via filtering (e.g., minor allele frequency in genomic data) and denominator stabilization.

For biobank-scale omics datasets ($p>10^6$), adaptations include:
- Minor-allele-frequency filtering (MAF $< 0.0005$ excluded).
- Stabilization of denominators for features with very low variance.
- Logistic regression adaptation for binary traits using IRLS and leave-one-out formulae [2511.22049].
- Parallelization over features for fast computation.

## 4. Extensions and Generalizations

The two-stage uniLasso procedure generalizes to other response models:
- **Generalized Linear Models (GLMs):** Univariate fits via iteratively reweighted least squares; LOO approximations via influence-function formulas. Stage 2 applies the penalty and constraints on the new set of univariate predictors.
- **Cox regression (survival analysis):** Univariate fit by partial-likelihood score; second stage penalized Cox-Lasso (with nonnegativity if desired).

For scenarios with available external summary statistics (e.g., GWAS effect sizes), uniLasso enables a single-stage adaptively weighted Lasso formulation with enforced sign agreement. Given univariate slopes $\tilde{\beta}_j$ from external data, fit
$
(\hat\gamma_0, \hat\gamma) = \arg\min_{ \gamma_0, \gamma }
	\sum_{i=1}^n \left( y_i - \gamma_0 - \sum_{j=1}^p x_{ij} \gamma_j \right)^2
	+ \lambda \sum_{j=1}^p \frac{ |\gamma_j| }{ |\tilde{\beta}_j| }
$
subject to $\mathrm{sign}(\gamma_j) = \mathrm{sign}( \tilde{\beta}_j )$ for all $j$ [2511.22049].

## 5. Model Selection, Interpretability, and Empirical Performance

Hyperparameters are selected via cross-validation, typically performed in Stage 2 on the composite model, minimizing prediction error. In biobank contexts, fixed train/validation/test splits are preferred over $k$-fold CV for efficiency.

Interpretability gains are substantial: uniLasso’s sign-consistency property ($\text{sign}(\gamma_j) = \text{sign}(\beta_j)$ or $\gamma_j=0$) eliminates sign-flips found in standard Lasso solutions. Variables with large marginal (univariate) effects incur less penalization and thus are preferentially selected. Empirically, uniLasso exhibits substantially sparser solutions than standard Lasso or PRS-CS, with matched or superior prediction. Table 1 from [2511.22049] quantifies sparsity:

| Phenotype | Lasso  | uniLasso | uniLasso ES |
|-----------|--------|----------|-------------|
| Height    | 55,038 | 34,256   | 44,788      |
| BMI       | 33,076 | 18,833   | 24,598      |
| CHD       | 2,615  | 1,009    | 1,310       |
| Asthma    | 5,556  | 3,030    | 5,698       |

Predictive accuracy (R²/AUC) is comparable across all approaches, with uniLasso matching Lasso while selecting 40% fewer features.

| Phenotype | Lasso | uniLasso | uniLasso ES | PRS-CS |
|-----------|-------|----------|-------------|--------|
| Height    | 0.713 | 0.707    | 0.716       | 0.649  |
| BMI       | 0.119 | 0.103    | 0.118       | 0.070  |
| Asthma    | 0.619 | 0.620    | 0.623       | 0.596  |
| CHD       | 0.758 | 0.757    | 0.760       | 0.760  |

Compute times for biobank-scale analyses are manageable with parallelization; uniLasso’s additional screening step incurs modest extra computation relative to Lasso, offset by faster training in its external-score variant.

## 6. Practical Recommendations and Limitations

- Nonnegativity constraints and avoidance of feature standardization in the guided Lasso stage are essential for maintaining the link between marginal and joint effects.
- MAF filtering and denominator stabilization are critical for numerical stability in very high $p$.
- When signs of marginal effects are volatile among correlated variables, uniLasso may underperform Lasso; it is advised to compare out-of-sample errors for both methods.
- The method is especially suitable in settings where interpretability and sign stability are prioritized, such as genomics and epidemiological risk modeling.
- In applications with available external summary statistics, the uniLasso ES variant is recommended due to training efficiency and preserved sparsity.

A plausible implication is that uniLasso, by enforcing sign-concordance and capitalizing on marginal effect sizes, offers a principled bridge between marginal screening and joint penalized regression, resolving some instabilities of classical Lasso in highly correlated and high-dimensional data [2501.18360], [2511.22049].

Source: https://www.emergentmind.com/topics/univariate-guided-sparse-regression-unilasso