Rank-Based Sparse Regression in Principal Components Space under Measurement Error

Abstract: We study high-dimensional regression in principal components space when the predictors are observed with additive measurement error and the response errors may be heavy-tailed. The starting point is the $\ell_1$-penalized principal-components estimator of Song and Zou (2026), which enjoys a blessing-of-dimensionality phenomenon under predictor contamination but senstive for heavy-tailed data or outliers. We replace the squared loss by a Wilcoxon-type rank loss and then apply a one-step adaptive reweighting scheme to reduce the shrinkage bias of the initial $\ell_1$ fit. The resulting procedure combines robustness to heavy-tailed response errors with the contamination geometry induced by the empirical principal-components basis. Our main theorem gives a prediction bound for the fixed-$λ$ second-stage fitted mean. Simulations show that the rank-based procedure is competitive under Gaussian noise and substantially more stable under heavy-tailed errors, especially when predictor contamination is present.

• The paper introduces a two-stage, rank-based approach that robustly handles measurement error and heavy-tailed noise in high-dimensional regression.
• It employs a Wilcoxon-type rank loss with adaptive reweighting to reduce shrinkage bias inherent in conventional ℓ1-penalized estimators.
• Simulation and real data analyses demonstrate that the method outperforms traditional techniques, leveraging a blessing-of-dimensionality effect.

Rank-Based Sparse Regression in Principal Components Space under Measurement Error

Introduction and Problem Setting

This paper addresses high-dimensional regression when predictors are linearly transformed by the empirical principal components and are contaminated by additive measurement error, while response errors may exhibit heavy tails. The core contribution is a robust, sparse regression methodology built upon the empirical principal components (PC) basis. Existing $\ell_1$-penalized least-squares estimators in the PC space [Song & Zou, 2026] yield improved prediction robustness to contamination due to an eigengap-dependent term, showcasing a "blessing-of-dimensionality" effect. However, these methods are not stable under heavy-tailed response noise or outliers—an acute limitation when measurement error is non-negligible. The present work substitutes the squared loss with a Wilcoxon-type rank loss and introduces a second-stage adaptive reweighting mechanism to ameliorate the shrinkage bias inherent in standard $\ell_1$ regularization.

Methodology: Two-Stage Robust Estimation in PC Space

The approach proceeds in two stages:

1. Rank-Lasso Pilot: The initial stage minimizes a convex, pairwise Wilcoxon-type rank loss penalized by an $\ell_1$ term. The design matrix used is the left singular vectors of the empirical PC decomposition of the contaminated predictors. The objective is

$$\mathcal Q_n(\bm\theta) = \frac{1}{n(n-1)}\sum_{i \neq j} |(y_i - y_j) - (\widetilde{\bm{u}}_i - \widetilde{\bm{u}}_j)^\top \bm\theta| + \lambda_0 \|\bm\theta\|_1$$

This estimator is solved as a linear program via slack variables.

2. Adaptive Reweighting (Bias Reduction): Following recent advances in robust high-dimensional variable selection [Wang et al., 2020], the method introduces a locally weighted $\ell_1$ penalty using weights derived from derivatives of nonconvex penalties (SCAD or MCP) evaluated at the pilot estimator’s coefficients. The weighted rank loss problem

$$\mathcal Q_n(\bm\theta) + \sum_{j=1}^m w_j |\theta_j|$$

is then solved, with the penalty weights $w_j$ chosen to zero out for sufficiently large active signals, thereby reducing selection bias for true nonzero coefficients.

For model selection, a high-dimensional BIC (HBIC) is minimized using a refitted loss on the variable support for selection of the final tuning parameter.

Theoretical Results

The main theoretical advance is an oracle inequality for the second-stage estimator. Explicit probability bounds are provided for the prediction error, decomposing into an estimation term and a contamination term. The central results are as follows:

• First-Stage Prediction: The Wilcoxon rank-lasso yields prediction risk

$$O\left( \frac{|A| \log m}{n} + R_q\, \tau_\star^{2-q} + \frac{\|\bm{y}^*\|_2^2}{n\kappa^2}\left(\frac{n}{p}+\frac{n^2}{p^2}+\frac{n\|\mathbf{X}\|_2^2}{p^2}\right) \right)$$

where $A$ denotes the set of active PCs, $\kappa$ is the relevant eigengap, and $\ell_1$0.

• Second-Stage Oracle Improvement: The adaptive procedure delivers estimation at the oracle rate ($\ell_1$1) in the PC space, even under predictor contamination and heavy-tailed response noise, with the contamination-induced term matching that of the least-squares PC regression estimator.
• Blessing-of-Dimensionality: The contamination penalty decreases as $\ell_1$2 grows (for fixed $\ell_1$3 and active signal), maintaining robust prediction accuracy at high dimensions even with moderate or severe measurement error.

Simulation Results

Comprehensive simulation studies investigate performance across regimes:

• Signal Structure: Both sparse and weakly sparse cases, with signal concentrated on low-index or high-index PCs.
• Error Distributions: Gaussian, Student-$\ell_1$4 (standardized), and a heavy-tailed mixture normal (90% $\ell_1$5, 10% $\ell_1$6), empirically standardized for variance.
• Contamination: Settings include no error, independent error, and correlated error in predictors.

Key findings:

• With heavy-tailed response errors and non-negligible measurement error, the proposed method (RPCR) yields uniformly smaller prediction errors than the classical $\ell_1$7 (L1PCR) approach.
• As $\ell_1$8 increases, the prediction error declines for both methods, validating the blessing-of-dimensionality in the presence of contamination.
• Under Gaussian errors, least-squares can sometimes have a slight efficiency edge in contaminated settings; under heavy-tailed errors, the rank-based estimator exhibits marked superiority, with the gap increasing as the dimensionality increases or as response tails become heavier.

Real Data Illustration

Analysis of the TRIM32 phenotype from a genetic eQTL dataset with synthetic measurement error corroborates the main simulation findings. RPCR outperforms both L1PCR and standard LASSO once moderate measurement error is introduced ($\ell_1$9), achieving the lowest cross-validated prediction error, whereas LASSO has marginal advantage in the unperturbed setting. QQ plots of leave-one-out residuals demonstrate significant non-Gaussian behavior in all methods, reinforcing the need for robust alternatives.

Figure 1: QQ plots of leave-one-out residuals for L1PCR, LASSO, and RPCR at the clean design ($\ell_1$0); all show clear non-Gaussian tails, with Shapiro–Wilk $\ell_1$1-values $\ell_1$2 (L1PCR), $\ell_1$3 (LASSO), and $\ell_1$4 (RPCR).

This empirical result demonstrates both the lack of normality in practical residuals and the advantages of robust procedures under model misspecification, especially in high-dimensional genomics.

Implications and Future Directions

The proposed two-stage rank-based procedure provides a statistically principled, computationally tractable way to achieve robust, sparse, high-dimensional prediction in the presence of measurement error and heavy-tailed noise, under sparsity constraints in PC space. The theoretical results characterize the robustness and the "contamination geometry" via eigengaps, underscoring the interaction between latent signal structure and noise propagation in high dimensions.

Potential avenues for further research include:

• Extension to high-dimensional generalized linear models, where nonlinear link functions and rank-based loss introduce new theoretical complexity.
• Alternative PC-guided or adaptive basis constructions, potentially leveraging data-driven structure for even greater robustness and adaptivity under signal heterogeneity.
• Full characterization of the sampling properties of the proposed estimator when the empirical PC basis is used in conjunction with other forms of contaminated or missing data.

Conclusion

By integrating rank-based loss with adaptive reweighted regularization in the empirical principal-component basis, this paper delivers a robust, high-dimensional regression technique that retains the blessing-of-dimensionality effect under substantial contamination and is demonstrably superior under heavy-tailed errors. Theoretical results, simulations, and real data all support the methodological and practical value of the approach for modern high-dimensional settings where Gaussianity and error-free designs are untenable.

---

Reference: "Rank-Based Sparse Regression in Principal Components Space under Measurement Error" (2604.04807)