Uncentered Covariance Estimation

• Uncentered covariance estimation is the process of estimating covariance matrices without subtracting the true mean, addressing bias through mean estimation or joint modeling.
• Techniques such as projection-based URE methods, regression-based Cholesky decomposition, and elementwise regularization are used to handle high-dimensional, contaminated, or structured data.
• These methods offer near-oracle risk performance and robust error bounds, making them practical for applications in genomics, finance, and signal processing.

Uncentered covariance estimation refers to the estimation of a covariance matrix when the mean vector of the underlying data is unknown or possibly nonzero, encompassing strategies that address both statistical bias and practical challenges. In multivariate and functional data analysis, uncentered covariance estimation is vital for high-dimensional, contaminated, or structured data settings. Recent research provides both robust estimators and principled risk selection methodologies, integrating projection methods, regularization, robust loss functions, and data-driven structural modeling.

1. Statistical Framework: Centered and Uncentered Covariance Estimation

The fundamental object of covariance estimation is the matrix $$\Sigma = \mathbb{E}\left[(X-\mu)(X-\mu)^\top\right]$$ for a random vector $X\in\mathbb{R}^p$ with (generally unknown) mean $\mu$. When $\mu$ is unknown, direct sample-based estimation of $\Sigma$ without correcting for the mean introduces bias. Classical covariance estimators either subtract the estimated mean or explicitly model its uncertainty.

In projection-driven approaches (as in the URE framework (Lescornel et al., 2011)):

• For each candidate subspace $m$, the covariance estimator is given by $\hat\Sigma_m = \Pi_m S \Pi_m$, where $S$ is the empirical covariance matrix and $\Pi_m$ is the projection onto the subspace spanned by the basis functions chosen for $m$.
• For uncentered data, a preliminary mean estimator (such as the sample mean $\bar{x}$) is typically calculated, followed by projection of the mean-corrected covariance matrix onto candidate subspaces.

In regression-based Cholesky decompositions (Pourahmadi, 2012), one may

• Center data by subtracting the sample mean, then apply the decomposition to residuals.
• Jointly estimate the mean and covariance using regression equations with intercepts, thereby modeling noncentered data within the regression framework.

2. Projection-Based Methods and Unbiased Risk Estimation

The unbiased risk estimation (URE) method adapts Stein’s risk estimator to covariance model selection, enabling optimal dimension choice for projection estimators in both centered and uncentered cases (Lescornel et al., 2011). The quadratic risk for a projected estimator is: $$\mathbb{E}\left[\|\Sigma - \hat\Sigma_m\|^2\right] = \|\Sigma-\Sigma_m\|^2 +\frac{1}{n}\operatorname{tr}\left((\Pi_m\otimes \Pi_m)\Phi\right),$$ where $\Phi$ is the variance of the vectorized covariance matrix, and the first term is the bias incurred by projection.

An empirical, unbiased estimator of this risk is

$$\operatorname{Crit}(m) = \|S-\hat\Sigma_m\|^2 + \frac{2}{n}\hat\gamma_m^2,$$

with $\hat\gamma_m^2$ estimating the variance term. Minimizing $\operatorname{Crit}(m)$ over the model collection yields the chosen projection subspace, and the final estimator $\hat\Sigma_{\hat m}$ achieves near-oracle performance. This approach readily generalizes to uncentered covariance via initial mean estimation and subsequent projection of residual covariance.

3. Regularization and Structural Decomposition for High Dimensionality

Contemporary uncentered covariance methodologies address high-dimensionality and non-stationarity via regularization or parameterized decompositions (Pourahmadi, 2012, Maurya, 2014, Farnè et al., 2017, Flasseur et al., 11 Mar 2024):

• Regression-based Cholesky decomposition: Reduces covariance estimation to a sequence of regression problems, with positive-definite guarantee. For uncentered data, joint estimation of mean and covariance can be implemented.
• Elementwise regularization: Applies thresholding, banding, or tapering to the empirical covariance matrix computed from mean-corrected data, sometimes followed by projection or "eigenvalue cleaning" steps to ensure positive-definiteness.

Table 1: Comparison of Covariance Estimation Strategies in the Uncentered Case

Method	Positive-Definiteness	Handles Uncentered Mean
Projection + URE	Yes	Via mean estimation or joint modeling
Cholesky regression-based	Yes	By residualization or joint mean-covariance modeling
Elementwise regularization	Asymptotic	Mean correction or joint modeling needed

Uncentered shrinkage estimators (Flasseur et al., 11 Mar 2024) apply bias correction to the empirical covariance and regularize using both average and individual sample variances. Convex combinations with regularization matrices capture nonstationary variability, and confidence weights can enhance robustness to outliers.

4. Robust Estimation in the Presence of Contamination and Heavy Tails

Several recent works address robust uncentered covariance estimation under model misspecification, outlier contamination, or weak moment assumptions (Culan et al., 2016, Minsker et al., 2017, Minsker et al., 2018, Lounici et al., 2023):

• Partial likelihood estimation (Culan et al., 2016): Fits covariance only on the subset of data with highest likelihood under the current parameter estimate, iteratively reordering the sample to reject outliers. This enhances robustness in high-contamination scenarios (e.g., radar signal processing), albeit with potential scale bias and increased variance.
• Truncated or robustified covariance estimators (Minsker et al., 2017): Use nonlinear transformations (e.g., thresholding via a function $\psi$) of the centered second-moment matrices, yielding sub-Gaussian deviation bounds under weak assumptions (intrinsic dimension-based rather than ambient).
• Robust U-statistics (Minsker et al., 2018): Replace least-squares losses in operator-valued U-statistics by Huber-type loss functions, providing non-asymptotic operator norm error bounds under minimal fourth moment conditions, without explicit mean estimation.
• Bias-corrected covariance estimators with missing values or cellwise contamination (Lounici et al., 2023): Provide exact correction formulas for the bias induced by missingness or contamination, based only on observable data, and maintain minimax accuracy.

5. Structural Modeling, Fiducial Inference, and Model Selection

Expanding beyond classical strategies, fiducial inference (Shi et al., 2017) offers a non-Bayesian, prior-free framework for uncentered covariance estimation, maintaining strong asymptotic properties:

• The generalized fiducial distribution inverts the data-generating equations, yielding an uncertainty quantification for the covariance matrix and enabling confidence regions and structural model selection.
• Extensions for clique models partition the covariance matrix into block-diagonal forms, aiding in high-dimensional and network-based settings.
• Limiting factors include computational intensity and the complexity of exploring sparse model spaces.

Joint low-rank plus sparse decomposition methods (e.g., UNALCE (Farnè et al., 2017)) further enable algebraic and parametric consistency in high-dimensional regimes, exactly recovering the latent rank and sparsity pattern. These estimators involve composite convex optimization, followed by targeted eigenvalue re-optimization, and are demonstrated on large, real-world datasets.

6. Practical Implementation, Performance, and Limitations

Empirical studies across the cited literature report:

• Near-oracle risk performance for URE-driven projection methods across various functional bases.
• Improved accuracy and positive-definiteness for regression-based and shrinkage estimators when the mean is properly included or corrected.
• Robust estimators (partial likelihood, truncated, robust U-statistics) demonstrate marked resilience against outliers and heavy-tailed data, achieving operator-norm error rates governed by intrinsic (effective) dimension.
• Bias-corrected estimators with missingness/cellwise contamination offer computational simplicity and stability, outperforming state-of-the-art imputation schemes in both run-time and accuracy, especially in high dimensions.
• Regularized and structured estimators (joint penalty, low-rank plus sparse) succeed in model selection and functional recovery, as exemplified in genomics and financial supervision datasets.

Potential limitations include:

• Sensitivity to model ordering in regression-based decompositions.
• Increased computational burden for iterative or fiducial inference algorithms.
• Requirement for robust initial mean estimation, particularly in contaminated or high-dimensional scenarios.
• Theoretical guarantees often depend on sample size regimes and structural assumptions (sparsity, spikiness, effective rank).

7. Summary and Outlook

Uncentered covariance estimation incorporates a spectrum of methodologies: projection with unbiased risk minimization, regression-based decompositions, robust loss-based estimators, regularization with adaptive shrinkage, and model selection via fiducial inference and structural decomposition. Each method integrates mean estimation or joint modeling to address statistical bias, extending classical approaches to high-dimensional, contaminated, and structured data landscapes. Operator-norm and intrinsic dimension-based bounds, robust empirical risks, and algebraic model recovery are common technical criteria. The ongoing integration of computational tractability, reliability under contamination, and strong statistical guarantees positions uncentered covariance estimation as a cornerstone for modern multivariate analysis, especially in signal processing, genomics, finance, and functional data contexts.