Estimating principal components of covariance matrices using the Nyström method

Abstract: Covariance matrix estimates are an essential part of many signal processing algorithms, and are often used to determine a low-dimensional principal subspace via their spectral decomposition. However, exact eigenanalysis is computationally intractable for sufficiently high-dimensional matrices, and in the case of small sample sizes, sample eigenvalues and eigenvectors are known to be poor estimators of their population counterparts. To address these issues, we propose a covariance estimator that is computationally efficient while also performing shrinkage on the sample eigenvalues. Our approach is based on the Nystr\"{o}m method, which uses a data-dependent orthogonal projection to obtain a fast low-rank approximation of a large positive semidefinite matrix. We provide a theoretical analysis of the error properties of our estimator as well as empirical results, including examples of its application to adaptive beamforming and image denoising.