Concentration Inequalities and Moment Bounds for Sample Covariance Operators (1405.2468v3)

Abstract: Let $X,X_1,\dots, X_n,\dots$ be i.i.d. centered Gaussian random variables in a separable Banach space $E$ with covariance operator $\Sigma:$ $$ \Sigma:E^{\ast}\mapsto E,\ \ \Sigma u = {\mathbb E}\langle X,u\rangle, u\in E^{\ast}. $$ The sample covariance operator $\hat \Sigma:E^{\ast}\mapsto E$ is defined as $$ \hat \Sigma u := n^{-1}\sum_{j=1}^n \langle X_j,u\rangle X_j, u\in E^{\ast}. $$ The goal of the paper is to obtain concentration inequalities and expectation bounds for the operator norm $|\hat \Sigma-\Sigma|$ of the deviation of the sample covariance operator from the true covariance operator. In particular, it is shown that $$ {\mathbb E}|\hat \Sigma-\Sigma|\asymp |\Sigma|\biggl(\sqrt{\frac{{\bf r}(\Sigma)}{n}}\bigvee \frac{{\bf r}(\Sigma)}{n}\biggr), $$ where $$ {\bf r}(\Sigma):=\frac{\Bigl({\mathbb E}|X|\Bigr)^2}{|\Sigma|}. $$ Moreover, under the assumption that ${\bf r}(\Sigma)\lesssim n,$ it is proved that, for all $t\geq 1,$ with probability at least $1-e^{-t}$ \begin{align*} \Bigl||\hat\Sigma - \Sigma|-{\mathbb E}|\hat\Sigma - \Sigma|\Bigr| \lesssim |\Sigma|\biggl(\sqrt{\frac{t}{n}}\bigvee \frac{t}{n}\biggr). \end{align*}