Estimation of a high-dimensional covariance matrix with the Stein loss

Abstract: The problem of estimating a normal covariance matrix is considered from a decision-theoretic point of view, where the dimension of the covariance matrix is larger than the sample size. This paper addresses not only the nonsingular case but also the singular case in terms of the covariance matrix. Based on James and Stein's minimax estimator and on an orthogonally invariant estimator, some classes of estimators are unifiedly defined for any possible ordering on the dimension, the sample size and the rank of the covariance matrix. Unified dominance results on such classes are provided under a Stein-type entropy loss. The unified dominance results are applied to improving on an empirical Bayes estimator of a high-dimensional covariance matrix.