Papers
Topics
Authors
Recent
Search
2000 character limit reached

Dimension-free Structured Covariance Estimation

Published 15 Feb 2024 in math.ST, eess.SP, and stat.TH | (2402.10032v2)

Abstract: Given a sample of i.i.d. high-dimensional centered random vectors, we consider a problem of estimation of their covariance matrix $\Sigma$ with an additional assumption that $\Sigma$ can be represented as a sum of a few Kronecker products of smaller matrices. Under mild conditions, we derive the first non-asymptotic dimension-free high-probability bound on the Frobenius distance between $\Sigma$ and a widely used penalized permuted least squares estimate. Because of the hidden structure, the established rate of convergence is faster than in the standard covariance estimation problem.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. R. Adamczak. A note on the Hanson-Wright inequality for random vectors with dependencies. Electronic Communications in Probability, 20:1–13, 2015.
  2. G. I. Allen and R. Tibshirani. Transposable regularized covariance models with an application to missing data imputation. The Annals of Applied Statistics, 4(2):764, 2010.
  3. J. Bai and S. Shi. Estimating high dimensional covariance matrices and its applications. Annals of Economics and Finance, 12(2):199–215, 2011.
  4. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. The Journal of Machine Learning Research, 9:485–516, 2008.
  5. P. J. Bickel and E. Levina. Covariance regularization by thresholding. The Annals of Statistics, 36(6):2577–2604, 2008a.
  6. P. J. Bickel and E. Levina. Regularized estimation of large covariance matrices. The Annals of Statistics, 36(1):199–227, 2008b.
  7. The spatiotemporal meg covariance matrix modeled as a sum of kronecker products. NeuroImage, 27(2):402–415, 2005.
  8. F. Bunea and L. Xiao. On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA. Bernoulli, 21(2):1200–1230, 2015.
  9. Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation. Electronic Journal of Statistics, 10(1):1–59, 2016.
  10. O. Catoni and I. Giulini. Dimension-free PAC-Bayesian bounds for matrices, vectors, and linear least squares regression. Preprint, arXiv:1712.02747, 2017.
  11. Estimating stationary dipoles from MEG/EEG data contaminated with spatially and temporally correlated background noise. IEEE Transactions on Signal Processing, 50(7):1565–1572, 2002.
  12. A maximum-likelihood estimator for trial-to-trial variations in noisy MEG/EEG data sets. IEEE Transactions on Biomedical Engineering, 51(12):2123–2128, 2004.
  13. Modeling the spatial and temporal dependence in fMRI data. Biometrics, 66(3):949–957, 2010.
  14. High dimensional covariance matrix estimation using a factor model. Journal of Econometrics, 147(1):186–197, 2008.
  15. D. R. Fuhrmann. Application of Toeplitz covariance estimation to adaptive beamforming and detection. IEEE Transactions on Signal Processing, 39(10):2194–2198, 1991.
  16. Matrix computations. JHU press, 2013.
  17. K. Greenewald and A. O. Hero. Robust Kronecker product PCA for spatio-temporal covariance estimation. IEEE Transactions on Signal Processing, 63(23):6368–6378, 2015.
  18. Kronecker sum decompositions of space-time data. In 2013 5th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), pages 65–68. IEEE, 2013.
  19. A test for kronecker product structure covariance matrix. Journal of Econometrics, 233(1):88–112, 2023.
  20. Estimation of a multiplicative correlation structure in the large dimensional case. Journal of Econometrics, 217(2):431–470, 2020.
  21. C. Leng and G. Pan. Covariance estimation via sparse Kronecker structures. Bernoulli, 24(4B):3833–3863, 2018.
  22. Tucker tensor analysis of matérn functions in spatial statistics. Computational Methods in Applied Mathematics, 19(1):101–122, 2019.
  23. Separable expansions for covariance estimation via the partial inner product. Biometrika, 110:225–247, 2022.
  24. A. McCormack and P. Hoff. Information geometry and asymptotics for Kronecker covariances. Preprint, arXiv:2308.02260, 2023.
  25. CANDECOMP/PARAFAC model order selection based on reconstruction error in the presence of Kronecker structured colored noise. Digital Signal Processing, 48:12–26, 2016.
  26. Sharper dimension-free bounds on the Frobenius distance between sample covariance and its expectation. Preprint, arXiv:2308.14739, 2023.
  27. High-dimensional covariance estimation by minimizing l1subscript𝑙1l_{1}italic_l start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-penalized log-determinant divergence. Electronic Journal of Statistics, 5:935–980, 2011.
  28. Skp-shrinkage estimator for sar multi-baselines applications. In 2010 IEEE Radar Conference, pages 701–706. IEEE, 2010.
  29. V. Spokoiny. Dimension free nonasymptotic bounds on the accuracy of high-dimensional laplace approximation. SIAM/ASA Journal on Uncertainty Quantification, 11(3):1044–1068, 2023.
  30. Knowledge-aided covariance matrix estimation via Kronecker product expansions for airborne STAP. IEEE Geoscience and Remote Sensing Letters, 15(4):527–531, 2018.
  31. T. Tsiligkaridis and A. O. Hero. Covariance estimation in high dimensions via Kronecker product expansions. IEEE Transactions on Signal Processing, 61(21):5347–5360, 2013.
  32. C. F. Van Loan and N. Pitsianis. Approximation with Kronecker Products, pages 293–314. Springer Netherlands, Dordrecht, 1993.
  33. R. Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2018.
  34. Kronecker-structured covariance models for multiway data. Statistics Surveys, 16:238–270, 2022.
  35. K. Werner and M. Jansson. Estimation of Kronecker structured channel covariances using training data. In 2007 15th European Signal Processing Conference, pages 1201–1205. IEEE, 2007.
  36. On estimation of covariance matrices with Kronecker product structure. IEEE Transactions on Signal Processing, 56(2):478–491, 2008.
  37. J. Xie and P. M. Bentler. Covariance structure models for gene expression microarray data. Structural Equation Modeling, 10(4):566–582, 2003.
  38. N. Zhivotovskiy. Dimension-free bounds for sums of independent matrices and simple tensors via the variational principle. Electronic Journal of Probability, 29:1–28, 2024.
  39. S. Zhou. Gemini: Graph estimation with matrix variate normal instances. The Annals of Statistics, 42(2):532–562, 2014.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

Sign Up for Free