Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Matrices with Gaussian noise: optimal estimates for singular subspace perturbation (1803.00679v3)

Published 2 Mar 2018 in stat.ML, cs.IT, cs.LG, math.IT, and math.PR

Abstract: The Davis-Kahan-Wedin $\sin \Theta$ theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin $\sin \Theta$ theorem when the perturbation is a Gaussian random matrix. Under certain structural assumptions, we obtain an optimal bound that significantly improves upon the classic Davis-Kahan-Wedin $\sin \Theta$ theorem. One of our key tools is a new perturbation bound for the singular values, which may be of independent interest.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (77)
  1. Entrywise eigenvector analysis of random matrices with low expected rank. Ann. Statist., 48(3):1452–1474, 2020.
  2. R. Allez and J.-P. Bouchaud. Eigenvector dynamics under free addition. Random Matrices Theory Appl., 3(3):1450010, 17, 2014.
  3. Z. Bai and J. W. Silverstein. Spectral analysis of large dimensional random matrices. Springer Series in Statistics. Springer, New York, second edition, 2010.
  4. Z. Bai and J.-f. Yao. Central limit theorems for eigenvalues in a spiked population model. Ann. Inst. Henri Poincaré Probab. Stat., 44(3):447–474, 2008.
  5. Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab., 33(5):1643–1697, 2005.
  6. Statistical inference for principal components of spiked covariance matrices. Ann. Statist., 50(2):1144–1169, 2022.
  7. Singular vector and singular subspace distribution for the matrix denoising model. The Annals of Statistics, 49(1):370–392, 2021.
  8. Z. Bao and D. Wang. Eigenvector distribution in the critical regime of BBP transition. Probab. Theory Related Fields, 182(1-2):399–479, 2022.
  9. Eigenvectors of a matrix under random perturbation. Random Matrices Theory Appl., 10(2):Paper No. 2150023, 17, 2021.
  10. Fluctuations of the extreme eigenvalues of finite rank deformations of random matrices. Electron. J. Probab., 16:no. 60, 1621–1662, 2011.
  11. F. Benaych-Georges and R. R. Nadakuditi. The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices. Adv. Math., 227(1):494–521, 2011.
  12. F. Benaych-Georges and R. R. Nadakuditi. The singular values and vectors of low rank perturbations of large rectangular random matrices. J. Multivariate Anal., 111:120–135, 2012.
  13. R. Bhatia. Matrix analysis, volume 169 of Graduate Texts in Mathematics. Springer-Verlag, New York, 1997.
  14. A. Björck and G. H. Golub. Numerical methods for computing angles between linear subspaces. Math. Comp., 27:579–594, 1973.
  15. Isotropic local laws for sample covariance and generalized Wigner matrices. Electron. J. Probab., 19:no. 33, 53, 2014.
  16. On the principal components of sample covariance matrices. Probability theory and related fields, 164(1):459–552, 2016.
  17. Optimal estimation and rank detection for sparse spiked covariance matrices. Probab. Theory Related Fields, 161(3-4):781–815, 2015.
  18. E. J. Candès and Y. Plan. Matrix completion with noise. Proceedings of the IEEE, 98:925–936, 2010.
  19. E. J. Candès and B. Recht. Exact matrix completion via convex optimization. Found. Comput. Math., 9(6):717–772, 2009.
  20. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory, 52(2):489–509, 2006.
  21. Signal-plus-noise matrix models: eigenvector deviations and fluctuations. Biometrika, 106(1):243–250, 2019.
  22. The two-to-infinity norm and singular subspace geometry with applications to high-dimensional statistics. Ann. Statist., 47(5):2405–2439, 2019.
  23. Central limit theorems for eigenvalues of deformations of Wigner matrices. Ann. Inst. Henri Poincaré Probab. Stat., 48(1):107–133, 2012.
  24. Stochastic block model and community detection in sparse graphs: A spectral algorithm with optimal rate of recovery. In P. Grünwald, E. Hazan, and S. Kale, editors, Proceedings of The 28th Conference on Learning Theory, volume 40 of Proceedings of Machine Learning Research, pages 391–423, Paris, France, 03–06 Jul 2015. PMLR.
  25. C. Davis and W. M. Kahan. The rotation of eigenvectors by a perturbation. III. SIAM J. Numer. Anal., 7:1–46, 1970.
  26. Unperturbed: spectral analysis beyond Davis-Kahan. In Algorithmic learning theory 2018, volume 83 of Proc. Mach. Learn. Res. (PMLR), page 38. Proceedings of Machine Learning Research PMLR, [place of publication not identified], 2018.
  27. Bulk universality for generalized Wigner matrices. Probab. Theory Related Fields, 154(1-2):341–407, 2012.
  28. Rigidity of eigenvalues of generalized Wigner matrices. Adv. Math., 229(3):1435–1515, 2012.
  29. Spectral statistics of erdős-rényi graphs ii: Eigenvalue spacing and the extreme eigenvalues. Communications in Mathematical Physics, 314(3):587–640, 2012.
  30. Spectral statistics of erdős-rényi graphs I: Local semicircle law. Ann. Probab., 41(3B):2279–2375, 2013.
  31. J. Fan and X. Han. Estimation of the false discovery proportion with unknown dependence. J. R. Stat. Soc. Ser. B. Stat. Methodol., 79(4):1143–1164, 2017.
  32. Large covariance estimation by thresholding principal orthogonal complements. J. R. Stat. Soc. Ser. B. Stat. Methodol., 75(4):603–680, 2013. With 33 discussions by 57 authors and a reply by Fan, Liao and Mincheva.
  33. An ℓ∞subscriptℓ\ell_{\infty}roman_ℓ start_POSTSUBSCRIPT ∞ end_POSTSUBSCRIPT eigenvector perturbation bound and its application to robust covariance estimation. J. Mach. Learn. Res., 18:Paper No. 207, 42, 2017.
  34. Matrix computations. JHU press, 2013.
  35. O. Guédon and R. Vershynin. Community detection in sparse networks via Grothendieck’s inequality. Probab. Theory Related Fields, 165(3-4):1025–1049, 2016.
  36. Isotropic self-consistent equations for mean-field random matrices. Probab. Theory Related Fields, 171(1-2):203–249, 2018.
  37. N. J. Higham. A survey of componentwise perturbation theory in numerical linear algebra. In Mathematics of Computation 1943–1993: a half-century of computational mathematics (Vancouver, BC, 1993), volume 48 of Proc. Sympos. Appl. Math., pages 49–77. Amer. Math. Soc., Providence, RI, 1994.
  38. Matrix analysis. Cambridge University Press, Cambridge, second edition, 2013.
  39. I. C. F. Ipsen and R. Rehman. Perturbation bounds for determinants and characteristic polynomials. SIAM J. Matrix Anal. Appl., 30(2):762–776, 2008.
  40. I. T. Jolliffe. Principal component analysis. Springer Series in Statistics. Springer-Verlag, New York, 1986.
  41. R. Kannan and S. Vempala. Spectral algorithms. Found. Trends Theor. Comput. Sci., 4(3-4):front matter, 157–288 (2009), 2008.
  42. A. Knowles and J. Yin. The isotropic semicircle law and deformation of Wigner matrices. Comm. Pure Appl. Math., 66(11):1663–1750, 2013.
  43. A. Knowles and J. Yin. Anisotropic local laws for random matrices. Probab. Theory Related Fields, 169(1-2):257–352, 2017.
  44. V. Koltchinskii and D. Xia. Perturbation of linear forms of singular vectors under Gaussian noise. In High dimensional probability VII, volume 71 of Progr. Probab., pages 397–423. Springer, [Cham], 2016.
  45. J. O. Lee and K. Schnelli. Local law and Tracy-Widom limit for sparse random matrices. Probab. Theory Related Fields, 171(1-2):543–616, 2018.
  46. Bulk universality for deformed Wigner matrices. Ann. Probab., 44(3):2349–2425, 2016.
  47. Reconstruction in the labelled stochastic block model. IEEE Trans. Network Sci. Eng., 2(4):152–163, 2015.
  48. F. McSherry. Spectral partitioning of random graphs. In 42nd IEEE Symposium on Foundations of Computer Science (Las Vegas, NV, 2001), pages 529–537. IEEE Computer Soc., Los Alamitos, CA, 2001.
  49. B. Nadler. Finite sample approximation results for principal component analysis: a matrix perturbation approach. Ann. Statist., 36(6):2791–2817, 2008.
  50. Eigenvectors of random matrices: A survey. Journal of Combinatorial Theory, Series A, 144:361 – 442, 2016. Fifty Years of the Journal of Combinatorial Theory.
  51. Random perturbation of low rank matrices: improving classical bounds. Linear Algebra Appl., 540:26–59, 2018.
  52. S. O’Rourke and N. Williams. Pairing between zeros and critical points of random polynomials with independent roots. Trans. Amer. Math. Soc., 371(4):2343–2381, 2019.
  53. C. Pozrikidis. An introduction to grids, graphs, and networks. Oxford University Press, Oxford, 2014.
  54. J. W. S. Rayleigh, Baron. The Theory of Sound. Dover Publications, New York, N. Y., 1945. 2d ed.
  55. Spectral clustering and the high-dimensional stochastic blockmodel. Ann. Statist., 39(4):1878–1915, 2011.
  56. M. Rudelson and R. Vershynin. Non-asymptotic theory of random matrices: extreme singular values. In Proceedings of the International Congress of Mathematicians. Volume III, pages 1576–1602. Hindustan Book Agency, New Delhi, 2010.
  57. P. Sarkar and P. J. Bickel. Role of normalization in spectral clustering for stochastic blockmodels. Ann. Statist., 43(3):962–990, 2015.
  58. E. Schrödinger. Quantisierung als eigenwertproblem. Annalen der Physik, 384(4):361–376, 1926.
  59. Matrix perturbation theory. Computer Science and Scientific Computing. Academic Press, Inc., Boston, MA, 1990.
  60. A semiparametric two-sample hypothesis testing problem for random graphs. J. Comput. Graph. Statist., 26(2):344–354, 2017.
  61. Asymptotically efficient estimators for stochastic blockmodels: the naive MLE, the rank-constrained MLE, and the spectral estimator. Bernoulli, 28(2):1049–1073, 2022.
  62. T. Tao. Outliers in the spectrum of iid matrices with bounded rank perturbations. Probab. Theory Related Fields, 155(1-2):231–263, 2013.
  63. C. Tomasi and T. Kanade. Shape and motion from image streams: a factorization method. Proceedings of the National Academy of Sciences, 90(21):9795–9802, 1993.
  64. R. Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
  65. U. von Luxburg. A tutorial on spectral clustering. Stat. Comput., 17(4):395–416, 2007.
  66. V. Vu. Singular vectors under random perturbation. Random Structures Algorithms, 39(4):526–538, 2011.
  67. V. Vu. A simple SVD algorithm for finding hidden partitions. Combin. Probab. Comput., 27(1):124–140, 2018.
  68. M. J. Wainwright. High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cambridge University Press, 2019.
  69. R. Wang. Singular vector perturbation under Gaussian noise. SIAM J. Matrix Anal. Appl., 36(1):158–177, 2015.
  70. P.-A. Wedin. Perturbation bounds in connection with singular value decomposition. Nordisk Tidskr. Informationsbehandling (BIT), 12:99–111, 1972.
  71. D. Xia and F. Zhou. The sup-norm perturbation of HOSVD and low rank tensor denoising. J. Mach. Learn. Res., 20:Paper No. 61, 42, 2019.
  72. A useful variant of the Davis-Kahan theorem for statisticians. Biometrika, 102(2):315–323, 2015.
  73. S.-Y. Yun and A. Proutiere. Accurate community detection in the stochastic block model via spectral algorithms. Available at arXiv:1412.7335, 2014.
  74. S.-Y. Yun and A. Proutiere. Optimal cluster recovery in the labeled stochastic block model. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016.
  75. Empirical Bayes PCA in high dimensions. J. R. Stat. Soc. Ser. B. Stat. Methodol., 84(3):853–878, 2022.
  76. Y. Zhong. Eigenvector under random perturbation: A nonasymptotic rayleigh-schrödinger theory. Available at arXiv:1702.00139, 2017.
  77. Y. Zhong and N. Boumal. Near-optimal bounds for phase synchronization. SIAM J. Optim., 28(2):989–1016, 2018.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now