2000 character limit reached
Entry-Specific Bounds for Low-Rank Matrix Completion under Highly Non-Uniform Sampling (2403.00184v1)
Published 29 Feb 2024 in stat.ML and cs.LG
Abstract: Low-rank matrix completion concerns the problem of estimating unobserved entries in a matrix using a sparse set of observed entries. We consider the non-uniform setting where the observed entries are sampled with highly varying probabilities, potentially with different asymptotic scalings. We show that under structured sampling probabilities, it is often better and sometimes optimal to run estimation algorithms on a smaller submatrix rather than the entire matrix. In particular, we prove error upper bounds customized to each entry, which match the minimax lower bounds under certain conditions. Our bounds characterize the hardness of estimating each entry as a function of the localized sampling probabilities. We provide numerical experiments that confirm our theoretical findings.
- Entrywise eigenvector analysis of random matrices with low expected rank. Annals of Statistics, 48 3:1452–1474, 2020.
- Causal matrix completion. arXiv preprint arXiv:2109.15154, 2021.
- Universal matrix completion. In International Conference on Machine Learning, pages 1881–1889. PMLR, 2014.
- Semidefinite programming based algorithms for sensor network localization. ACM Trans. Sen. Netw., 2(2):188–220, may 2006.
- A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4):1956–1982, 2010.
- Matrix completion via max-norm constrained optimization. Electronic Journal of Statistics, 10:1493–1525, 2016.
- Exact matrix completion via convex optimization. Communications of the ACM, 55(6):111–119, 2012.
- Matrix completion with noise. Proceedings of the IEEE, 98(6):925–936, 2010.
- Sourav Chatterjee. A deterministic theory of low rank matrix completion. IEEE Transactions on Information Theory, 66(12):8046–8055, 2020.
- Harnessing structures in big data via guaranteed low-rank matrix estimation: Recent theory and fast algorithms via convex and nonconvex optimization. IEEE Signal Processing Magazine, 35(4):14–31, 2018.
- Noisy matrix completion: Understanding statistical guarantees for convex relaxation via nonconvex optimization. SIAM journal on optimization, 30(4):3098–3121, 2020.
- An overview of low-rank matrix recovery from incomplete observations. IEEE Journal of Selected Topics in Signal Processing, 10(4):608–622, 2016.
- Weighted matrix completion from non-random, non-uniform sampling patterns. IEEE Transactions on Information Theory, 67(2):1264–1290, Feb 2021.
- Computational limits for matrix completion. In Conference on Learning Theory, pages 703–725. PMLR, 2014.
- Deterministic algorithms for matrix completion. Random Struct. Algorithms, 45(2):306–317, sep 2014.
- Low-rank matrix completion using alternating minimization. In Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing, STOC ’13, page 665–674, New York, NY, USA, 2013. Association for Computing Machinery.
- Matrix completion from noisy entries. Advances in neural information processing systems, 22, 2009.
- Matrix completion from a few entries. IEEE transactions on information theory, 56(6):2980–2998, 2010.
- Matrix factorization techniques for recommender systems. Computer, 42(8):30–37, 2009.
- Matrix completion from any given set of observations. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
- Interior-point method for nuclear norm approximation with application to system identification. SIAM Journal on Matrix Analysis and Applications, 31(3):1235–1256, 2010.
- László Lovász. Large networks and graph limits, volume 60. American Mathematical Soc., 2012.
- Jasson D. M. Rennie and Nathan Srebro. Fast maximum margin matrix factorization for collaborative prediction. In Proceedings of the 22nd International Conference on Machine Learning, ICML ’05, page 713–719, New York, NY, USA, 2005. Association for Computing Machinery.
- Joel A Tropp et al. An introduction to matrix concentration inequalities. Foundations and Trends® in Machine Learning, 8(1-2):1–230, 2015.
- Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
- Martin J. Wainwright. High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2019.