Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Randomized Approach to Matrix Completion: Applications in Recommendation Systems and Image Inpainting (2403.01919v4)

Published 4 Mar 2024 in cs.LG

Abstract: We present a novel method for matrix completion, specifically designed for matrices where one dimension significantly exceeds the other. Our Columns Selected Matrix Completion (CSMC) method combines Column Subset Selection and Low-Rank Matrix Completion to efficiently reconstruct incomplete datasets. In each step, CSMC solves a convex optimization task. We introduce two algorithms that implement CSMC, each tailored to different problem sizes. A formal analysis outlines the necessary assumptions and the probability of a correct solution. To assess the impact of matrix size, rank, and the proportion of missing entries on solution quality and computation time, we conducted experiments on synthetic data. The method was applied to two real-world problems: recommendation systems and image inpainting. Our results show that CSMC delivers solutions comparable to state-of-the-art matrix completion algorithms based on convex optimization, but with significant runtime savings. This makes CSMC especially valuable for systems that require efficient processing of large, incomplete datasets while maintaining the integrity of the derived insights.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Structured matrix completion with applications to genomic data integration. Journal of the American Statistical Association, 111, 04 2015.
  2. Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Transactions on Image Processing, 24(9):2658–2670, 2015.
  3. Matrix completion with cross-concentrated sampling: Bridging uniform sampling and cur sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  4. Light field inpainting propagation via low rank matrix completion. IEEE Transactions on Image Processing, 27(4):1981–1993, 2018.
  5. Ankur Moitra. Algorithmic Aspects of Machine Learning. Cambridge University Press, USA, 1st edition, 2018.
  6. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3):471–501, 2010.
  7. Benjamin Recht. A simpler approach to matrix completion. J. Mach. Learn. Res., 12(null):3413–3430, dec 2011.
  8. David Gross. Recovering low-rank matrices from few coefficients in any basis. IEEE Transactions on Information Theory, 57(3):1548–1566, Mar 2011.
  9. The power of convex relaxation: Near-optimal matrix completion. IEEE Transactions on Information Theory, 56(5):2053–2080, 2010.
  10. Harnessing structures in big data via guaranteed low-rank matrix estimation: Recent theory and fast algorithms via convex and nonconvex optimization. IEEE Signal Processing Magazine, 35(4):14–31, 2018.
  11. David P. Woodruff. Sketching as a tool for numerical linear algebra. Found. Trends Theor. Comput. Sci., 10(1–2):1–157, oct 2014.
  12. Computational methods for sparse solution of linear inverse problems. Proceedings of the IEEE, 98(6):948–958, 2010.
  13. Faster least squares approximation. Numerische Mathematik, 117, 10 2007.
  14. Low-rank matrix completion using alternating minimization. In Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing, STOC ’13, page 665–674, New York, NY, USA, 2013. Association for Computing Machinery.
  15. Spectral regularization algorithms for learning large incomplete matrices. Journal of Machine Learning Research, 11(80):2287–2322, 2010.
  16. A singular value thresholding algorithm for matrix completion. SIAM Journal on optimization, 20(4):1956–1982, 2010.
  17. Exact matrix completion via convex optimization. Commun. ACM, 55(6):111–119, jun 2012.
  18. Moritz Hardt. Understanding alternating minimization for matrix completion. 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, pages 651–660, 2014.
  19. Bart Vandereycken. Low-rank matrix completion by riemannian optimization. SIAM Journal on Optimization, 23(2):1214–1236, 2013.
  20. Scaled gradients on grassmann manifolds for matrix completion. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012.
  21. Robust low-rank matrix completion by riemannian optimization. SIAM Journal on Scientific Computing, 38(5):S440–S460, 2016.
  22. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20:1956–1982, 03 2010.
  23. Martin Jaggi. Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In Sanjoy Dasgupta and David McAllester, editors, Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 427–435, Atlanta, Georgia, USA, 17–19 Jun 2013. PMLR.
  24. An extended frank–wolfe method with “in-face” directions, and its application to low-rank matrix completion. SIAM Journal on Optimization, 27, 11 2015.
  25. Sketchy decisions: Convex low-rank matrix optimization with optimal storage. In Artificial intelligence and statistics, pages 1188–1196. PMLR, 2017.
  26. Local minima and convergence in low-rank semidefinite programming. Mathematical Programming, 103:427–444, 07 2005.
  27. Matrix approximation and projective clustering via volume sampling. Theory of Computing, 2:225–247, 01 2006.
  28. Relative-error c⁢u⁢r𝑐𝑢𝑟curitalic_c italic_u italic_r matrix decompositions. SIAM Journal on Matrix Analysis and Applications, 30:844–881, 05 2008.
  29. A deim induced cur factorization. SIAM Journal on Scientific Computing, 38(3):A1454–A1482, 2016.
  30. Perspectives on cur decompositions. Applied and Computational Harmonic Analysis, 48(3):1088–1099, 2020.
  31. Efficient algorithms for cur and interpolative matrix decompositions. Advances in Computational Mathematics, 43:495–516, 2017.
  32. Optimal cur matrix decompositions. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing, pages 353–362, 2014.
  33. The pseudoinverse of A=C⁢R𝐴𝐶𝑅A=CRitalic_A = italic_C italic_R is A+=R++C+superscript𝐴superscript𝑅superscript𝐶A^{+}=R^{+}+C^{+}italic_A start_POSTSUPERSCRIPT + end_POSTSUPERSCRIPT = italic_R start_POSTSUPERSCRIPT + end_POSTSUPERSCRIPT + italic_C start_POSTSUPERSCRIPT + end_POSTSUPERSCRIPT(?). arXiv preprint arXiv:2305.01716, 2023.
  34. Provably correct algorithms for matrix column subset selection with selectively sampled data. J. Mach. Learn. Res., 18(1):5699–5740, jan 2017.
  35. MC2: a two-phase algorithm for leveraged matrix completion. Information and Inference: A Journal of the IMA, 7(3):581–604, 02 2018.
  36. On the power of adaptivity in matrix completion and approximation. CoRR, abs/1407.3619, 2014.
  37. Low-rank matrix and tensor completion via adaptive sampling. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
  38. Cur algorithm for partially observed matrices. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML’15, page 1412–1421. JMLR.org, 2015.
  39. Robust cur decomposition: Theory and imaging applications. SIAM J. Imaging Sci., 14:1472–1503, 2021.
  40. Brendan O’Donoghue. Operator splitting for a homogeneous embedding of the linear complementarity problem. SIAM Journal on Optimization, 31:1999–2023, August 2021.
  41. Array programming with NumPy. Nature, 585(7825):357–362, September 2020.
  42. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019.
  43. Globally convergent type–I Anderson acceleration for non-smooth fixed-point iterations. SIAM Journal on Optimization, 30(4):3170–3197, 2020.
  44. fancyimpute: An imputation library for python.
  45. A. W. van der Vaart. Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1998.
  46. Paweł Szynkiewicz. A comparative study of pso and cma-es algorithms on black-box optimization benchmarks. Journal of Telecommunications and Information Technology, 8, 01 2019.
  47. Implicit regularization in nonconvex statistical estimation: Gradient descent converges linearly for phase retrieval, matrix completion and blind deconvolution. Foundations of Computational Mathematics, 20, 11 2017.
  48. Missing value estimation methods for dna microarrays. Bioinformatics, 17 6:520–5, 2001.
  49. The movielens datasets: History and context. 5(4), dec 2015.
  50. A random walk method for alleviating the sparsity problem in collaborative filtering. In Proceedings of the 2008 ACM Conference on Recommender Systems, RecSys ’08, page 131–138, New York, NY, USA, 2008. Association for Computing Machinery.
  51. Convex optimization. Cambridge university press, 2004.
  52. Joel Tropp. Improved analysis of the subsamples randomized hadamard transform. Advances in Adaptive Data Analysis, 03, 11 2010.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now