2000 character limit reached
Matrix factorization with neural networks (2212.02105v1)
Published 5 Dec 2022 in cond-mat.dis-nn and cs.LG
Abstract: Matrix factorization is an important mathematical problem encountered in the context of dictionary learning, recommendation systems and machine learning. We introduce a new `decimation' scheme that maps it to neural network models of associative memory and provide a detailed theoretical analysis of its performance, showing that decimation is able to factorize extensive-rank matrices and to denoise them efficiently. We introduce a decimation algorithm based on ground-state search of the neural network, which shows performances that match the theoretical prediction.
- B. A. Olshausen and D. J. Field, Nature 381, 607 (1996).
- B. A. Olshausen and D. J. Field, Vision Research 37, 3311 (1997).
- Y. Bengio, A. Courville, and P. Vincent, IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 1798 (2013).
- D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, Cognitive Science 9, 147 (1985).
- J. Mairal, M. Elad, and G. Sapiro, IEEE Transactions on Image Processing 17, 53 (2008).
- M. Elad and M. Aharon, Trans. Img. Proc. 15, 3736–3745 (2006).
- I. M. Johnstone, The Annals of Statistics 29, 295 (2001).
- F. Krzakala, J. Xu, and L. Zdeborová, in 2016 IEEE Information Theory Workshop (ITW) (2016) pp. 71–75.
- J. Barbier and N. Macris, Journal of Physics A: Mathematical and Theoretical 52, 294002 (2019).
- A. E. Alaoui, F. Krzakala, and M. Jordan, The Annals of Statistics 48, 863 (2020).
- J. Baik, G. Ben-Arous, and S. Péché, The Annals of Probability 33, 1643 (2005).
- F. Benaych-Georges and R. R. Nadakuditi, Advances in Mathematics 227, 494 (2011).
- Y. Kabashima, Journal of Physics A: Mathematical and General 36, 11111 (2003).
- D. L. Donoho, A. Maleki, and A. Montanari, Proceedings of the National Academy of Sciences 106, 18914 (2009).
- J. Barbier and N. Macris, Phys. Rev. E 106, 024136 (2022).
- J. J. Hopfield, D. I. Feinstein, and R. G. Palmer, Nature 304, 158 (1983).
- J. J. Hopfield, Proceedings of the National Academy of Sciences 79, 2554 (1982), https://www.pnas.org/doi/pdf/10.1073/pnas.79.8.2554 .
- D. J. Amit, H. Gutfreund, and H. Sompolinsky, Phys. Rev. A 32, 1007 (1985a).
- D. J. Amit, H. Gutfreund, and H. Sompolinsky, Phys. Rev. Lett. 55, 1530 (1985b).
- M. Mézard, G. Parisi, and M. Virasoro, Spin Glass Theory and Beyond (WORLD SCIENTIFIC, 1986) https://www.worldscientific.com/doi/pdf/10.1142/0271 .