Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Understanding quantum machine learning also requires rethinking generalization (2306.13461v2)

Published 23 Jun 2023 in quant-ph, cond-mat.quant-gas, cs.LG, and stat.ML

Abstract: Quantum machine learning models have shown successful generalization performance even when trained with few data. In this work, through systematic randomization experiments, we show that traditional approaches to understanding generalization fail to explain the behavior of such quantum models. Our experiments reveal that state-of-the-art quantum neural networks accurately fit random states and random labeling of training data. This ability to memorize random data defies current notions of small generalization error, problematizing approaches that build on complexity measures such as the VC dimension, the Rademacher complexity, and all their uniform relatives. We complement our empirical results with a theoretical construction showing that quantum neural networks can fit arbitrary labels to quantum states, hinting at their memorization ability. Our results do not preclude the possibility of good generalization with few training data but rather rule out any possible guarantees based only on the properties of the model family. These findings expose a fundamental challenge in the conventional understanding of generalization in quantum machine learning and highlight the need for a paradigm shift in the study of quantum models for machine learning tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (94)
  1. P. W. Shor, “Algorithms for quantum computation: discrete logarithms and factoring,” in Proceedings 35th Ann. Symp. Found. Compu. Sc. (IEEE, 1994) pp. 124–134.
  2. A. Montanaro, “Quantum algorithms: an overview,” npj Quant. Inf. 2, 15023 (2016).
  3. F. Arute et al., “Quantum supremacy using a programmable superconducting processor,” Nature 574, 505–510 (2019).
  4. Y. Wu et al., “Strong quantum computational advantage using a superconducting quantum processor,” Phys. Rev. Lett. 127, 180501 (2021a).
  5. D. Hangleiter and J. Eisert, “Computational advantage of quantum random sampling,” Rev. Mod. Phys. 95, 035001 (2023).
  6. J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe,  and S. Lloyd, “Quantum machine learning,” Nature 549, 195–202 (2017).
  7. V. Dunjko and H. J. Briegel, “Machine learning & artificial intelligence in the quantum domain: a review of recent progress,” Rep. Prog. Phys. 81, 074001 (2018).
  8. M. Schuld and F. Petruccione, Machine Learning with Quantum Computers (Springer International Publishing, 2021).
  9. G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto,  and L. Zdeborová, “Machine learning and the physical sciences,” Rev. Mod. Phys. 91, 045002 (2019).
  10. M. Schuld, M. Fingerhuth,  and F. Petruccione, “Implementing a distance-based classifier with a quantum interference circuit,” Europhys. Lett. 119, 60002 (2017).
  11. V. Havlíček, A. D. Córcoles, K. Temme, A. W. Harrow, A. Kandala, J. M. Chow,  and J. M. Gambetta, “Supervised learning with quantum-enhanced feature spaces,” Nature 567, 209–212 (2019).
  12. M. Schuld and N. Killoran, “Quantum machine learning in feature Hilbert spaces,” Phys. Rev. Lett. 122, 040504 (2019).
  13. M. Benedetti, D. Garcia-Pintos, O. Perdomo, V. Leyton-Ortega, Y. Nam,  and A. Perdomo-Ortiz, “A generative modeling approach for benchmarking and training shallow quantum circuits,” npj Quant. Inf. 5, 45 (2019a).
  14. D. Zhu et al., “Training of quantum circuits on a hybrid quantum computer,” Science Advances 5, eaaw9918 (2019).
  15. A. Pérez-Salinas, A. Cervera-Lierta, E. Gil-Fuster,  and J. I. Latorre, “Data re-uploading for a universal quantum classifier,” Quantum 4, 226 (2020).
  16. B. Coyle, D. Mills, V. Danos,  and E. Kashefi, “The born supremacy: quantum advantage and training of an ising born machine,” npj Quant. Inf. 6, 60 (2020).
  17. S. Lloyd, M. Schuld, A. Ijaz, J. Izaac,  and N. Killoran, “Quantum embeddings for machine learning,” arXiv:2001.03622  (2020).
  18. T. Hubregtsen, D. Wierichs, E. Gil-Fuster, P.-J. H. S. Derks, P. K. Faehrmann,  and J. J. Meyer, “Training quantum embedding kernels on near-term quantum computers,” Phys. Rev. A 106, 042431 (2022).
  19. M. S. Rudolph, N. B. Toussaint, A. Katabarwa, S. Johri, B. Peropadre,  and A. Perdomo-Ortiz, “Generation of high-resolution handwritten digits with an ion-trap quantum computer,” Phys. Rev. X 12, 031010 (2022a).
  20. C. Bravo-Prieto, J. Baglio, M. Cè, A. Francis, D. M. Grabowska,  and S. Carrazza, “Style-based quantum generative adversarial networks for Monte Carlo events,” Quantum 6, 777 (2022).
  21. M. Benedetti, E. Lloyd, S. Sack,  and M. Fiorentini, “Parameterized quantum circuits as machine learning models,” Quantum Sc. Tech. 4, 043001 (2019b).
  22. M. Cerezo et al., “Variational quantum algorithms,” Nature Rev. Phys. 3, 625–644 (2021a).
  23. K. Bharti et al., “Noisy intermediate-scale quantum algorithms,” Rev. Mod. Phys. 94, 015004 (2022).
  24. R. Sweke, J.-P. Seifert, D. Hangleiter,  and J. Eisert, “On the quantum versus classical learnability of discrete distributions,” Quantum 5, 417 (2021).
  25. Y. Liu, S. Arunachalam,  and K. Temme, “A rigorous and robust quantum speed-up in supervised machine learning,” Nature Phys. 17, 1013 (2021).
  26. S. Jerbi, L. M. Trenkwalder, H. Poulsen Nautrup, H. J. Briegel,  and V. Dunjko, “Quantum enhancements for deep reinforcement learning in large spaces,” PRX Quantum 2, 010328 (2021).
  27. N. Pirnay, R. Sweke, J. Eisert,  and J.-P. Seifert, “A super-polynomial quantum-classical separation for density modelling,” Phys. Rev. A 107, 042416 (2023).
  28. S. Sim, P. D. Johnson,  and A. Aspuru-Guzik, “Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms,” Adv. Quant. Tech. 2, 1900070 (2019).
  29. C. Bravo-Prieto, J. Lumbreras-Zarapico, L. Tagliacozzo,  and J. I. Latorre, “Scaling of variational quantum circuit depth for condensed matter systems,” Quantum 4, 272 (2020).
  30. Y. Wu, J. Yao, P. Zhang,  and H. Zhai, “Expressivity of quantum neural networks,” Phys. Rev. Res. 3, L032049 (2021b).
  31. D. Herman, R. Raymond, M. Li, N. Robles, A. Mezzacapo,  and M. Pistoia, “Expressivity of variational quantum machine learning on the Boolean cube,” IEEE Trans. Quant. Eng. 4, 1–18 (2023).
  32. T. Hubregtsen, J. Pichlmeier, P. Stecher,  and K. Bertels, “Evaluation of parameterized quantum circuits: on the relation between classification accuracy, expressibility, and entangling capability,” Quant. Mach. Intell. 3, 1–19 (2021).
  33. T. Haug, K. Bharti,  and M. S. Kim, “Capacity and quantum geometry of parametrized quantum circuits,” PRX Quantum 2, 040309 (2021).
  34. Z. Holmes, K. Sharma, M. Cerezo,  and P. J. Coles, “Connecting ansatz expressibility to gradient magnitudes and barren plateaus,” PRX Quantum 3, 010313 (2022).
  35. J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush,  and H. Neven, “Barren plateaus in quantum neural network training landscapes,” Nature Comm. 9, 4812 (2018).
  36. M. Cerezo, Akira Sone, T. Volkoff, L. Cincio,  and P. J Coles, “Cost function dependent barren plateaus in shallow parametrized quantum circuits,” Nature Comm. 12, 1791 (2021b).
  37. A. Arrasmith, M. Cerezo, P. Czarnik, L. Cincio,  and P. J. Coles, “Effect of barren plateaus on gradient-free optimization,” Quantum 5, 558 (2021).
  38. J. Kim, J. Kim,  and D. Rosa, “Universal effectiveness of high-depth circuits in variational eigenproblems,” Phys. Rev. Res. 3, 023203 (2021).
  39. S. Wang, E. Fontana, M. Cerezo, K. Sharma, A. Sone, L. Cincio,  and P. J Coles, “Noise-induced barren plateaus in variational quantum algorithms,” Nature Comm. 12, 6961 (2021).
  40. A. Pesah, M. Cerezo, S. Wang, T. Volkoff, A. T. Sornborger,  and P. J. Coles, “Absence of barren plateaus in quantum convolutional neural networks,” Phys. Rev. X 11, 041011 (2021).
  41. C. Ortiz Marrero, M. Kieferová,  and N. Wiebe, “Entanglement-induced barren plateaus,” PRX Quantum 2, 040316 (2021).
  42. M. Larocca, N. Ju, D. García-Martín, P. J. Coles,  and M. Cerezo, “Theory of overparametrization in quantum neural networks,” Nature Comp. Sc. 3, 542–551 (2023).
  43. K. Sharma, M. Cerezo, L. Cincio,  and P. J Coles, “Trainability of dissipative perceptron-based quantum neural networks,” Phys. Rev. Lett. 128, 180505 (2022).
  44. M. S. Rudolph, S. Lerch, S. Thanasilp, O. Kiss, S. Vallecorsa, M. Grossi,  and Z. Holmes, “Trainability barriers and opportunities in quantum generative modeling,” arXiv:2305.02881  (2023).
  45. M. C. Caro and I. Datta, “Pseudo-dimension of quantum circuits,” Quant, Mach. Intell. 2, 14 (2020).
  46. A. Abbas, D. Sutter, C. Zoufal, A. Lucchi, A. Figalli,  and S. Woerner, “The power of quantum neural networks,” Nature Comp. Sc. 1, 403–409 (2021).
  47. L. Banchi, J. Pereira,  and S. Pirandola, “Generalization in quantum machine learning: A quantum information standpoint,” PRX Quantum 2, 040321 (2021).
  48. K. Bu, D. E. Koh, L. Li, Q. Luo,  and Y. Zhang, “Effects of quantum resources and noise on the statistical complexity of quantum circuits,” Quant. Sc. Tech. 8, 025013 (2023).
  49. K. Bu, D. E. Koh, L. Li, Q. Luo,  and Y. Zhang, “Rademacher complexity of noisy quantum circuits,” arXiv:2103.03139  (2021).
  50. K. Bu, D. E. Koh, L. Li, Q. Luo,  and Y. Zhang, “Statistical complexity of quantum circuits,” Phys. Rev. A 105, 062431 (2022).
  51. Y. Du, Z. Tu, X. Yuan,  and D. Tao, “Efficient measure for the expressivity of variational quantum algorithms,” Phys. Rev. Lett. 128, 080506 (2022).
  52. C. Gyurik and V. Dunjko, “Structural risk minimization for quantum linear classifiers,” Quantum 7, 893 (2023).
  53. M. C. Caro, E. Gil-Fuster, J. Jakob Meyer, J. Eisert,  and R. Sweke, “Encoding-dependent generalization bounds for parametrized quantum circuits,” Quantum 5, 582 (2021).
  54. M. C. Caro, H.-Y. Huang, M. Cerezo, K. Sharma, A. Sornborger, L. Cincio,  and P. J. Coles, “Generalization in quantum machine learning from few training data,” Nature Comm. 13, 4919 (2022).
  55. M. C. Caro, H.-Y. Huang, N. Ezzell, J. Gibbs, A. T. Sornborger, L. Cincio, P. J. Coles,  and Z. Holmes, “Out-of-distribution generalization for learning quantum dynamics,” Nature Comm. 14, 3751 (2023).
  56. Y. Qian, X. Wang, Y. Du, X. Wu,  and D. Tao, “The dilemma of quantum neural networks,” IEEE Trans. Neu. Net. Learn. Sys. , 1–13 (2022).
  57. Y. Du, Y. Yang, D. Tao,  and M.-H. Hsieh, “Problem-dependent power of quantum neural networks on multiclass classification,” Phys. Rev. Lett. 131, 140601 (2023).
  58. L. Schatzki, M. Larocca, F. Sauvage,  and M. Cerezo, “Theoretical guarantees for permutation-equivariant quantum neural networks,” arXiv:2210.09974  (2022).
  59. E. Peters and M. Schuld, “Generalization despite overfitting in quantum machine learning models,” arXiv:2209.05523  (2022).
  60. T. Haug and M. S. Kim, “Generalization with quantum geometry for learning unitaries,” arXiv:2303.13462  (2023).
  61. V. N. Vapnik and A. Y. Chervonenkis, “On the uniform convergence of relative frequencies of events to their probabilities,” Th. Prob. Appl. 16, 264–280 (1971).
  62. C. Zhang, S. Bengio, M. Hardt, B. Recht,  and O. Vinyals, “Understanding deep learning requires rethinking generalization,” in Int. Conf. Learn. Rep. (2017).
  63. E. S. Edgington and P. Onghena, Randomization Tests, 4th ed., Statistics: A Series of Textbooks and Monographs (Chapman & Hall/CRC, Philadelphia, PA, 2007).
  64. L. G. Valiant, “A theory of the learnable,” Commun. ACM 27, 1134–1142 (1984).
  65. S. Shalev-Shwartz and S. Ben-David, Understanding machine learning (Cambridge University Press, Cambridge, England, 2014).
  66. V. Vapnik, The nature of statistical learning theory (Springer science & business media, 1999).
  67. P. L. Bartlett and Shahar Mendelson, “Rademacher and gaussian complexities: Risk bounds and structural results,” J. Mach. Learn. Res. 3, 463–482 (2003).
  68. S. Mukherjee, P. Niyogi, T. Poggio,  and R. Rifkin, “Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization,” Advances in Computational Mathematics 25, 161–193 (2006).
  69. I. Cong, S. Choi,  and M. D. Lukin, “Quantum convolutional neural networks,” Nature Phys. 15, 1273–1278 (2019).
  70. K. Kottmann, F. Metz, J. Fraxanet,  and N. Baldelli, “Variational quantum anomaly detection: Unsupervised mapping of phase diagrams on a physical quantum computer,” Phys. Rev. Res. 3, 043184 (2021).
  71. S. Jerbi, J. Gibbs, M. S. Rudolph, M. C. Caro, P. J. Coles, H.-Y. Huang,  and Z. Holmes, “The power and limitations of learning quantum dynamics incoherently,” arXiv:2303.12834  (2023).
  72. J. Carrasquilla and R. G. Melko, “Machine learning phases of matter,” Nature Phys. 13, 431–434 (2017).
  73. S. Sachdev, Quantum phases of matter (Cambridge University Press, Massachusetts, 2023).
  74. P. Broecker, J. Carrasquilla, R. G. Melko,  and S. Trebst, “Machine learning quantum phases of matter beyond the fermion sign problem,” Sc. Rep. 7, 8823 (2017).
  75. R. Verresen, R. Moessner,  and F. Pollmann, “One-dimensional symmetry protected topological phases and their transitions,” Phys. Rev. B 96, 165124 (2017).
  76. G. Vidal, “Class of quantum many-body states that can be efficiently simulated,” Phys. Rev. Lett. 101, 110501 (2008).
  77. H.-Y. Huang, R. Kueng,  and J. Preskill, “Predicting many properties of a quantum system from very few measurements,” Nature Phys. 16, 1050–1057 (2020).
  78. M. S. Rudolph, J. Chen, J. Miller, A. Acharya,  and A. Perdomo-Ortiz, “Decomposition of matrix product states into shallow quantum circuits,” arXiv:2209.00595  (2022b).
  79. A. M. Childs and N. Wiebe, “Hamiltonian simulation using linear combinations of unitary operations,” Quantum Information and Computation 12, 901–924 (2012).
  80. A. Anshu, Z. Landau,  and Y. Liu, “Distributed quantum inner product estimation,” in Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2022 (Association for Computing Machinery, 2022) p. 44–51.
  81. Y. Jiang, B. Neyshabur, H. Mobahi, D. Krishnan,  and S. Bengio, “Fantastic generalization measures and where to find them,” arXiv:1912.02178  (2019).
  82. A. Kandala, A. Mezzacapo, K. Temme, Maika Takita, M. Brink, J. M. Chow,  and J. M. Gambetta, “Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets,” Nature 549, 242–246 (2017).
  83. J. J. Meyer, M. Mularski, E. Gil-Fuster, A. A. Mele, F. Arzani, A. Wilms,  and J. Eisert, “Exploiting symmetry in variational quantum machine learning,” PRX Quantum 4, 010328 (2023).
  84. A. Skolik, M. Cattelan, S. Yarkoni, T. Bäck,  and V. Dunjko, “Equivariant quantum circuits for learning on weighted graphs,” npj Quant. Inf. 9, 47 (2023).
  85. M. Larocca, F. Sauvage, F. M. Sbahi, G. Verdon, P. J. Coles,  and M. Cerezo, “Group-invariant quantum machine learning,” PRX Quantum 3, 030341 (2022).
  86. S. R. White, “Density matrix formulation for quantum renormalization groups,” Phys. Rev. Lett. 69, 2863 (1992).
  87. J. Gray, “QUIMB: a python library for quantum information and many-body calculations,” J. Open Source Soft. 3, 819 (2018).
  88. S.-X. Zhang, J. Allcock, Z.-Q. Wan, S. Liu, J. Sun, H. Yu, X.-H. Yang, J. Qiu, Z. Ye, Y.-Q. Chen, et al., “Tensorcircuit: a quantum software framework for the NISQ era,” Quantum 7, 912 (2023).
  89. S. Efthymiou, S. Ramos-Calderer, C. Bravo-Prieto, A. Pérez-Salinas, D. García-Martín, A. Garcia-Saez, J. I. Latorre,  and S. Carrazza, “Qibo: a framework for quantum simulation with hardware acceleration,” Quantum Sc. Tech. 7, 015018 (2021).
  90. N. Hansen, Y. Akimoto,  and P. Baudis, “CMA-ES/pycma on Github,”  (2019).
  91. E. Gil-Fuster, J. Eisert,  and C. Bravo-Prieto, “Understanding quantum machine learning also requires rethinking generalization,” Zenodo database  (2023), 10.5281/zenodo.10277124.
  92. S. A. Gershgorin, “Über die Abgrenzung der Eigenwerte einer Matrix,” Bulletin de l’Académie des Sciences de l’URSS. Classe des sciences mathématiques et naturelles , 749–754 (1931).
  93. S. Boyd and L. Vanderberghe, Convex optimization (Cambridge University Press, Cambridge, 2004).
  94. M. Mottonen, J. J. Vartiainen, V. Bergholm,  and M. M. Salomaa, “Transformation of quantum states using uniformly controlled rotations,” arXiv:0407010  (2004).
Citations (30)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now