Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Quantum Self-Attention Neural Networks for Text Classification (2205.05625v2)

Published 11 May 2022 in quant-ph, cs.AI, and cs.LG

Abstract: An emerging direction of quantum computing is to establish meaningful quantum applications in various fields of artificial intelligence, including NLP. Although some efforts based on syntactic analysis have opened the door to research in Quantum NLP (QNLP), limitations such as heavy syntactic preprocessing and syntax-dependent network architecture make them impracticable on larger and real-world data sets. In this paper, we propose a new simple network architecture, called the quantum self-attention neural network (QSANN), which can compensate for these limitations. Specifically, we introduce the self-attention mechanism into quantum neural networks and then utilize a Gaussian projected quantum self-attention serving as a sensible quantum version of self-attention. As a result, QSANN is effective and scalable on larger data sets and has the desirable property of being implementable on near-term quantum devices. In particular, our QSANN outperforms the best existing QNLP model based on syntactic analysis as well as a simple classical self-attention neural network in numerical experiments of text classification tasks on public data sets. We further show that our method exhibits robustness to low-level quantum noises and showcases resilience to quantum neural network architectures.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. John Preskill. Quantum computing 40 years later. arXiv preprint arXiv:2106.10522, 2021.
  2. Quantum computational supremacy. Nature, 549(7671):203–209, 2017.
  3. Andrew M Childs and Wim van Dam. Quantum algorithms for algebraic problems. Reviews of Modern Physics, 82(1):1–52, jan 2010.
  4. Ashley Montanaro. Quantum algorithms: an overview. npj Quantum Information, 2(1):15023, nov 2016.
  5. Toward the first quantum simulation with quantum speedup. Proceedings of the National Academy of Sciences, 115(38):9456–9461, sep 2018.
  6. Quantum machine learning. Nature, 549(7671):195–202, Sep 2017.
  7. Quantum Speed-Ups for Solving Semidefinite Programs. In 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pages 415–426. IEEE, oct 2017.
  8. Secure quantum key distribution with realistic devices. Reviews of Modern Physics, 92(2):25002, 2020.
  9. Quantum computational chemistry. Reviews of Modern Physics, 92(1):015003, mar 2020.
  10. Quantum Chemistry in the Age of Quantum Computing. Chemical Reviews, 119(19):10856–10915, oct 2019.
  11. Quantum support vector machine for big data classification. Physical Review Letters, 113(3):130503, Sep 2014.
  12. Power of data in quantum machine learning. Nature Communications, 12(1):1–9, 2021.
  13. Machine Learning with Quantum Computers. 2021.
  14. John Preskill. Quantum Computing in the NISQ era and beyond. Quantum, 2:79, Aug 2018.
  15. Quantum supremacy using a programmable superconducting processor. Nature, 574(7779):505–510, 2019.
  16. Quantum computational advantage using photons. Science, 370(6523):1460–1463, 2020.
  17. Noisy intermediate-scale quantum (nisq) algorithms. arXiv preprint arXiv:2101.08448, 2021.
  18. Variational quantum algorithms. Nature Reviews Physics, pages 1–29, aug 2021.
  19. Hybrid Quantum-Classical Algorithms and Quantum Error Mitigation. Journal of the Physical Society of Japan, 90(3):032001, mar 2021.
  20. A variational eigenvalue solver on a photonic quantum processor. Nature Communications, 5(1):4213, dec 2014.
  21. A Quantum Approximate Optimization Algorithm. arXiv:1411.4028, pages 1–16, Nov 2014.
  22. Supervised learning with quantum-enhanced feature spaces. Nature, 567(7747):209–212, Mar 2019.
  23. Circuit-centric quantum classifiers. Physical Review A, 101(3):032308, Mar 2020.
  24. Quantum circuit learning. Physical Review A, 98(3):032309, Sep 2018.
  25. When bert meets quantum temporal convolution learning for text classification in heterogeneous computing. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8602–8606. IEEE, 2022.
  26. Classical-to-quantum transfer learning for spoken command recognition based on quantum neural networks. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8627–8631. IEEE, 2022.
  27. A quantum kernel learning approach to acoustic modeling for spoken command recognition. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023.
  28. Parameterized quantum circuits as machine learning models. Quantum Science and Technology, 4(4):043001, Jun 2019.
  29. Classification with Quantum Neural Networks on Near Term Processors. arXiv:1802.06002, pages 1–21, Feb 2018.
  30. Power and limitations of single-qubit native quantum neural networks. In S Koyejo, S Mohamed, A Agarwal, D Belgrave, K Cho, and A Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 27810–27823. Curran Associates, Inc., 2022.
  31. Generalization in quantum machine learning from few training data. Nature Communications, 13(1):4919, aug 2022.
  32. Concentration of Data Encoding in Parameterized Quantum Circuits. In 36th Conference on Neural Information Processing Systems (NeurIPS 2022), 2022.
  33. Efficient measure for the expressivity of variational quantum algorithms. Physical Review Letters, 128(8):80506, 2022.
  34. Quantum machine learning beyond kernel methods. Nature Communications, 14(1):517, jan 2023.
  35. Optimal Quantum Dataset for Learning a Unitary Transformation. Physical Review Applied, 19(3):034017, mar 2023.
  36. Power of data in quantum machine learning. Nature Communications, 12(1):2631, dec 2021.
  37. Detecting and quantifying entanglement on near-term quantum devices. npj Quantum Information, 8(1):52, dec 2022.
  38. Practical distributed quantum information processing with LOCCNet. npj Quantum Information, 7(1):159, dec 2021.
  39. Recent Advances for Quantum Neural Networks in Generative Learning. arXiv:2206.03066, jun 2022.
  40. A hybrid quantum-classical Hamiltonian learning algorithm. SCIENCE CHINA-INFORMATION SCIENCES, 66(2), 2023.
  41. The power of quantum neural networks. Nature Computational Science, 1(6):403–409, jun 2021.
  42. Modeling term dependencies with quantum language models for IR. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval - SIGIR ’13, page 653, New York, New York, USA, 2013. ACM Press.
  43. End-to-End Quantum-Like Language Models with Application to Question Answering. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018.
  44. A quantum-inspired sentiment representation model for twitter sentiment analysis. Applied Intelligence, 49(8):3093–3108, 2019.
  45. Towards quantum language models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1840–1849, 2017.
  46. Quantum algorithms for compositional natural language processing. arXiv preprint arXiv:1608.01406, 2016.
  47. Quantum natural language processing on near-term quantum computers. arXiv preprint arXiv:2005.04147, 2020.
  48. Quantum language processing. arXiv preprint arXiv:1902.05162, 2019.
  49. Quantum long short-term memory. arXiv preprint arXiv:2009.01783, 2020.
  50. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  51. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 6000–6010, 2017.
  52. Beyond rnns: Positional self-attention with co-attention for video question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 8658–8665, 2019.
  53. Multi-scale self-attention for text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7847–7854, 2020.
  54. Attention-based quantum tomography. arXiv preprint arXiv:2006.12469, 2020.
  55. Qnlp in practice: Running compositional models of meaning on a quantum computer. arXiv preprint arXiv:2102.12846, 2021.
  56. Quantum Computation and Quantum Information. American Journal of Physics, 70(5):558–559, May 2002.
  57. Universal kernels. Journal of Machine Learning Research, 7(12), 2006.
  58. The dawn of quantum natural language processing. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8612–8616. IEEE, 2022.
  59. Theory of Point Estimation. Technometrics, 41(3):274, Aug 1999.
  60. Léon Bottou. Stochastic Learning. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 3176, pages 146–168. 2004.
  61. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org.
  62. A rigorous and robust quantum speed-up in supervised machine learning. Nature Physics, pages 1–5, 2021.
  63. Theoretical error performance analysis for variational quantum circuit based functional regression. npj Quantum Information, 9(1):4, 2023.
  64. Generalization in quantum machine learning from few training data. Nature communications, 13(1):4919, 2022.
  65. From group to individual labels using deep features. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 597–606, 2015.
  66. PaddlePaddle: An Open-Source Deep Learning Platform from Industrial Practice. Frontiers of Data and Domputing, 1(1):105–115, 2019.
  67. UCI machine learning repository, 2017.
  68. Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, Dec 2015.
Citations (31)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now