Entangling Machine Learning with Quantum Tensor Networks (2403.12969v1)
Abstract: This paper examines the use of tensor networks, which can efficiently represent high-dimensional quantum states, in LLMing. It is a distillation and continuation of the work done in (van der Poel, 2023). To do so, we will abstract the problem down to modeling Motzkin spin chains, which exhibit long-range correlations reminiscent of those found in language. The Matrix Product State (MPS), also known as the tensor train, has a bond dimension which scales as the length of the sequence it models. To combat this, we use the factored core MPS, whose bond dimension scales sub-linearly. We find that the tensor models reach near perfect classifying ability, and maintain a stable level of performance as the number of valid training examples is decreased.
- Exact holographic tensor networks for the motzkin spin chain. Quantum, 5:546.
- Tai-Danae Bradley. 2020. At the interface of algebra and statistics.
- Modeling sequences with quantum states: a look under the hood. Machine Learning: Science and Technology, 1(3):035008.
- Criticality without frustration for quantum spin-1 chains. Physical Review Letters, 109(20).
- Wikimedia Commons. An interpretation of motzkin numbers, the 9 paths from (0, 0) to (4, 0) using only steps northeast, east, and southeast, never dipping below the y-axis. [online]. 2021.
- Robert Donaghey and Louis W Shapiro. 1977. Motzkin numbers. Journal of Combinatorial Theory, Series A, 23(3):291–301.
- Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366.
- The relationship between data skewness and accuracy of artificial neural network predictive model. IOP Conference Series: Materials Science and Engineering, 523(1):012070.
- Henry W. Lin and Max Tegmark. 2017. Critical behavior in physics and probabilistic formal languages. Entropy, 19(7).
- Machine learning by unitary tensor network of hierarchical tree structure. New Journal of Physics, 21(7):073059.
- Entanglement and tensor networks for supervised image classification.
- Tensor networks for probabilistic sequence modeling.
- Exponential machines.
- Language as a matrix product state. ArXiv, abs/1711.01416.
- Ulrich Schollwöck. 2011. The density-matrix renormalization group in the age of matrix product states. Annals of Physics, 326(1):96–192.
- James Stokes and John Terilla. 2019. Probabilistic modeling with matrix product states. Entropy, 21(12):1236.
- E. Miles Stoudenmire and David J. Schwab. 2017. Supervised learning with quantum-inspired tensor networks.
- Edwin Stoudenmire and David J Schwab. 2016. Supervised learning with tensor networks. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc.
- Explainable natural language processing with matrix product states. New Journal of Physics, 24(5):053032.
- Constantijn van der Poel. 2023. A quantum approach to language modeling.