Quantum-inspired tensor networks in machine learning models

Abstract: Tensor networks were developed in the context of many-body physics as compressed representations of multiparticle quantum states. These representations mitigate the exponential complexity of many-body systems by capturing only the most relevant dependencies. Due to the formal similarity between quantum entanglement and statistical correlations, tensor networks have recently been integrated in machine learning, operating both as alternative learning architectures and as decompositions of components of neural networks. The expectation is that the theoretical understanding of tensor networks developed within quantum many-body physics leads to novel methods that offer advantages in terms of computational efficiency, explainability, or privacy. Here we review the use of tensor networks in the context of machine learning, providing a critical assessment of the state of the art, the potential advantages, and the challenges that must be overcome.

• The paper reviews 181 studies and distinguishes native tensor network models from tensorized neural networks used to compress conventional architectures.
• Tensor networks deliver practical compression and interpretability benefits, including a 441× parameter reduction for a TT embedding with a 1% accuracy gain and exact SHAP computation for some architectures.
• The review identifies nonlinear activations, hardware mismatch, weak large-scale benchmarking, and missing privacy and interpretability standards as the main barriers to wider adoption.

"Quantum-inspired tensor networks in machine learning models" (2604.14287) is a critical review of how tensor networks (TNs), originally developed for simulating many-body quantum systems, have been adopted in ML. The authors—Valverde, García Olaizola, Scarpa, and Pozas-Kerstjens—organize the literature into two distinct paradigms: native tensor network architectures (Tensor Neural Networks, TNNs) used directly as learning models, and tensorized neural networks, in which TNs compress components of conventional deep learning models. Beyond cataloguing applications, the review offers a pointed assessment: TNs provide provable advantages in efficiency, interpretability, and privacy, yet remain absent from state-of-the-art AI practice. The authors attribute this gap not to fundamental theoretical limitations but to the sociological structure of a field historically led by physicists and mathematicians who prioritized formal understanding over software usability and industrial validation.

The review is grounded in a systematic bibliographic analysis: starting from 1,071 articles retrieved from OpenAlex, filtering reduced the corpus to 181 high-quality publications, which the authors organize into an interactive topic–author graph covering supervised learning, quantum ML, deep learning, unsupervised learning, computer vision, generative modeling, and NLP.

The paper first establishes the technical vocabulary: Penrose graphical notation, reshaping, contraction, and singular value decomposition (SVD). Tensorization—the conversion of an arbitrary tensor into a linear chain of three-legged tensors via successive reshapings and SVDs—is presented as the core mechanism underlying both compression and native TN modeling. Truncating bond dimensions after each SVD yields the best fixed-rank approximation in Fröbenius norm locally (by the Eckart–Young theorem), but the authors are careful to note that local optimality does not guarantee a globally optimal low-rank approximation; iterative sweeps or variational methods such as DMRG and Riemannian optimization are required to improve global fidelity. They also flag a subtlety often glossed over: reshaping destroys spatial or temporal index structure (e.g., flattening an image breaks row adjacency), so reshape choices directly affect compression capacity, motivating newer tensor-algebra approaches that operate on multidimensional data directly.

A distinctive section covers gauge freedom: because TNs are built from scalar products, inserting $GG^{-1} = \mathbb{I}$ on any virtual bond leaves the global tensor invariant while transforming local tensors. Canonical forms (left-, right-, mixed-canonical) exploit this redundancy to simplify contractions; tree-structured networks admit exact canonicalization, whereas looped architectures such as PEPS only admit quasi-canonical forms approximated via belief propagation. Crucially, the authors connect gauge freedom to privacy: since equivalent parameterizations of the same model exist, redundant degrees of freedom can be randomized to reduce parameter-side information leakage—a structural protection against extraction attacks that requires no noise injection, unlike differential privacy.

The review then surveys the main architecture families:

Architecture	Structure	Key property	Cost note
MPS / Tensor Train	linear chain	efficient scalar products, $\mathcal{O}(N\chi^3 d)$ contraction	exact canonicalization
MPO	chain with input/output legs	represents operators; layer tensorization	MPO–MPS application scales as $\mathcal{O}(N\chi^6 d)$
PEPS	2D lattice	area-law correlations, suited to images	exact contraction is computationally hard
TTN	hierarchical tree	multiscale features, tractable probabilities	less expressive than looped TNs
MERA	TTN + disentanglers	long-range, scale-invariant correlations	efficient contraction retained

The paper also bridges the two historical lineages—quantum information (White, Vidal) and numerical analysis (Oseledets, Hackbusch)—noting that the TT decomposition is algebraically equivalent to MPS and hierarchical Tucker to TTNs, with CP and Tucker decompositions completing the standard toolkit.

The most mature application is compression of neural network components. Dense layers, convolutional filters (e.g., in ResNet-18), recurrent networks, Transformer attention via Tucker/CP decompositions, and embedding layers have all been tensorized, with reported compression rates ranging from a factor of seven (the original dense-layer tensorization) to a factor of forty in RNNs. In NLP, the results are particularly concrete: a TT-embedding achieved a 441× parameter reduction on IMDB sentiment classification while increasing accuracy by 1%, and a 15× compressed TT-embedding on WMT'14 English–German translation cost only 0.3 BLEU. A recent TT-based adaptation of LoRA extends these ideas to parameter-efficient fine-tuning. TN methods are also compatible with pruning and quantization, enabling higher combined compression rates.

Beyond model compression, the review documents dataset-level applications: image and hyperspectral data compression, distributed TT computation across nodes, and incremental/streaming TT updates for dynamic data. However, the authors state plainly that tensorizing a pre-trained network remains "neither systematic nor robust": it demands architectural knowledge, careful rank selection, and post-compression fine-tuning to recover accuracy. They characterize fine-tuning as "a workaround, not a solution," and identify task-informed surgical compression—using criteria richer than matrix variance—as an open avenue.

This section contains the review's most distinctive claims. On interpretability, hierarchical TNs expose intrinsic complexity measures—von Neumann entropy and quantum mutual information—that quantify what the model has learned, going beyond post-hoc explanation techniques. A notable theoretical result cited is that SHAP values, NP-hard to compute exactly for general neural networks, can be computed exactly for TN architectures and in polylogarithmic time for TT structures. On privacy, gauge freedom enables model obfuscation intrinsic to the architecture, applicable either to natively trained TNs or to tensorized pre-trained networks, with reported preservation (and in some cases improvement) of predictive performance. Applications in healthcare—explainable ensembles for Alzheimer's disease prediction from brain structure, tensorized GANs with high-order pooling—are highlighted as domains where interpretability and privacy are regulatory requirements rather than preferences.

The authors' summary claim here is strong: TNs are "among the few architectures that simultaneously combine efficiency, interpretability, and privacy." They immediately qualify it, however: no standard metrics or protocols exist for evaluating the degree of interpretability, and empirical validation in complex real-world models remains limited. They also stress a distinction the literature frequently overlooks: gauge-based privacy guarantees apply to TN-native architectures, not necessarily to a posteriori tensorizations—an omission with real consequences in regulated sectors.

The review establishes theoretical bridges between TNs and conventional deep learning: CNNs and RNNs admit equivalent TN representations exhibiting volume-law entanglement scaling, and TNs correspond naturally to probabilistic graphical models. In computer vision, tensorized convolutions, TT-RNNs for video summarization, quantized TT networks for 3D recognition, and MERA-based anomaly detection in hyperspectral imagery are surveyed. In supervised learning—the largest application category in their bibliographic analysis—the foundational result is the MPS classifier trained with DMRG-style sweeps, alongside Exponential Machines, which implicitly represent feature interactions up to arbitrarily high order via the TT format. Representative benchmarks include a TTN classifier reaching 89.9% accuracy on Fashion-MNIST with mostly unsupervised training, and PEPS classifiers matching feed-forward networks with significantly fewer parameters while outperforming tree-like TNs on MNIST and Fashion-MNIST.

The recurring caveat is blunt: evaluations remain dominated by small datasets such as MNIST and Fashion-MNIST, and benchmarks against modern large-scale models are scarce. Until that gap closes, the practical competitiveness of TN-based supervised models cannot be established.

In unsupervised settings, Born machines represent distributions via squared wavefunction amplitudes, with the advantage that two-site DMRG allows the bond dimension—and hence capacity—to grow adaptively during training, unlike classical generative models whose architecture is fixed beforehand. Provable separations in expressive power between TN formats exist, and extensions to continuous data and diffusion processes broaden the scope. Anomaly detection spans particle-collision events, hyperspectral change detection, and unsupervised compressed sensing. Again, the authors concede that most empirical results are confined to small or domain-specific datasets, comparisons with modern deep generative models are scarce, and clustering/semi-supervised applications remain underdeveloped.

In quantum ML, TNs serve as the state-of-the-art classical simulator of quantum computers and enable a class of fully classical "quantum-inspired" algorithms with non-unitary factorizations; locally purified states are provably strictly more expressive than their unitary components, with unbounded resource separations. TN pretraining of parameterized quantum circuits accelerates convergence and mitigates barren plateaus, and isometric TN architectures have been shown to avoid exponentially vanishing gradients even at scale. The authors argue TNs constitute an autonomous research program in quantum ML rather than a mere bridge to future hardware.

The review catalogs the tooling landscape: physics-rooted libraries (ITensor, TeNPy), the now-obsolete Google TensorNetwork, and the two currently mature options—TensorLy (tensor decompositions, multi-backend) and TensorKrowch (native TN models on PyTorch, enabling hybrid TN–NN training). Specialized tools (TedNet, tntorch) and quantum-oriented stacks (Quimb, PennyLane, Qiskit's MPS backend, cuTensorNet) round out the survey. A practical point emphasized throughout: optimal contraction path selection is NP-hard, and suboptimal paths introduce exponential memory overhead, making heuristic tools like opt_einsum essential.

The discussion section is unusually candid about why TNs have not achieved mainstream adoption despite their formal advantages:

• Nonlinearity: activation functions in deep learning are applied point-wise, forcing reconstruction of dense tensors before application whenever TN layers are present. Finding a tensor representation of nonlinearities is identified as likely the single most important open technical problem.
• Hardware mismatch: GPUs and TPUs are optimized for large dense matrix multiplications, not the many small structured contractions TNs require; the theoretical computational advantage does not translate into inference-time reductions unless bond dimensions are very low or contractions batch efficiently. Realizing the advantage may require FPGAs or ASICs.
• Compression criteria: local SVD retains high-variance directions, but variance is not necessarily the task-relevant criterion for downstream performance.
• Evaluation gaps: small-scale benchmarks dominate; interpretability lacks standardized metrics; privacy guarantees do not automatically transfer to tensorized networks.

The authors frame the field's position explicitly: TNs today resemble neural networks before GPUs and backpropagation—solid theory, missing infrastructure. Whether they consolidate as a general-purpose tool or remain marginal depends on abstraction-friendly libraries, specialized hardware, and cross-community collaboration.

This review delivers a rigorous, dual-track account of tensor networks in machine learning, cleanly separating native TN architectures from tensorization-based compression and grounding its claims in a curated corpus of 181 publications. Its strongest quantitative evidence lies in compression (up to 441× parameter reduction with accuracy gains in embedding layers) and in structural guarantees unavailable to conventional architectures (exact polylogarithmic SHAP computation, gauge-freedom-based privacy). Its most valuable contribution, however, is diagnostic: it identifies precisely which obstacles—nonlinearities, hardware, evaluation standards, and community structure—stand between proven theoretical advantages and practical adoption, and states plainly which current claims rest on small-scale evidence.