Tensor Networks: Core Concepts

Updated 1 January 2026

Tensor Networks are graph-based decompositions that break high-order tensors into networks of low-order core tensors with controlled bond dimensions.
They enable efficient simulation and compression of exponential-size data by capturing essential entanglement and structural information.
They have diverse applications in quantum physics, machine learning, and optimization, balancing expressivity with computational tractability.

A tensor network is a graph-based decomposition of a high-order tensor into a network of smaller, low-order tensors, whose indices (or “legs”) are contracted or left open according to the connectivity of the network. Originating in quantum many-body physics for variational representations of entangled states, tensor networks have developed into a foundational multilinear-algebraic formalism for the compression, simulation, and optimization of exponential-size data objects across physics, machine learning, signal processing, optimization, and scientific computing (Sengupta et al., 2022). The principal motivation is expressivity versus computational tractability: by encoding only those components of the full tensor that respect low-entanglement structure (quantified by bond dimension), tensor networks interpolate between exact representations and highly compressed approximations, with complexity controlled by the network topology and core tensor ranks.

1. Mathematical Structure and Graphical Language

A rank- $d$ tensor $T\in\mathbb{R}^{n_1\times \cdots \times n_d}$ associates a multidimensional array to a multilinear map $T: V_1\times\cdots\times V_d \to \mathbb{R}$ , where $V_k$ is a vector space of dimension $n_k$ (Sengupta et al., 2022, Biamonte et al., 2017). In a tensor network (TN), $T$ is expressed as

$T_{i_1\cdots i_d} \approx \sum_{\{\alpha\}} \prod_{v \in V} A^{(v)}_{\{\text{physical}, \text{virtual}\}}$

where each $A^{(v)}$ is a low-order core tensor (typically order-2 or 3), with “physical” indices $i_k$ and auxiliary “virtual” indices $\alpha$ known as bond indices. Contracting shared virtual indices implements the network topology. Graphically, a tensor is a node with $r$ legs (edges representing indices). Connections represent contractions, and open (dangling) legs are the output indices of the network (Milsted et al., 2019, Biamonte et al., 2017, Roberts et al., 2019).

Tensor contraction generalizes matrix multiplication: $C_{jk} = \sum_{i} A_{ji} B_{ik}$ is represented as two nodes $A$ , $B$ joined by an edge labeled $i$ .

The “bond dimension” ( $D$ or $\chi$ ) is the range of the contracted virtual index, controlling the entanglement capacity and the accuracy/compression of the TN representation (Sengupta et al., 2022, Milsted et al., 2019).

2. Principal Tensor Network Architectures

Tensor network classes correspond to network topologies designed for different data or state structures (Silvi, 2012, Sengupta et al., 2022, Berezutskii et al., 11 Mar 2025):

Matrix Product States (MPS) / Tensor Train (TT):

A 1D chain: each core $A^{(k)}$ is $D_{k-1} \times n_k \times D_k$ , with open or periodic boundary. MPS efficiently capture “area law” entanglement in 1D systems, with contraction complexity $O(N D^3 n)$ (Cichocki, 2014).

Projected Entangled Pair States (PEPS):

A 2D (or higher) grid: each core has four virtual legs (neighbors) and one physical leg. Captures area-law states in higher dimensions; exact contraction is $\#$ P-hard, but approximate methods exist (cost scales as $O(L D^{10})$ in 2D) (Sengupta et al., 2022, Berezutskii et al., 11 Mar 2025).

Tree Tensor Networks (TTN):

Arranged as a hierarchical binary (or $k$ -ary) tree: each isometry $w$ maps two child bonds to one parent. TTNs efficiently encode logarithmic entanglement scaling in critical systems (cost $O(N D^4)$ ) (Milsted et al., 2019, Silvi, 2012).

Multiscale Entanglement Renormalization Ansatz (MERA):

Layered, scale-invariant networks with both isometries and disentanglers, designed to capture critical behavior and renormalization flows. Contraction cost is polynomial (in log $N$ layers), typical in quantum physics (Silvi, 2012, Rieser et al., 2023).

Other architectures: Tensor Ring (periodic TTN), MPO (operator-valued networks), hierarchical Tucker, block-term, and problem-specific grids.

Expressivity increases with bond dimension but so does contraction and optimization cost; the scaling is architecture-dependent ( $O(D^3)$ for MPS/TTN, $O(D^4)$ – $O(D^{10})$ for PEPS) (Cichocki, 2014, Sengupta et al., 2022).

3. Algorithms for Decomposition, Optimization, and Contraction

Fundamental TN algorithms employ sequential and block-wise factorization (Phan et al., 2016, Cichocki, 2014, Milsted et al., 2019):

Sequential SVD (TT-SVD):

Unfold tensor across successive modes, perform (truncated) SVDs with controlled error $\varepsilon$ , re-shape $U$ factors as cores. Pseudocode in (Cichocki, 2014):

# TT-SVD sketch
for n = 1 to N-1:
    [U,S,V] = truncated_SVD(Q)
    G[n] = reshape(U)
    Q = S @ V.T

Alternating Least Squares (ALS):

Sweep over cores, locally optimize while fixing others. Converges to stationary points when cost is continuous and bounded (Phan et al., 2016).

Variational Optimization (DMRG-style):

Used in physics for ground-state searches. Compute local “environment” by contracting the full network except one core; optimize via SVD/truncated eigenproblem under isometric constraints (Milsted et al., 2019, Silvi, 2012).

Gradient-based Training:

Treat all core tensors as parameters, enable automatic differentiation (AD) through the computational graph for end-to-end learning, typically feasible in frameworks supporting differentiable programming (Liao et al., 2019, Roberts et al., 2019).

Contraction Ordering:

Determining the optimal contraction sequence is NP-hard for arbitrary TN topologies (Sengupta et al., 2022). Heuristics (greedy, treewidth, dynamic programming) are employed for efficiency.

4. Applications across Quantum Physics, Machine Learning, and Optimization

Tensor networks are pervasive in diverse research domains (Berezutskii et al., 11 Mar 2025, Sengupta et al., 2022, Roberts et al., 2019, Selvan et al., 2020):

Quantum Many-Body Physics:
- Variational ground-state representation for spin chains, lattices, statistical and quantum field models (Milsted et al., 2019).
- Simulation of quantum circuits, time evolution (TEBD), estimation of entanglement spectra, and critical exponents (Berezutskii et al., 11 Mar 2025, Silvi, 2012).
Machine Learning:
- MPS-based classifiers for images, tabular, and sequence data. Feature maps (e.g., $\phi(x_p) = [\cos(\pi x_p/2), \sin(\pi x_p/2)]$ ) embed classical data into exponentially large spaces, then contract through TNs with polynomial parameter counts (Selvan et al., 2020, Sengupta et al., 2022).
- Hybrid tensor network–neural network architectures for memory, compression, and fusion (Tensorial Neural Networks, TNNs) (Wang et al., 2023).
- Tensor networks for explainable anomaly detection, offering interpretable marginal/conditional statistics via reduced density matrices (Hohenfeld et al., 6 May 2025).
- Multi-modal data processing with graph and sequence structure (e.g., Multi-Graph Tensor Networks) (Xu et al., 2021).
Big Data Analytics and Optimization:
- TT/QTT decompositions for supercompression, regression, and principal component analysis (Cichocki, 2014).
- Low-rank approximations for large linear systems, generalized eigenvalue problems, and penalized CCA (Cichocki, 2014).
Quantum Machine Learning:
- TN architectures mapped to quantum circuits (MPS→sequential circuits, TTN→tree-like circuits, MERA→layered circuits), enabling hardware-efficient implementations, gradient estimation, and error mitigation in quantum variational algorithms (Rieser et al., 2023).

5. Computational Complexity, Expressivity, and Entanglement

The complexity and representational power of tensor networks are tightly linked to the bond dimension and network topology:

Expressivity: The bond dimension $D$ determines the amount of correlation/entanglement representable; MPS/TTN are limited to area-law scaling ( $S\le \log D$ ), PEPS and MERA capture higher/larger regions and critical behavior (Sengupta et al., 2022, Silvi, 2012).
Approximation Error: Controlled by SVD truncation; for TT-SVD, truncating singular values incurs $\|X - \hat{X}\|_F \le \sqrt{\sum_n \varepsilon_n^2}$ (Cichocki, 2014).
Compression: TNs reduce $I^N$ parameters to $O(N I R^2)$ for TT/QTT or $O(L^2 n D^4)$ for PEPS. Quantization (QTT) enables “super-compression,” e.g., rank-3 TT for $2^{20}$ points with $<10^{-6}$ error (Cichocki, 2014).
Contractibility and Scaling:
- 1D/Trees: Polynomial cost in $D$ ( $O(N D^3)$ ).
- Loopy Graphs (2D or higher): Contraction is $\#$ P-hard; approximate methods, variational Monte Carlo, and tensor network functions sidestep the need for exact contraction, preserving variational property (Liu et al., 2024).

6. Software Implementations and Computational Tools

Practical TN research relies on specialized libraries and frameworks (Roberts et al., 2019, Wang et al., 2023):

TensorNetwork (Python, TensorFlow): General graphical API for building, contracting, and optimizing arbitrary TNs with automatic differentiation, and support for CPUs, GPUs, and TPUs (Roberts et al., 2019, Milsted et al., 2019).
TensorLy (Python): CP, Tucker, TT/TR decompositions; multi-backend support (Wang et al., 2023).
T3F (TensorFlow): Efficient TT decomposition; GPU acceleration.
ITensor (C++): High-performance TN computations for physics applications.
torchmps, TT-Toolbox, Scikit-TT: Specialized TT/MPS tools (MATLAB/Python/PyTorch).
TedNet, TensorLy-Torch: Drop-in tensorized layers for CNNs, RNNs, Transformers.
Quantum Simulators: PennyLane, Qiskit, Cirq, Yao.jl, TeD-Q—mapping TN architectures to parameterized quantum circuits for hardware deployment (Rieser et al., 2023).

7. Advances, Challenges, and Outlook

Current research fronts include:

Efficient Contraction Algorithms: Hyper-optimized contraction tree finding, automated TN rewriting for computational and combinatorial inference (Berezutskii et al., 11 Mar 2025, Sengupta et al., 2022).
Expressivity versus Complexity: Development of tensor network functions (TNFs) as generalized computational graphs enabling strict variational bounds and efficient neural-network mapping, even on loopy graphs with small bond dimension (Liu et al., 2024).
Adaptivity and Automated Rank Selection: Bayesian, reinforcement learning, and evolutionary algorithms for dynamic bond-rank tuning and adaptive compression (Wang et al., 2023).
Quantum–Classical Hybrid Integration: Pre-training PQC parameters with classical TNs, hybrid circuit-TN approaches for error correction and simulation (Rieser et al., 2023).
Explainability and Structure Learning: Marginal, conditional, and entropic analysis via reduced density matrices for interpretable machine learning and anomaly detection (Hohenfeld et al., 6 May 2025).
Domain-Specific Architectures: Custom TN constructions for chemistry, spatio-temporal modeling, and multi-modal learning exploiting problem symmetries (Wang et al., 2023, Xu et al., 2021).

Tensor networks remain a centerpiece of research in high-dimensional data modeling, quantum simulation, and the development of scalable, interpretable, and hardware-efficient algorithms across physical and computational sciences (Berezutskii et al., 11 Mar 2025, Sengupta et al., 2022, Wang et al., 2023).