Tensor Decomposition Techniques

• Tensor Decomposition is a framework that generalizes matrix factorization to multi-dimensional arrays for extracting low-dimensional, interpretable structures.
• It comprises various models including CP, Tucker, Tensor Train, Tensor Ring, and Tensor Star, each with unique storage, scalability, and identifiability properties.
• Recent advances focus on overcoming challenges like rank selection, computational scalability, and integration with deep-network and quantum algorithms.

Tensor decomposition generalizes matrix factorization to higher-order arrays, providing a framework for extracting interpretable, low-dimensional structures from multiway data. A $d$-way tensor $$\mathcal{X}\in\mathbb{R}^{N_1\times N_2\times\cdots\times N_d}$$ is decomposed as a sum or product of lower-rank components; these decompositions underlie advanced methodologies in signal processing, statistics, machine learning, and computational biomedicine. Canonical Polyadic (CP), Tucker, and emerging forms such as Tensor Star (TS), Tensor Train (TT), and Tensor Ring (TR) decompositions encode various structural hypotheses, each with distinct storage, scaling, and identifiability properties. Recent algorithmic innovations address classical bottlenecks regarding rank selection, identifiability, scalability, and interpretability, including extensions to quantum computation, dictionary-structured models, and deep-network priors. This article reviews key formulations, computational methodologies, application domains, and ongoing research directions.

1. Mathematical Foundations and Principal Tensor Decomposition Models

Tensor decompositions seek to represent a tensor $\mathcal{X}$ as a sum or contraction of lower-rank terms. The two dominant classical frameworks are Tucker and CANDECOMP/PARAFAC (CP) decompositions (Burch et al., 18 Feb 2025):

• Tucker decomposition:

$$\mathcal{X} = \mathcal{G}\times_1 U^{(1)} \times_2 U^{(2)} \cdots\times_d U^{(d)}$$

where $$\mathcal{G}\in\mathbb{R}^{T_1\times\cdots\times T_d}$$ is the core tensor and $U^{(n)}\in\mathbb{R}^{N_n\times T_n}$ are factor matrices.

• CP (CANDECOMP/PARAFAC) decomposition:

$$\mathcal{X} \approx \sum_{r=1}^{R} \lambda_r\,a^{(1)}_r\circ a^{(2)}_r\circ\cdots\circ a^{(d)}_r$$

with factor vectors $a^{(n)}_r\in\mathbb{R}^{N_n}$ and component weights $\lambda_r$.

• Spiked tensor model (statistical context):

$T_0 = \lambda\,v_{\mathrm{sig}}^{\otimes p} + G$

where $v_{\mathrm{sig}}$ is a unit-norm signal, $\lambda$ the SNR, and $G$ noise.

Generalizations include:

• Tensor Train (TT): chain of 3-way cores, storage $\mathcal{O}(NdR^2)$ for uniform size $d$ and rank $R$.
• Tensor Ring (TR): cyclic connection of TT-cores, circular permutation invariance (Zhao et al., 2016).
• Tensor Star (TS): ring of 4-way core tensors with skip-connected 3-way factors, offering pairwise mode correlation with $\mathcal{O}(NIR^2+NR^4)$ storage (Zhou et al., 2024).

2. Algorithmic Frameworks and Computational Complexity

Decomposition methods rely on alternating least squares (ALS), spectral methods, or stochastic and randomized algorithms.

• ALS for CP/Tucker: Iteratively updates one set of factors by fixing others; costs per sweep scale as $\mathcal{O}(RdN^d)$ for CP and $\mathcal{O}(d\,N\,T + T^d)$ for Tucker (Burch et al., 18 Feb 2025).
• Randomized/sketched ALS: Projects tensor to subspaces for tractability in exascale settings (zhang et al., 2023, Erichson et al., 2017).
• Stochastic gradients for large-scale tensors: Allows unbiased gradient estimates and scaling to tensors with trillions of entries (Kolda et al., 2019).
• TS algorithms: Alternating update of factor and core tensors via contraction and least-squares; Proximal Alternating Minimization (PAM) handles missing data (Zhou et al., 2024).
• Spectral/joint diagonalization: Jennrich’s algorithm and higher-order SVD (HOSVD) yield efficient closed-form solutions under strong conditions (Vijayaraghavan, 2020, Wang et al., 2016, Turchetti, 2023).

Comparative Storage/Compute Complexity Table

Method	Storage Complexity	Time per Iteration	Scalability
CP	$\mathcal{O}(RdN)$	$\mathcal{O}(RN^d)$	Bottleneck at large $d$ or $N$
Tucker	$\mathcal{O}(dNT + T^d)$	$\mathcal{O}(dN^{d-1}T)$	Large $d$ not practical
TT/TR	$\mathcal{O}(NdR^2)$/$\mathcal{O}(NIR^2)$	Sequential/ALS, scalable	Efficient for high order
TS	$\mathcal{O}(NIR^2+NR^4)$	Dense normal equations, moderate $R$	Superior pairwise coupling

3. Identifiability, Uniqueness, and Rank Selection

CP decomposition is unique under mild factor Kruskal-rank conditions, allowing interpretable latent-component identification (Cichocki, 2013). Tucker and most tensor-network decompositions (TT, TR, TS) lack inherent uniqueness, requiring additional structure or constraints (e.g., sparsity, nonnegativity, domain knowledge).

Rank determination remains a major challenge; classical approaches include:

• Cross-validation and explained variance,
• Core-consistency diagnostics,
• Information/Bayesian criteria (e