Tensor Rank Analysis

• Tensor rank is a measure of a tensor's complexity; key aspects include classical, multilinear, and border ranks.
• Rank determination involves sophisticated methods due to NP-hardness, impacting fields like quantum information and signal processing.
• Applications span robust PCA, quantum info, and combinatorial analysis, despite challenges in computational complexity and uniqueness.

Tensor rank analysis concerns the study and quantification of the complexity, structure, and representational power of multi-dimensional arrays (tensors) through various notions of “rank.” Unlike the well-understood case of matrices, tensor rank is a subtle, multifaceted concept with deep implications for data analysis, algebraic complexity, quantum information, signal processing, and numerical computation. Multiple formalizations and generalizations have emerged to accommodate the algebraic richness of higher-order arrays, their symmetries, and their diverse application domains.

1. Fundamental Notions of Tensor Rank

Classical tensor rank, or CP (CANDECOMP/PARAFAC) rank, generalizes matrix rank to higher orders. For a tensor $$\mathcal{X} \in \mathbb{K}^{n_1 \times n_2 \times \cdots \times n_d}$$ (typically $\mathbb{K} = \mathbb{R}$ or $\mathbb{C}$), the CP rank is the minimal $R$ such that

$$\mathcal{X} = \sum_{r=1}^R a_r^{(1)} \otimes a_r^{(2)} \otimes \cdots \otimes a_r^{(d)}$$

with $a_r^{(j)} \in \mathbb{K}^{n_j}$. This set of rank-$R$ tensors is not closed for $d \geq 3$, motivating the introduction of border rank: the smallest $r$ such that $\mathcal{X}$ is a limit of rank-$\mathbb{K} = \mathbb{R}$0 tensors. Still further notions include:

• Multilinear (Tucker) rank: the tuple $\mathbb{K} = \mathbb{R}$1 where each $\mathbb{K} = \mathbb{R}$2 is the rank of the mode-$\mathbb{K} = \mathbb{R}$3 matricization (unfolding).
• Tensor-Train (TT) rank: the tuple of matrix ranks arising from recursively partitioning the modes, e.g., $\mathbb{K} = \mathbb{R}$4.
• Tensor Network (TN) rank: bond dimensions associated to the edges of an arbitrary undirected graph $\mathbb{K} = \mathbb{R}$5, unifying and generalizing TT, Tucker, and other decompositions (Zhou et al., 14 Jul 2025, Ye et al., 2018).
• Border rank: the minimal $\mathbb{K} = \mathbb{R}$6 for which $\mathbb{K} = \mathbb{R}$7 lies in the Zariski closure of rank-$\mathbb{K} = \mathbb{R}$8 tensors (Allman et al., 2012, Bruzda et al., 2019).
• Specialized ranks (tubal, N-tubal, Q-rank): context-dependent extensions, e.g., for tensors recoverable via t-SVD or flexible mode-wise transforms (Zheng et al., 2018, Kong et al., 2019).
• Combinatorial and analytic ranks (slice, partition, analytic): designed for combinatorial, algebraic, or analytic settings, often over finite fields or in additive combinatorics (Juvekar et al., 2022, Lovett, 2018).

Each rank notion encodes distinct aspects of tensor complexity and can lead to large disparities in value for the same data object (Ye et al., 2018).

2. Methods for Rank Determination and Lower Bounds

The simplest algorithmic method for matrices—Gaussian elimination—fails for higher-order tensors due to the NP-hardness of rank computation for $\mathbb{K} = \mathbb{R}$9 (0802.2371, Calvi et al., 2019). For generic or random tensors of given dimensions, the “generic rank” is typically unique and can be estimated numerically via Jacobian rank conditions or algebraic geometry (0802.2371, Bruzda et al., 2019).

Lower bounds for tensor rank or its variants are often obtained by:

• Matrix unfoldings: The maximal rank over all possible matricizations provides a lower bound (and, for certain configurations, achieves the exact rank) (Calvi et al., 2019).
• Flattenings and generalized flattenings: Useful for border rank, including Young flattenings (Christandl et al., 2017).
• Maximally square unfolding: Choosing a matricization as close to square as possible often maximizes the attainable lower bound in practice (Calvi et al., 2019).
• Representation-theoretic and interpolation techniques: For explicit tensors such as group tensors and permutation tensors (Alexeev et al., 2011).
• Algebraic invariants: Detecting dense orbits under group actions via determinant and Hessian-type invariants (Allman et al., 2012).

Tightness of these bounds is ensured for generic tensors whenever the associated flattened matrix is not of full rank; otherwise, further structural or algebraic arguments are needed.

3. Axiomatic and Graph-Based Generalizations

To systematize rank notions, (Qi et al., 2020) proposes a set of axioms:

• Normalization, Identity, Matrix Consistency, Homogeneity, Permutation Invariance, and Monotonicity under subtensors.

Proper tensor rank functions are required to never exceed any mode dimension on cubic tensors. Max-Tucker rank and its closure fulfill these requirements and encapsulate matrix-theoretic properties like the max-full-rank-submatrix property for matrices. Submax-Tucker rank, defined as the second largest mode-unfolding rank, provides additional flexibility in modeling and is effective for compression in large-scale data (Qi et al., 2020). These formalizations establish a partial order on all possible tensor rank functions, admitting a unique smallest rank by pointwise comparison.

Tensor network (G-)ranks allow tailoring the notion of rank to the topology of a computational or physical network, dramatically reducing the effective complexity in many practical settings. For an undirected graph $\mathbb{C}$0, rank$\mathbb{C}$1 is defined as the tuple of minimal bond dimensions that allow a contraction matching the tensor, which may be exponentially smaller than the CP or multilinear ranks for appropriately chosen $\mathbb{C}$2 (Ye et al., 2018).

4. Rank Adaptation, Learning, and Function Spaces

Continuous-parameter or data-driven adaptations of rank arise in both computational and applied contexts:

• Functional tensor rank: For continuous-indexed (functional) tensors, rank is defined as the minimal $\mathbb{C}$3 such that the (possibly infinite-dimensional) data admit a CP-type decomposition into separable functions of each argument (Chen et al., 10 Feb 2025). Modern approaches such as GreT parameterize these functions via neural ODEs on Fourier features, with sparsity-inducing priors to enable automatic adaptation of effective rank through variational inference.
• Complexity-adaptive models: Automatic relevance determination (ARD) mechanisms prune unnecessary factors by driving their associated precisions $\mathbb{C}$4 to large values, which can be monitored via posterior means or component powers (Chen et al., 10 Feb 2025).
• Multimode and pairwise correlation ranks (N-tubal): Unfold the tensor into all pairs of modes, with tubal rank along each unfolding serving as a vector-valued "N-tubal rank." Convex relaxations (WSTNN) lead to tractable low-rank regularization problems with robust empirical performance (Zheng et al., 2018).
• Coordinate flow methods: Find linear (or nonlinear) transforms of variables that yield tensors of reduced rank in function or data representation (e.g., for PDE solution tensors), via Riemannian optimization on the Stiefel or special linear manifold (Dektor et al., 2022).
• Data-adaptive spectral norms (Q-rank): By learning an orthogonal basis for one mode, tensors can be aligned such that most frontal slices become low-rank, enhancing the effectiveness of nuclear norm proxies for low-rank recovery under complex or non-smooth data (Kong et al., 2019).

5. Applications, Uniqueness, and Computational Complexity

Tensor rank analysis underpins algorithms for:

• Low-rank recovery, completion, and robust PCA: By choosing an appropriate rank definition and convex surrogate (e.g., nuclear norm, WSTNN, Q-nuclear norm), one can design ADMM-type solvers for high-dimensional data with provable convergence and robust empirical accuracy (Zhong et al., 2013, Zheng et al., 2018, Kong et al., 2019).
• Algebraic complexity theory: The tensor rank of the matrix multiplication tensor directly determines the exponent $\mathbb{C}$5 of matrix multiplication, with decompositions like Strassen’s providing key upper bounds (Ottaviani et al., 2020, Christandl et al., 2016). Tensor surgery leverages known low-rank decompositions to produce upper bounds for structured tensors associated to classes of graphs (Christandl et al., 2016).
• Quantum information: Tensor rank and its variants serve as entanglement measures for multipartite pure states, with border rank delineating closure properties of SLOCC orbits. Norm inequalities establish quantitative connections between rank and entanglement (Bruzda et al., 2019).
• Combinatorics and finite field theory: Slice, partition, and analytic ranks capture combinatorial complexity of structures such as cap-sets and free sets, displaying asymptotic equivalence over large finite fields (Juvekar et al., 2022, Lovett, 2018).

Uniqueness of decomposition is guaranteed for the matrix case and under sufficiently strong Kruskal-type conditions for CP decomposition in higher orders. However, for most tensors, especially real ones, non-uniqueness is generic unless tensor and decomposition dimensions meet strict identifiability criteria (0802.2371, Bruzda et al., 2019). The computational complexity of rank determination is NP-hard, and even approximate computation admits no known polynomial-time algorithms for $\mathbb{C}$6 (0802.2371, Juvekar et al., 2022).

6. Structural Properties and Limit Phenomena

Key structural and asymptotic features of tensor rank include:

• Non-multiplicativity: Tensor rank is submultiplicative but not multiplicative under the tensor product; if border rank is strictly smaller than rank, there exist powers $\mathbb{C}$7 for which $\mathbb{C}$8 (Christandl et al., 2017).
• Border and symmetric ranks: For many symmetric tensors, the symmetric (Waring) rank may differ from the ordinary rank, with Comon's conjecture holding in many settings but failing for sufficiently high-order or dimension (Ottaviani et al., 2020, Bruzda et al., 2019).
• Invariant theory and algebraic geometry: Polynomial invariants (e.g., Hessians, hyperdeterminants) precisely characterize the orbits or components of tensors of fixed rank, and their vanishing or nonvanishing serves to stratify tensor varieties (Allman et al., 2012).
• Typical and generic ranks: Over $\mathbb{C}$9, almost all tensors have a fixed generic rank; over $R$0, multiple typical ranks may coexist, whose enumeration and geometric structure remains a challenging open problem (0802.2371, Bruzda et al., 2019).

7. Practical Considerations, Tradeoffs, and Outstanding Problems

Practical application of tensor rank analysis requires:

• Balancing expressivity and parsimony: Larger ranks yield greater representational power but risk overfitting, while too-small ranks can underfit or oversimplify mode-wise dependencies (Zhou et al., 14 Jul 2025).
• Mode-wise heterogeneity: Tucker or TT ranks enable fine-grained control when different modes exhibit different intrinsic dimensions, as in high-resolution image or video data (Zhou et al., 14 Jul 2025).
• Rank selection heuristics: Empirical singular value spectra, domain knowledge, cross-validation, and ARD-inspired sparsity priors inform the effective choice and tuning of rank parameters (Chen et al., 10 Feb 2025, Zhou et al., 14 Jul 2025).
• Algorithmic scalability: Alternating optimization, truncated SVDs, and efficient coordinate-updating schemes, especially in TT or Tucker network formats, address complexity barriers for large-scale or high-dimensional data (Franc, 2022, Dektor et al., 2022, Kong et al., 2019).
• Open problems: Full classification of typical or generic ranks, characterizing identifiability regimes, understanding essential differences between combinatorial and algebraic ranks, and extending equivalences between analytic/slice/partition ranks to higher orders and other fields remain central research challenges (Juvekar et al., 2022, 0802.2371, Bruzda et al., 2019, Ottaviani et al., 2020).

Tensor rank analysis thus provides a rigorous and adaptable framework for understanding multidimensional interactions in both mathematical theory and a broad array of scientific domains. Its evolution continues to be shaped by demands for theoretical precision, algorithmic efficiency, and practical utility across disciplines.