Low Multilinear Rank Approximation

• LMLRA is a tensor approximation method defined via Tucker decomposition that captures low multilinear rank structure for compressing and analyzing high-dimensional data.
• Key computational methods such as HOSVD, ALS, and Riemannian optimization are employed to efficiently compute factor matrices and core tensors with quantifiable approximation errors.
• Applications span high-dimensional PDE solvers, quantum chemistry, and data analysis, addressing the 'curse of dimensionality' through scalable, structured tensor representations.

Low Multilinear Rank Approximation (LMLRA) is a class of tensor approximation methodologies where the objective is to represent a high-dimensional tensor using a compact Tucker structure, such that all mode-unfoldings have small matrix rank. LMLRA underpins much of modern tensor analysis and efficient high-dimensional computation in scientific computing, machine learning, signal processing, and numerical analysis. The essential principle is to exploit the multilinear (mode-wise) structure of data for compression, regularization, and computational tractability in settings where classical matrix methods are infeasible due to dimensionality.

1. Mathematical Formulation and Tucker Decomposition

LMLRA is fundamentally expressed via the Tucker decomposition. Given a tensor $$X \in \mathbb{R}^{n_1 \times n_2 \times \cdots \times n_d}$$, the decomposition takes the form

$$\mathrm{vec}(X) = (U_d \otimes U_{d-1} \otimes \cdots \otimes U_1) \mathrm{vec}(C),$$

where $U_\mu \in \mathbb{R}^{n_\mu \times r_\mu}$ are factor matrices with orthonormal columns, $C \in \mathbb{R}^{r_1 \times \cdots \times r_d}$ is the core tensor, and $\otimes$ denotes the Kronecker product. The tuple $(r_1, \ldots, r_d)$ is the multilinear rank. The mode-$\mu$ unfolding $X^{(\mu)}$ (or matricization) has rank $r_\mu = \mathrm{rank}(X^{(\mu)})$, i.e., every unfolding is well-approximated by a low-rank factorization.

Unlike the matrix case, the best low multilinear rank approximation is generally NP-hard and not always attainable by simple truncation, but the Tucker/HOSVD structure remains the central model.

2. Core Algorithms: HOSVD, ALS, and Manifold Optimization

The principal computational methods for LMLRA are:

• Higher-Order Singular Value Decomposition (HOSVD): Compute the $k_\mu$ leading left singular vectors of each mode-unfolding $X^{(\mu)}$ to obtain the factor matrices $U_\mu$. The core tensor is projected as $$\mathrm{vec}(C) = (U_d \otimes \cdots \otimes U_1)^T \mathrm{vec}(X)$$. The approximation error obeys

$$\|X - \tilde{X}\| \leq \sqrt{d} \cdot \min\{ \|X - Y\| : Y \in T(k_1, \ldots, k_d)\}$$

where $T(k_1, ..., k_d)$ is the set of tensors with multilinear rank at most $(k_1, ..., k_d)$.

• Alternating Least Squares (ALS): Iteratively update each factor matrix while keeping all others fixed; each update reduces to a matrix approximation or regression problem. ALS and variants are widely used for Tucker-based LMLRA and extended to manifold optimization approaches, leveraging the geometry of the set of fixed multilinear rank tensors.
• Newton-type and Riemannian Optimization: The smooth manifold structure of fixed-rank tensors is exploited using Newton-like algorithms or Riemannian gradient descent. Such methods offer superior convergence properties and have underpinned recent advances in efficient LMLRA for both dense and sparse, high-dimensional data.
• Iterative Truncation Methods: For linear algebraic problems such as large-scale PDEs or parameter-dependent linear systems, LMLRA is maintained during iterative solvers (e.g., Richardson iteration) by truncating the tensor after each step:

$X_{k+1} = T(X_k + \omega P (B - A(X_k))),$

with $T$ denoting truncation to prescribed rank, $P$ a preconditioner, and $\omega$ a relaxation parameter.

3. Computational Complexity and Scalability

The most prominent computational bottleneck is the cost of SVD on large mode-unfoldings (with each $X^{(\mu)}$ of size $n_\mu \times \prod_{\nu \ne \mu} n_\nu$), as well as the exponential scaling in storage if $d$ is large (the so-called "curse of dimensionality"). To address this:

• Randomized SVD algorithms and black box approximation schemes are employed to efficiently estimate leading singular subspaces.
• Structured methods such as tensor networks (e.g., Tensor Train, Hierarchical Tucker) mitigate exponential growth in the number of parameters for very high dimensions.
• Manifold-aware parallel and distributed ALS schemes allow for efficient computation on contemporary high-performance architectures.

Nevertheless, for systems with very high order $d$, research continues on scalable methods, particularly for truncation and storage management.

4. Practical Applications

LMLRA enables significant advances in applied mathematics and data science:

• High-Dimensional PDEs and Parameterized Problems: LMLRA compresses and manipulates tensors representing discretized functions, fundamental in uncertainty quantification and stochastic PDE simulation.
• Electronic Structure Calculations: Hartree–Fock and DFT computations benefit from low multilinear rank decompositions, exploiting separability in the underlying quantum states.
• Fast Multidimensional Integration and Convolution: Green's function approximations or convolutions in multidimensional domains with structured kernels are tractable via LMLRA.
• Data Analysis, Feature Extraction, and Machine Learning: Multimodal and temporally resolved data benefit from LMLRA, particularly in cases where smoothness or separability (not merely observed low rank) is inherent. In such cases, LMLRA supports compression, denoising, and interpretable feature extraction.

5. Advances and Theoretical Guarantees

Recent developments have expanded the theoretical and practical utility of LMLRA:

• Gradient-based approaches on manifolds have improved the robustness and efficiency of optimization-based solvers.
• Convergence theory for ALS and those iterative methods on fixed-rank manifolds is being refined, with error bounds and local/global convergence criteria extending foundational matrix results.
• Hybrid approaches combine Tucker LMLRA with tensor network methods to blend flexibility and scalability, adapting approximation format based on problem structure.
• Preconditioning strategies formulated within the low multilinear rank framework help constrain rank growth and improve convergence in iterative solvers.
• Quasi-optimality: Despite the lack of a true analog to the Eckart–Young theorem, the error of HOSVD-based LMLRA is always within a factor $\sqrt{d}$ of the best possible among all multilinear rank-constrained tensors.

6. Remaining Challenges

Key challenges remain in the widespread application and further development of LMLRA:

• Curse of dimensionality: As $d$ increases (especially for isotropic problems where all $n_\mu$ are large), even highly compressed representations can become intractable.
• Efficient truncation: Ensuring that after arithmetic or iterative updates, the rank remains controlled without substantial loss of accuracy.
• Black-box and stochastic approximation: Methods capable of "learning" the multilinear structure with partial or indirect access to the full tensor.
• Algorithmic stability: Constructing robust, stable methods for truncation and updating, especially within iterative or online contexts, remains an active area of research.

7. Summary Table of Key LMLRA Concepts

Principle	Mathematical Formulation	Computational Feature
Tucker Decomposition	$X = C \times_1 U_1 \cdots \times_d U_d$	Compression via multilinear ranks
Multilinear Rank	$(r_1, \ldots, r_d)$ where $r_\mu = \mathrm{rank}(X^{(\mu)})$	Controls mode-wise complexity
HOSVD Approximation	Truncate SVD on each unfolding, reconstruct core	Quasi-optimal, SVD-based
ALS/Riemannian Methods	Alternating optimization or manifold descent over factors	Improved convergence, adaptability
Randomized Algorithms	Subspace estimation from random projections	Faster for large-scale problems
Application Contexts	High-dim. PDEs, quantum chemistry, data analysis, integration	Compression/solving/feature extract

A detailed mathematical and algorithmic framework for LMLRA is essential for any numerical or data-intensive application involving multidimensional arrays, where storage, computation, and structural exploitation of data are critical to feasibility and performance (Grasedyck et al., 2013).