Low-Rank Tucker Decomposition

• Low-Rank Tucker Decomposition is a method for representing multi-dimensional tensors through a compact core tensor and low-dimensional factors, capturing both global correlations and local structures.
• It employs adaptive regularization approaches, including weighted nuclear norms and sparsity penalties, to enhance robustness in tensor completion and recovery tasks.
• Modern optimization strategies, such as PALM, proximal methods, and randomized algorithms, significantly improve the scalability and accuracy of Tucker decomposition in large-scale applications.

Low-rank Tucker decomposition is a foundational model for representing multi-dimensional tensors via a small core tensor and low-dimensional mode-wise factors, enabling joint modeling of global correlation and local structure in high-dimensional data. In contemporary research, low-rank Tucker frameworks are extended by adaptive regularization mechanisms, scalable randomized solvers, and statistical modeling approaches that significantly impact completion, recovery, and analysis tasks in scientific computing, computer vision, and signal processing.

1. Mathematical Formulation and Tucker Rank

A given tensor $$\mathcal{X} \in \mathbb{R}^{I_1 \times \cdots \times I_N}$$ admits a Tucker factorization: $$\mathcal{X} \approx \mathcal{G} \times_1 A^{(1)} \times_2 A^{(2)} \cdots \times_N A^{(N)}$$ where $\mathcal{G}$ is the core tensor ($r_1 \times \cdots \times r_N$), and each factor $A^{(n)}$ ($I_n\times r_n$) encodes the subspace for mode $n$. The Tucker rank of $\mathcal{X}$ is the tuple $(r_1,\dots,r_N)$ with $r_n = \mathrm{rank}(\mathcal{X}_{(n)})$.

Classical algorithms, such as HOSVD and STHOSVD, compute these factors via per-mode SVDs or sequential truncations, respectively; error bounds are governed by the tail singular value energy in each mode (Minster et al., 2019).

2. Low-rank Regularization Approaches

Recent models incorporate regularizations that enforce or exploit low-rank structure beyond simple multilinear constraints:

• Weighted Nuclear Norms: Penalize individual factor matrix ranks via nuclear norms, e.g., $\sum_n \omega_n\|A^{(n)}\|_*$, where weights $\omega_n$ are dynamically adapted using singular value sums of other factors (Gong et al., 4 Aug 2025). This mechanism enables mode-wise scaling and automatic balancing.
• Sparse Tucker Core: $\ell_1$ penalties on the core tensor, $\|\mathcal{G}\|_1$, induce explicit sparsity, favoring a minimal set of interactions for true low-rankness and facilitating compressed representations (Pan et al., 2020, Gong et al., 4 Aug 2025).
• Nonnegativity and Sparsity: Nonnegative Tucker Decomposition (NTD) combines nonnegativity constraints and core/factor sparsity, which improves essential uniqueness and parts-based representations in applications such as clustering and face recognition (Zhou et al., 2014).

These regularizations are typically embedded in tensor completion and regression objectives, e.g.,

$$\min_{\mathcal{G},\{A^{(n)}\},\mathcal{X}} (1-\alpha)\sum_n \omega_n\|A^{(n)}\|_* + \alpha\|\mathcal{G}\|_1$$

subject to multilinear structure and data-fidelity constraints (Gong et al., 4 Aug 2025).

3. Optimization Strategies and Algorithmic Advances

Low-rank Tucker decomposition is a nonconvex optimization problem. Modern solvers exploit the geometry of the Tucker variety and advanced proximal techniques:

• PALM and ProADM: Proximal Alternating Linearized Minimization (PALM) applies block-separable majorizers and blockwise proximal steps, e.g., soft-thresholding for the core, singular-value shrinkage for factors, and Lipschitz-adapted updates. ProADM (proximal ADMM) introduces dual multipliers for equality and observation constraints and alternates primal-dual updates with global convergence guarantees under the Kurdyka–Łojasiewicz property (Gong et al., 4 Aug 2025).
• Iterative Reweighted Schemes: Structured core sparsity via majorization-minimization and overrelaxed MFISTA enable automatic rank determination and efficient solves for incomplete tensor decomposition (Yang et al., 2015).
• Riemannian and Tangent-cone Methods: Manifold-based optimization projects ambient gradients onto tangent cones, uses HOSVD as retraction, and incorporates fixed-rank and rank-adaptive routines (GRAP, TRAM) for robust completion and adaptive rank selection (Gao et al., 2023).
• Randomized Algorithms: Sketching and range-finder randomized SVDs (R-STHOSVD, Sketch-STHOSVD, RTSMS) dramatically cut memory and computation; single-mode sketching further reduces overhead for massive tensors and enables adaptive rank discovery (Minster et al., 2019, Dong et al., 2023, Hashemi et al., 2023).

Algorithm	Principle	Scalability
PALM/ProADM	Proximal blockwise updates	Linear per-block solves, KL global convergence
Iterative Reweighted	MM, FISTA-based	Linear per-iteration complexity
GRAP/TRAM	Riemannian gradient, rank adaptation	Sublinear/linear convergence via tangent-cone geometry
RTSMS, Sketch-STHOSVD	Randomized sketching/least squares	Nearly optimal for large dense/sparse tensors

4. Applications and Empirical Performance

Low-rank Tucker decomposition is fundamental in:

• Tensor Completion: Image inpainting (multispectral, MRI, RGB), traffic and internet flow data. Modern low-rank models with combined sparse-core and adaptive factor regularization yield superior PSNR, SSIM, and MAPE metrics under up to 95% missingness (Gong et al., 4 Aug 2025, Pan et al., 2020, Yang et al., 2015).
• Regression: NA$_0$CT$^2$ achieves exact $\ell_0$ regularization in the core tensor via noise-augmentation, outperforming $\ell_1$ methods in prediction error and sparsity recovery (Yan et al., 2023).
• Robust Recovery: Tucker-$L_2$E and robust CUR-based decompositions address outlier-contaminated data, deliver sharper feature extraction and denoising, and maintain performance in high-rank scenarios (Heng et al., 2022, Cai et al., 2023).
• Functional and Bayesian Extensions: FunBaT generalizes Tucker models to continuous-indexed data via GP-modulated latent functions and scalable state-space inference, improving supervised learning in climate, pollution, and geospatial datasets (Fang et al., 2023).

5. Computational Complexity and Scalability

Classical deterministic algorithms (HOSVD, STHOSVD) scale as $O(\sum_n I_n \prod_k I_k)$, with sequential truncation and randomized variants (R-STHOSVD, RTSMS) reducing the memory and flop cost by orders of magnitude—often to $O(d n^d r)$ for $d$ modes and rank $r$ (Hashemi et al., 2023, Dong et al., 2023, Minster et al., 2019).

Regularization strategies (weighted nuclear norms, sparse cores) are embedded in blockwise updates solvable via efficient proximal operators. Randomized algorithms guarantee expected Frobenius norm error bounds by explicitly controlling the sketch size and mode-wise truncation—often matching HOSVD to within a factor involving the low-rank tail energy.

Method	Per-Iter. Cost	Error Control
HOSVD/STHOSVD	$O(I_n^d)$	$\sum_n$ mode-$n$ tail energy
R-STHOSVD	$O(d n^d r)$	Probabilistic, oversampling factor
RTSMS	$O(d n^d r)$	Product error bound over modes
PALM/ProADM	Linear in observed entries	Global convergence via KL property

6. Connections, Extensions, and Open Problems

Low-rank Tucker decomposition is generalized by:

• Tucker tensor varieties: Exploiting their geometry enables adaptive rank selection that avoids overfitting and underfitting, with tangent-cone characterizations for optimization (Gao et al., 2023).
• Nonconvex and manifold optimization: Algorithms incorporate projection/retraction onto Tucker variety or the Stiefel/Grassmann manifold for higher-order tensors (Jin et al., 2022).
• Randomized subspace and sketching methods: Single-mode and multi-mode sketching, leverage scores, and TensorSketch are now well-established for scaling to arbitrarily large tensors (Ma et al., 2021, Hashemi et al., 2023).
• Nonnegative and sparse tensor models: Uniqueness and identifiability are enhanced by core-factor sparsity and nonnegativity, with theoretical rank inequalities and practical feature extraction for clustering and face recognition (Zhou et al., 2014).
• Robustness, CUR factorizations, and statistical models: Outlier isolation, $\ell_0$ regularization, and functional Bayesian models (GP-based) all connect to Tucker low-rank structure as a principle for tensor inference under real-world uncertainty (Cai et al., 2023, Fang et al., 2023, Yan et al., 2023).

Open problems include optimal selection of multilinear ranks for arbitrary data, provable guarantees under non-independent missingness, and adaptive regularization that jointly tunes sparsity and smoothness across factors and core. Recent work also extends these principles to generalized tensor networks, tree tensor networks, and symmetric moment tensor decomposition in high-dimensional statistics (Mahankali et al., 2022, Jin et al., 2022).

7. Summary and Impact

Low-rank Tucker decomposition is central to modern multiway data analysis. Advances in regularization (adaptive nuclear norms, sparse cores), scalable optimization (proximal, randomized sketching), and robust, functional, and statistical modeling approaches have sharply increased both the accuracy and efficiency of Tucker-based methods for data completion, recovery, regression, and knowledge discovery. The development of tensor-variety geometry and rank-adaptive solvers signals continued progress on model selection and theoretical guarantees. Empirical evidence consistently confirms the superiority of integrated low-rank and local regularization models, particularly in extreme data-missing or contaminated regimes.