---
title: Low-rank Tucker Decomposition
url: https://www.emergentmind.com/topics/low-rank-tucker-decomposition-58162965-442c-4bf1-b3b3-ccb5a7a74b26
type: topic
---

# Low-rank Tucker Decomposition

A low-rank Tucker decomposition represents a high-dimensional tensor as the contraction of a modest-sized core tensor with a collection of low-dimensional factor matrices, each corresponding to a particular mode. This provides an efficient multilinear parameterization that generalizes low-rank matrix factorization to higher-order arrays. Classical, stochastic, and randomized algorithms for low-rank Tucker recovery, as well as application-structured variants and regularized objectives, underlie much of modern tensor analysis in machine learning, signal processing, and scientific computing.

## 1. Problem Formulation and Tucker Model

Given a $d$-way tensor $X^* \in \mathbb{R}^{n_1 \times n_2 \times \cdots \times n_d}$, the low-rank Tucker model expresses $X^*$ as
\[
X^* = S \times_1 U^{(1)} \times_2 \cdots \times_d U^{(d)}
\]
where $S \in \mathbb{R}^{r_1 \times \cdots \times r_d}$ is the core tensor and $U^{(i)} \in \mathbb{R}^{n_i \times r_i}$ are the factor matrices. The vector $(r_1, ..., r_d)$ is the multilinear (Tucker) rank, where each $r_i = \mathrm{rank}(X^{\{i\}})$ with $X^{\{i\}}$ the mode-$i$ unfolding of $X$.

The low-Tucker-rank tensor recovery problem, a central focus in high-dimensional inverse problems and data analysis, is
\[
\min_X \; \tfrac{1}{2} \|\mathcal{A}(X) - y\|_2^2 \qquad \mathrm{s.t.}\ \mathrm{rank}_\mathrm{Tucker}(X) \leq (r_1, ..., r_d)
\]
where $y = \mathcal{A}(X^*) + \text{noise}$ is observed via a linear operator $\mathcal{A}$. The hard constraint on Tucker rank distinguishes this class from convex nuclear-norm surrogates or matrix unfoldings [1909.10132].

## 2. Deterministic and Stochastic Hard-Thresholding Algorithms

Iterative hard thresholding (IHT) algorithms for low-rank tensor recovery generalize established matrix techniques: each step alternates a gradient update of the least-squares loss with a projection onto the low-Tucker-rank manifold, operationally implemented by truncated higher-order SVD (HOSVD). The *stochastic* variant (StoTIHT) splits the measurements into batches and performs each gradient step using only a randomly sampled batch, yielding significant per-iteration speedup at the cost of gradient variance.

The StoTIHT procedure [1909.10132] is:
- Randomly sample a batch of measurements;
- Compute the partial gradient;
- Take a stochastic gradient step (rescaled for unbiasedness);
- Project onto rank-$(r_1,\dots,r_d)$ tensors via
  - For each mode, mode-$i$ SVD, truncating to $r_i$;
  - Form the core by projecting onto the corresponding left singular vector subspaces.

Linear convergence in expectation is attained under a tensor restricted isometry property (TRIP) and an approximate-optimality condition on the projection. StoTIHT achieves an error floor and contraction factor characterized explicitly by the noise level and TRIP constant.

Empirical results indicate that smaller batch sizes reduce wall-clock time, with rapid convergence even at drastic subsampling (e.g., $b=0.25m$), and that the method robustly recovers both synthetic and real data tensors at large scale [1909.10132].

## 3. Randomized and Sketching-Based Approaches

Randomized algorithms for Tucker decomposition exploit data compression and dimensionality reduction by sketching tensor unfoldings with random projections or sampling schemes. Notable variants include:
- Single-mode or two-sided sketches per unfolding, followed by low-rank approximation and iterative truncation [2311.14873, 2301.11598, 1905.07311];
- CUR-type tensor approximations using selected fibers and subtensors for robust projections and alternating minimization [2305.04080].

Sketch-based ALS (alternating least squares) leverages the Kronecker structure of the core tensor update, recasting it as a (possibly regularized) ridge regression with a design matrix $K$ formed via factor matrices. Ridge leverage scores or classic leverage scores guide adaptive row sampling, yielding guarantees of $(1+\varepsilon)$-approximation to the exact fit with sample complexity sublinear in the full tensor size [2107.10654, 2104.01101]. Randomized range-finding and power iterations enhance spectral decay handling.

Theoretical error bounds for randomized Slide/Sketch-STHOSVD and CUR techniques precisely track the singular value tail energies of each unfolding, with constants depending on oversampling and sketch dimensions. Randomized fiber sampling with range finding (mode-parallel HOSVD) enables strong-scaling in high mode dimensions, with flops and memory scaling as $\mathcal{O}(d n s r)$, compared to classical HOSVD’s $\mathcal{O}(d n^{d+1})$ [2603.21379].

## 4. Structured and Regularized Low-Rank Tucker Recovery

Modern formulations integrate additional priors for real-world interpretability and robust estimation:
- Weighted nuclear-norm penalties on factor matrices, sparse cores, and Laplacian-based smoothness regularization yield models for joint global low-rank structure and local smoothness [2508.03755].
- Proximal gradient, PALM, and ADMM-based solvers are applied to nonconvex but structured objectives, with theoretical convergence (to critical points under KL or semi-algebraicity) shown for the PALM/ProADM algorithms [2508.03755, 2010.00359].
- Structured sparsity (group log-sum) on the core supports automatic rank determination and model selection [1511.04695].

In robust settings, Tucker decomposition is coupled with hard thresholding for sparse-corruption separation (TRPCA), as in robust tensor CUR alternating projections, or with robust loss functions (e.g., $L_2$-E trimming) for outlier tolerance [2305.04080, 2208.11806]. $\ell_0$ (exact sparsity) on the core tensor is attainable in tensor regression via noise augmentation, producing exact model pruning and improved predictor identification [2302.10775].

## 5. Symmetric, Structured, and Application-Driven Tucker Decompositions

Symmetric subtensors and structured domains motivate specialized Tucker variants:
- Symmetric moment tensor decomposition leverages the Grassmann and Stiefel manifold geometry, with projected gradient descent and higher-order eigenvalue approximations (HOEVD) achieving both statistical efficiency and scalable implementation [2204.10824].
- O-minus (ring-plus-bridge) architectures provide more balanced and compact cores for multi-view clustering, outperforming standard Tucker and tensor ring decompositions in multi-view latent information capture [2210.12638].
- Nonnegative and sparse Tucker decomposition, accelerated via preceding low-multilinear-rank approximation, enables efficient first-order methods, improved uniqueness (with sparsity), and interpretability in nonnegative data settings [1404.4412].

In large-scale structured problems, implicit and streaming versions circumvent the explicit construction of massive moment tensors or unfoldings—e.g., via moment outer-product identities or online QR-retraction [2204.10824, 2603.21379].

## 6. Geometry, Optimization, and Rank-Adaptivity

The set of low-Tucker-rank tensors forms a real-algebraic variety with a geometry richer than that of matrix varieties. Explicit tangent cone parametrizations reveal a multilayered block structure, with gradient-projection methods (GRAP) and HOSVD-based retractions ensuring provable convergence to stationary points. Adaptive algorithms detect and modify multilinear ranks during iteration, supporting automatic model selection and robust tensor completion without prior knowledge of true rank [2311.18324, 1511.04695].

Riemannian optimization on these varieties and associated manifolds underpins much of modern tensor optimization, supporting line search, retraction, and adaptation strategies crucial for practical performance and reliability.

## 7. Applications and Impact

Low-rank Tucker decomposition is foundational across high-dimensional data analysis, inverse problems, scientific computing, and machine learning. Empirical evidence demonstrates:
- Superior accuracy and speed in tensor completion, denoising, and inpainting under high missingness and outlier regimes [2508.03755, 2010.00359, 2208.11806];
- Robust traffic data imputation, image and video restoration, and multivariate regression with interpretable parameterizations and regularization [2508.03755, 2010.00359, 2302.10775];
- Structured manifold estimation critical for moment tensor analysis, hyperspectral imaging, and multi-view clustering [2204.10824, 2210.12638, 2311.18324].

The continual development of scalable, stochastic, and regularized algorithms—anchored in the multilinear structure and geometry of the Tucker model—has expanded the range and reliability of tensor methods for large-scale, high-dimensional, and application-structured problems.

Source: https://www.emergentmind.com/topics/low-rank-tucker-decomposition-58162965-442c-4bf1-b3b3-ccb5a7a74b26