---
title: 'Rank-Adaptive HOOI: Efficient Tucker Decomposition'
url: https://www.emergentmind.com/topics/rank-adaptive-higher-order-orthogonal-iteration-hooi
type: topic
---

# Rank-Adaptive HOOI: Efficient Tucker Decomposition

Rank-adaptive higher-order orthogonal iteration (HOOI) is an iterative algorithm for computing the truncated Tucker decomposition of a tensor to within a prescribed error tolerance. It advances the classical HOOI approach by adaptively selecting mode-wise ranks, achieving minimal multilinear rank representation subject to accuracy constraints. This approach directly addresses longstanding inefficiencies associated with fixed-rank and non-orthogonal alternatives. Rank-adaptive HOOI is locally optimal within each iterative update and features monotonic convergence, providing strong theoretical guarantees and practical compression benefits across synthetic and real-world tensors [2110.12564].

## 1. Problem Setup and Objective

Given an $N$th-order tensor
\[
\mathcal{X} \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}
\]
and an error tolerance $0 < \varepsilon < 1$, the goal is the truncated Tucker decomposition:
\[
\mathcal{X} \approx \mathcal{G} \times_1 U^{(1)} \times_2 \cdots \times_N U^{(N)},
\]
where each $U^{(n)} \in \mathbb{R}^{I_n \times R_n}$ is column-orthonormal and $\mathcal{G} \in \mathbb{R}^{R_1 \times \cdots \times R_N}$ is the core. The decomposition must satisfy:
\[
\| \mathcal{X} - \mathcal{G} \times_1 U^{(1)} \cdots \times_N U^{(N)} \|_F \leq \varepsilon \| \mathcal{X} \|_F.
\]
Unlike standard HOOI, the multilinear ranks $R_n$ are adaptively determined during the iteration to be the smallest possible integers yielding the specified approximation accuracy [2110.12564].

## 2. Rank-Adaptive HOOI Algorithm

The rank-adaptive HOOI algorithm integrates mode-wise orthogonal updates via truncated singular value decomposition (SVD) and a minimal rank selection procedure adhering to the prescribed error. The stepwise algorithm is as follows:

### Pseudocode

1. **Initialization**:
   - Provide orthonormal $\{U_0^{(n)}\}$ and initial ranks $\{R_n^0\}$.
   - Compute initial core:
     \[
     \mathcal{G}_0 = \mathcal{X} \times_1 (U_0^{(1)})^T \cdots \times_N (U_0^{(N)})^T
     \]
   - Set iteration $k \leftarrow 0$.

2. **Iterative Update** (repeat until $\|\mathcal{G}_k\|_F \leq \sqrt{1-\varepsilon^2}\|\mathcal{X}\|_F$):

   For $n=1,\ldots,N$:
   - Form intermediate tensor
     \[
     \mathcal{B} = \mathcal{X} \times_1 (U^{(1)}_{k+1})^T \cdots \times_{n-1} (U^{(n-1)}_{k+1})^T \times_{n+1} (U^{(n+1)}_k)^T \cdots \times_N (U^{(N)}_k)^T
     \]
   - Unfold $\mathcal{B}$ into $B_{(n)}$ (mode-$n$ unfolding).
   - Compute full SVD: $B_{(n)} = U \Sigma V^T$ with $\Sigma = \mathrm{diag}(\sigma_1,\sigma_2,\dots)$.
   - Determine minimal $R$ such that
     \[
     \sum_{r>R} \sigma_r^2 \leq \|\mathcal{B}\|_F^2 - (1-\varepsilon^2)\|\mathcal{X}\|_F^2
     \]
     Set $R_n^{k+1} = R$.
   - Update $U_{k+1}^{(n)} = U_{:,1:R_n^{k+1}}$.

   Set $k \leftarrow k+1$.

By selecting the minimal $R_n$ per mode that maintains feasibility, the algorithm ensures locally optimal rank minimization at each step [2110.12564].

## 3. Theoretical Guarantees: Local Optimality and Convergence

The procedure is supported by two central theorems:

- **Local Optimality:** For fixed factors except $U^{(n)}$, the update rule for $R_n^{k+1}$ provides the smallest rank such that the reconstructed error does not exceed $\varepsilon \|\mathcal{X}\|_F$. The subproblem reduces to a best-rank approximation of $B_{(n)}$, where the truncation threshold is dictated by the tolerance constraint.

- **Monotonicity:** Each sequence of mode-wise ranks $\{R_n^k\}_k$ is non-increasing and eventually stabilizes. At each update, a larger rank could always be retained without exceeding the error bound, but the algorithm explicitly selects the smallest feasible rank.

These results are direct consequences of the orthogonality-enforced SVD subproblem and orthogonal invariance of the Frobenius norm in HOOI [2110.12564].

## 4. Computational Complexity

Let $J_n = \prod_{i \ne n} I_i$. For each mode-$n$ update:
- **Matricized product:** $O(I_n \times J_n \times R_{\mathrm{rest}})$, $R_{\mathrm{rest}} = \prod_{i \ne n} R_i$
- **Full SVD:** $O(I_n\, J_n\, R_n)$ for rank-$R_n$ truncation

Summed over all modes, per-iteration cost is
\[
O\left(\sum_{n=1}^N (I_1 \cdots I_N + I_n J_n R_n)\right)
\]
For $I_1 = \cdots = I_N = I$, $R_1 = \cdots = R_N = R$:
\[
O(N I^N R)
\]
Compared to classical HOOI, this approach requires a full SVD rather than (potentially less expensive) fixed-rank truncation, but manages lower effective ranks in practice [2110.12564].

## 5. Comparative Numerical Results

Experiments highlight the advantages of rank-adaptive HOOI over fixed-rank and greedy strategies:

| Problem Type                | Fixed-Rank Methods    | Greedy HOSVD      | Rank-Adaptive HOOI  |
|-----------------------------|----------------------|-------------------|---------------------|
| Synthetic $500^3$ tensor    | $(440,480,350)$      | $(330,330,330)$   | $(100,100,100)$     |
| Coulomb kernel, $I^4$, $I=200$ | up to $200\times$ more parameters | $2-5\times$ more parameters | minimal parameter count |
| MNIST $28\times28\times5000\times10$ | compression $50-300$ | compression $200-800$ | compression $1000-5000$ |

- On synthetic tensors with added noise, rank-adaptive HOOI exactly recovers the true rank and achieves an order of magnitude lower reconstruction error than fixed-rank strategies.
- For the regularized Coulomb kernel, rank-adaptive HOOI matches error thresholds with up to $200\times$ fewer parameters than $t$-HOSVD and $2$–$5\times$ improvements over greedy approaches, with comparable computation time—only at the tightest tolerances does SVD computation dominate.
- On MNIST digit tensors, rank-adaptive HOOI obtains much higher compression (up to $5000\times$) at comparable classification accuracy ($93\%$–$95\%$), with testing time reduced from tens of seconds to $\approx 0.4$s [2110.12564].

## 6. Rank Adaptivity in Context

**Rationale for Rank Adaptivity:** Fixed-rank Tucker decompositions require a priori specification of ranks $R_n$, typically set conservatively large to avoid exceeding error tolerance, resulting in redundancy and decreased computational efficiency. Greedy or uniform $t$-HOSVD variants improve upon this but still overestimate ranks. Rank-adaptive HOOI automatically determines minimal feasible ranks mode-by-mode, avoiding unnecessary storage and computation without compromising accuracy.

**Distinction from Fixed-Rank HOOI and ALS:** Fixed-rank HOOI maintains constant $R_n$, precluding rank minimization even if a lower rank suffices. Classical alternating least squares (ALS) solves unconstrained least-squares problems per mode, lacking an explicit rank constraint and mechanism for elimination of insignificant singular vectors. By enforcing orthonormal updates and truncated SVD solutions, HOOI is functionally a modified ALS (MALS) that uniquely admits natural rank adaptivity [2110.12564].

The algorithm guarantees per-iteration feasibility ($\|\mathcal{X} - \widehat{\mathcal{X}}\|_F \leq \varepsilon \|\mathcal{X}\|_F$), achieves local optimality in rank selection, converges as ranks are monotonically nonincreasing, and yields efficient representations that accelerate downstream tasks.

## 7. Practical Implications and Significance

Rank-adaptive HOOI provides an automated, accuracy-driven mechanism for compact multilinear tensor approximation. It combines the structural advantages of orthogonal tensor decompositions with dynamic rank minimization, resulting in storage-efficient and computationally competitive solutions. Empirical evidence corroborates the superiority of this approach over existing fixed-rank and greedy schemes in both synthetic and real-world application domains, such as scientific data compression and image classification tasks [2110.12564].

Source: https://www.emergentmind.com/topics/rank-adaptive-higher-order-orthogonal-iteration-hooi