---
title: TR-ALSAR Algorithm Overview
url: https://www.emergentmind.com/topics/tr-alsar-algorithm
type: topic
---

# TR-ALSAR Algorithm Overview

Tensor Ring Alternating Least Squares with Advanced Reduction (TR-ALSAR) algorithms form a family of practical, numerically stable routines for fitting low-rank tensor ring (TR) decompositions to multi-dimensional data. They address computational challenges inherent to classical TR-ALS—including intermediate data explosion and numerical instability—by exploiting algebraic structure and QR-based stabilization. This approach yields efficient, scalable solutions applicable to large-scale and ill-conditioned tensor decomposition problems [2210.11362].

## 1. Mathematical Formulation of Tensor Ring Decomposition

Given an $N$th-order tensor $X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$, the TR decomposition expresses $X$ as the trace over a product of $N$ third-order core tensors:
\[
X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]
\]
where $G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$ for $n=1,\ldots,N$ with $R_{N+1}=R_1$, and $G_n(i_n) := G_n(:,i_n,:)$.

The goal is to minimize the Frobenius reconstruction error:
\[
\min_{G_1,\ldots,G_N} \| \operatorname{TR}(\{G_n\}) - X \|_F^2
\]
Alternating Least Squares (ALS) is performed by cyclically updating one core $G_n$ at a time, keeping the others fixed, via a least-squares subproblem formulated on appropriate unfoldings of $X$ and subchains $G^{\ne n}$ of the remaining cores.

## 2. Normal Equations and Subproblem Structure

For core $G_n$, the subproblem in unfolded form is:
\[
\min_{G_{n(2)}} \| X_{[n]} - G_{n(2)} \cdot (G^{\ne n}_{[2]})^T \|_F^2
\]
where $X_{[n]}$ is the mode-$n$ unfolding of $X$ and $G_{n(2)}$ is the mode-2 unfolding of $G_n$.

Setting the derivative to zero yields the normal equations:
\[
G_{n(2)} \cdot (G^{\ne n}_{[2]} (G^{\ne n}_{[2]})^T) = X_{[n]} G^{\ne n}_{[2]}
\]
Solving this $(R_n R_{n+1}) \times (R_n R_{n+1})$ linear system is efficient for small $R$, but direct formation of $G^{\ne n}_{[2]}$ is generally computationally prohibitive for large-scale tensors.

## 3. Coefficient Matrix Simplification: TR-ALS-SC

TR-ALS-SC leverages TR algebraic structure to factorize and contract the Gram matrices required in the normal equations, thus avoiding explicit computation of large unfoldings and their associated data explosion.

- For each $j \ne n$, define the Gram-tensor:
  \[
  P_j := \sum_{i_j=1}^{I_j} G_j(i_j)^T \circ G_j(i_j)^T
  \]
  with $P_j \in \mathbb{R}^{R_{j+1} \times R_{j+1} \times R_j \times R_j}$, where $\circ$ denotes the matrix outer product.
- The cumulative Gram matrix is built via chains of contractions:
  \[
  M_n = (P_{n-1} \times_{2,4}^{1,3} P_{n-2} \cdots \times_{2,4}^{1,3} P_{n+1})_{<2>}
  \]
  with contraction operator $\times_{2,4}^{1,3}$ merging matching $R$-modes.

The right-hand side of the normal equations is computed implicitly by means of Matricized-Tensor-Times-Subchain-Product (MTTSP), implemented as a sequence of small matrix multiplications. All steps avoid forming intermediate objects larger than the input tensor $X$ or the cores. The resulting system
\[
G_{n(2)} M_n = \text{RHS}_n
\]
is solved per core update.

## 4. QR-Based Numerical Stabilization: TR-ALS-QR

For ill-conditioned or collinear core scenario, TR-ALS-QR stabilizes the ALS subproblems by casting them in orthogonal bases via QR factorizations:

- Compute a mode-2 QR of each core: $G_n = R_n \times_2 Q_n$, where $Q_n$ is orthonormal and $R_n$ is triangular in its mode-2 unfolding.
- Construct the subchain $V_n$ of concatenated $R_j$ (excluding $n$), then obtain its QR factorization $V_n = R_0 \times_2 Q_0$.
- Form the projected tensor $Y$ by mode-wise multiplying $X$ with the conjugate transposes $Q_j^T$ for all $j \ne n$.
- The TR-ALS-QR update for $G_{n(2)}$ solves the triangular system:
  \[
  G_{n(2)} R_{0[2]}^T = Y_{[n]} Q_0
  \]

This approach guarantees that the least-squares solves are well-conditioned. The computational overhead from QR factorizations is moderate in practice, especially when $I \gg R^2$ and explicit formation of $V_n$ is avoided.

A fourth hybrid, QRNE, combines coefficient simplification from SC with QR stabilization by interleaving these strategies within the update, further improving speed and stability.

## 5. Algorithmic Descriptions

The following table summarizes key steps in the three principal TR-ALSAR variants:

| Variant       | Gram Construction           | Solve Type        | Stabilization     |
|---------------|----------------------------|-------------------|-------------------|
| TR-ALS        | Explicit unfolding         | Normal equations  | None              |
| TR-ALS-SC     | Chain contraction of $P_j$ | Normal equations  | None              |
| TR-ALS-QR     | (Not needed)               | Triangular system | QR orthogonalization |

Pseudocode for each variant cycles over the cores, forming the appropriate subchain data (either explicitly, via chain contractions, or QR bases), computes the right-hand side by MTTSP, performs a solve (normal equations or triangular system), and updates the core.

## 6. Computational Complexity and Memory

Three main cost components are considered: upfront initialization, per-iteration update cost, and memory footprint. Let $N$ be the tensor order, $I$ the uniform mode size, and $R$ the uniform TR rank.

Time complexity per sweep:

| Part / Method        | TR-ALS               | TR-ALS-SC           | TR-ALS-QR         |
|----------------------|----------------------|---------------------|-------------------|
| Upfront init         | $O(NIR^2)$           | $O(NIR^2 + NIR^4)$  | $O(NIR^2 + NIR^4)$|
| MTTSP                | $O(NI^N R^2)$        | $O(NI^N R^2)$       | $O(NI^N R^2)$     |
| Gram construction    | $O(NI^{N-1} R^4)$    | $O(NR^6)$           | –                 |
| System solve         | $O(NIR^6)$           | $O(NIR^6)$          | $O(NIR^4)$        |
| QR factorization     | –                    | –                   | $O(NR^{2N+2})^*$  |
| Other                | $O(NI^{N-1}R^3)$     | $O(NR^6+NI^{N-1}R^3)$| $O(NR^{2N+1})$   |

$^*$Typically avoided for large $I$ by implicit computation.

Memory footprint:

- Data tensor $X$: $O(I^N)$
- Cores: $O(NIR^2)$
- Gram-tensors $P_n$: $O(NR^4)$
- Temporaries: $O(I^{N-1}R^2)$ (baseline), $O(R^{2N})$ (QR).

## 7. Empirical Properties and Trade-Offs

Extensive experiments on synthetic and real data reveal the following characteristics:

- TR-ALS-SC reduces per-iteration cost by roughly half compared to baseline TR-ALS, converges in the same number of iterations, and can achieve 2–3$\times$ faster wall-clock time for large $I$ and $N$.
- TR-ALS-QR exhibits significant stability advantages on ill-conditioned or collinear core problems, where TR-ALS and TR-ALS-SC can stagnate or produce inaccurate solutions, while TR-ALS-QR maintains robust convergence and lower errors.
- On real imaging and video datasets (e.g., DC-Mall hyperspectral, "Park Bench," "Tabby Cat"), all methods yield identical reconstruction errors for $R=3\ldots10$, while TR-ALS-SC and TR-ALS-QRNE run 5–10$\times$ faster than TR-ALS; TR-ALS-QRNE is the fastest stable variant.

The selection of algorithmic variant is thus dictated by the desired balance of speed and numerical robustness:

- TR-ALS-SC is optimal for well-conditioned data and maximizes speed.
- TR-ALS-QR (and QRNE) ensure numerical stability, crucial when data are noisy or core collinearity is high, with only moderate computational overhead.
- The hybrid QRNE provides near-SC speed and QR stability, without formation of large intermediate tensors [2210.11362].

Source: https://www.emergentmind.com/topics/tr-alsar-algorithm