TR-ALSAR Algorithm Overview

• TR-ALSAR is a family of algorithms that fits low-rank tensor ring decompositions to multi-dimensional data using efficient, numerically stable ALS routines.
• It mitigates issues such as intermediate data explosion and instability by incorporating QR-based techniques and chain contractions in the solution process.
• Empirical evaluations demonstrate that variants like TR-ALS-SC and TR-ALS-QR offer faster convergence and robust performance on large-scale, ill-conditioned datasets.

Tensor Ring Alternating Least Squares with Advanced Reduction (TR-ALSAR) algorithms form a family of practical, numerically stable routines for fitting low-rank tensor ring (TR) decompositions to multi-dimensional data. They address computational challenges inherent to classical TR-ALS—including intermediate data explosion and numerical instability—by exploiting algebraic structure and QR-based stabilization. This approach yields efficient, scalable solutions applicable to large-scale and ill-conditioned tensor decomposition problems (Yu et al., 2022).

1. Mathematical Formulation of Tensor Ring Decomposition

Given an $N$th-order tensor $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$, the TR decomposition expresses $X$ as the trace over a product of $N$ third-order core tensors: $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$ where $G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$ for $n=1,\ldots,N$ with $R_{N+1}=R_1$, and $G_n(i_n) := G_n(:,i_n,:)$.

The goal is to minimize the Frobenius reconstruction error: $$\min_{G_1,\ldots,G_N} \| \operatorname{TR}(\{G_n\}) - X \|_F^2$$ Alternating Least Squares (ALS) is performed by cyclically updating one core $G_n$ at a time, keeping the others fixed, via a least-squares subproblem formulated on appropriate unfoldings of $X$ and subchains $G^{\ne n}$ of the remaining cores.

2. Normal Equations and Subproblem Structure

For core $G_n$, the subproblem in unfolded form is: $$\min_{G_{n(2)}} \| X_{[n]} - G_{n(2)} \cdot (G^{\ne n}_{[2]})^T \|_F^2$$ where $X_{[n]}$ is the mode-$n$ unfolding of $X$ and $G_{n(2)}$ is the mode-2 unfolding of $G_n$.

Setting the derivative to zero yields the normal equations: $$G_{n(2)} \cdot (G^{\ne n}_{[2]} (G^{\ne n}_{[2]})^T) = X_{[n]} G^{\ne n}_{[2]}$$ Solving this $(R_n R_{n+1}) \times (R_n R_{n+1})$ linear system is efficient for small $R$, but direct formation of $G^{\ne n}_{[2]}$ is generally computationally prohibitive for large-scale tensors.

3. Coefficient Matrix Simplification: TR-ALS-SC

TR-ALS-SC leverages TR algebraic structure to factorize and contract the Gram matrices required in the normal equations, thus avoiding explicit computation of large unfoldings and their associated data explosion.

• For each $j \ne n$, define the Gram-tensor:

$$P_j := \sum_{i_j=1}^{I_j} G_j(i_j)^T \circ G_j(i_j)^T$$

with $$P_j \in \mathbb{R}^{R_{j+1} \times R_{j+1} \times R_j \times R_j}$$, where $\circ$ denotes the matrix outer product.

• The cumulative Gram matrix is built via chains of contractions:

$$M_n = (P_{n-1} \times_{2,4}^{1,3} P_{n-2} \cdots \times_{2,4}^{1,3} P_{n+1})_{<2>}$$

with contraction operator $\times_{2,4}^{1,3}$ merging matching $R$-modes.

The right-hand side of the normal equations is computed implicitly by means of Matricized-Tensor-Times-Subchain-Product (MTTSP), implemented as a sequence of small matrix multiplications. All steps avoid forming intermediate objects larger than the input tensor $X$ or the cores. The resulting system

$G_{n(2)} M_n = \text{RHS}_n$

is solved per core update.

4. QR-Based Numerical Stabilization: TR-ALS-QR

For ill-conditioned or collinear core scenario, TR-ALS-QR stabilizes the ALS subproblems by casting them in orthogonal bases via QR factorizations:

• Compute a mode-2 QR of each core: $G_n = R_n \times_2 Q_n$, where $Q_n$ is orthonormal and $R_n$ is triangular in its mode-2 unfolding.
• Construct the subchain $V_n$ of concatenated $R_j$ (excluding $n$), then obtain its QR factorization $V_n = R_0 \times_2 Q_0$.
• Form the projected tensor $Y$ by mode-wise multiplying $X$ with the conjugate transposes $Q_j^T$ for all $j \ne n$.
• The TR-ALS-QR update for $G_{n(2)}$ solves the triangular system:

$G_{n(2)} R_{0[2]}^T = Y_{[n]} Q_0$

This approach guarantees that the least-squares solves are well-conditioned. The computational overhead from QR factorizations is moderate in practice, especially when $I \gg R^2$ and explicit formation of $V_n$ is avoided.

A fourth hybrid, QRNE, combines coefficient simplification from SC with QR stabilization by interleaving these strategies within the update, further improving speed and stability.

5. Algorithmic Descriptions

The following table summarizes key steps in the three principal TR-ALSAR variants:

Variant	Gram Construction	Solve Type	Stabilization
TR-ALS	Explicit unfolding	Normal equations	None
TR-ALS-SC	Chain contraction of $P_j$	Normal equations	None
TR-ALS-QR	(Not needed)	Triangular system	QR orthogonalization

Pseudocode for each variant cycles over the cores, forming the appropriate subchain data (either explicitly, via chain contractions, or QR bases), computes the right-hand side by MTTSP, performs a solve (normal equations or triangular system), and updates the core.

6. Computational Complexity and Memory

Three main cost components are considered: upfront initialization, per-iteration update cost, and memory footprint. Let $N$ be the tensor order, $I$ the uniform mode size, and $R$ the uniform TR rank.

Time complexity per sweep:

Part / Method	TR-ALS	TR-ALS-SC	TR-ALS-QR
Upfront init	$O(NIR^2)$	$O(NIR^2 + NIR^4)$	$O(NIR^2 + NIR^4)$
MTTSP	$O(NI^N R^2)$	$O(NI^N R^2)$	$O(NI^N R^2)$
Gram construction	$O(NI^{N-1} R^4)$	$O(NR^6)$	–
System solve	$O(NIR^6)$	$O(NIR^6)$	$O(NIR^4)$
QR factorization	–	–	$O(NR^{2N+2})^*$
Other	$O(NI^{N-1}R^3)$	$O(NR^6+NI^{N-1}R^3)$	$O(NR^{2N+1})$

$^*$Typically avoided for large $I$ by implicit computation.

Memory footprint:

• Data tensor $X$: $O(I^N)$
• Cores: $O(NIR^2)$
• Gram-tensors $P_n$: $O(NR^4)$
• Temporaries: $O(I^{N-1}R^2)$ (baseline), $O(R^{2N})$ (QR).

7. Empirical Properties and Trade-Offs

Extensive experiments on synthetic and real data reveal the following characteristics:

• TR-ALS-SC reduces per-iteration cost by roughly half compared to baseline TR-ALS, converges in the same number of iterations, and can achieve 2–3$\times$ faster wall-clock time for large $I$ and $N$.
• TR-ALS-QR exhibits significant stability advantages on ill-conditioned or collinear core problems, where TR-ALS and TR-ALS-SC can stagnate or produce inaccurate solutions, while TR-ALS-QR maintains robust convergence and lower errors.
• On real imaging and video datasets (e.g., DC-Mall hyperspectral, "Park Bench," "Tabby Cat"), all methods yield identical reconstruction errors for $R=3\ldots10$, while TR-ALS-SC and TR-ALS-QRNE run 5–10$\times$ faster than TR-ALS; TR-ALS-QRNE is the fastest stable variant.

The selection of algorithmic variant is thus dictated by the desired balance of speed and numerical robustness:

• TR-ALS-SC is optimal for well-conditioned data and maximizes speed.
• TR-ALS-QR (and QRNE) ensure numerical stability, crucial when data are noisy or core collinearity is high, with only moderate computational overhead.
• The hybrid QRNE provides near-SC speed and QR stability, without formation of large intermediate tensors (Yu et al., 2022).