TR-ALSAR Algorithm Overview

• TR-ALSAR is a family of algorithms that fits low-rank tensor ring decompositions to multi-dimensional data using efficient, numerically stable ALS routines.
• It mitigates issues such as intermediate data explosion and instability by incorporating QR-based techniques and chain contractions in the solution process.
• Empirical evaluations demonstrate that variants like TR-ALS-SC and TR-ALS-QR offer faster convergence and robust performance on large-scale, ill-conditioned datasets.

Tensor Ring Alternating Least Squares with Advanced Reduction (TR-ALSAR) algorithms form a family of practical, numerically stable routines for fitting low-rank tensor ring (TR) decompositions to multi-dimensional data. They address computational challenges inherent to classical TR-ALS—including intermediate data explosion and numerical instability—by exploiting algebraic structure and QR-based stabilization. This approach yields efficient, scalable solutions applicable to large-scale and ill-conditioned tensor decomposition problems (Yu et al., 2022).

1. Mathematical Formulation of Tensor Ring Decomposition

Given an $N$th-order tensor $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$, the TR decomposition expresses $X$ as the trace over a product of $N$ third-order core tensors: $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$ where $G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$ for $n=1,\ldots,N$ with $R_{N+1}=R_1$, and $G_n(i_n) := G_n(:,i_n,:)$.

The goal is to minimize the Frobenius reconstruction error: $$\min_{G_1,\ldots,G_N} \| \operatorname{TR}(\{G_n\}) - X \|_F^2$$ Alternating Least Squares (ALS) is performed by cyclically updating one core $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$0 at a time, keeping the others fixed, via a least-squares subproblem formulated on appropriate unfoldings of $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$1 and subchains $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$2 of the remaining cores.

2. Normal Equations and Subproblem Structure

For core $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$3, the subproblem in unfolded form is: $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$4 where $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$5 is the mode-$$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$6 unfolding of $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$7 and $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$8 is the mode-2 unfolding of $$X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$9.

Setting the derivative to zero yields the normal equations: $X$0 Solving this $X$1 linear system is efficient for small $X$2, but direct formation of $X$3 is generally computationally prohibitive for large-scale tensors.

3. Coefficient Matrix Simplification: TR-ALS-SC

TR-ALS-SC leverages TR algebraic structure to factorize and contract the Gram matrices required in the normal equations, thus avoiding explicit computation of large unfoldings and their associated data explosion.

• For each $X$4, define the Gram-tensor:

$X$5

with $X$6, where $X$7 denotes the matrix outer product.

• The cumulative Gram matrix is built via chains of contractions:

$X$8

with contraction operator $X$9 merging matching $N$0-modes.

The right-hand side of the normal equations is computed implicitly by means of Matricized-Tensor-Times-Subchain-Product (MTTSP), implemented as a sequence of small matrix multiplications. All steps avoid forming intermediate objects larger than the input tensor $N$1 or the cores. The resulting system

$N$2

is solved per core update.

4. QR-Based Numerical Stabilization: TR-ALS-QR

For ill-conditioned or collinear core scenario, TR-ALS-QR stabilizes the ALS subproblems by casting them in orthogonal bases via QR factorizations:

• Compute a mode-2 QR of each core: $N$3, where $N$4 is orthonormal and $N$5 is triangular in its mode-2 unfolding.
• Construct the subchain $N$6 of concatenated $N$7 (excluding $N$8), then obtain its QR factorization $N$9.
• Form the projected tensor $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$0 by mode-wise multiplying $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$1 with the conjugate transposes $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$2 for all $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$3.
• The TR-ALS-QR update for $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$4 solves the triangular system:

$$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$5

This approach guarantees that the least-squares solves are well-conditioned. The computational overhead from QR factorizations is moderate in practice, especially when $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$6 and explicit formation of $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$7 is avoided.

A fourth hybrid, QRNE, combines coefficient simplification from SC with QR stabilization by interleaving these strategies within the update, further improving speed and stability.

5. Algorithmic Descriptions

The following table summarizes key steps in the three principal TR-ALSAR variants:

Variant	Gram Construction	Solve Type	Stabilization
TR-ALS	Explicit unfolding	Normal equations	None
TR-ALS-SC	Chain contraction of $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$8	Normal equations	None
TR-ALS-QR	(Not needed)	Triangular system	QR orthogonalization

Pseudocode for each variant cycles over the cores, forming the appropriate subchain data (either explicitly, via chain contractions, or QR bases), computes the right-hand side by MTTSP, performs a solve (normal equations or triangular system), and updates the core.

6. Computational Complexity and Memory

Three main cost components are considered: upfront initialization, per-iteration update cost, and memory footprint. Let $$X(i_1, \ldots, i_N) = \operatorname{Trace}[G_1(i_1) \cdot G_2(i_2) \cdots G_N(i_N)]$$9 be the tensor order, $G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$0 the uniform mode size, and $G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$1 the uniform TR rank.

Time complexity per sweep:

Part / Method	TR-ALS	TR-ALS-SC	TR-ALS-QR
Upfront init	$G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$2	$G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$3	$G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$4
MTTSP	$G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$5	$G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$6	$G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$7
Gram construction	$G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$8	$G_n \in \mathbb{R}^{R_n \times I_n \times R_{n+1}}$9	–
System solve	$n=1,\ldots,N$0	$n=1,\ldots,N$1	$n=1,\ldots,N$2
QR factorization	–	–	$n=1,\ldots,N$3
Other	$n=1,\ldots,N$4	$n=1,\ldots,N$5	$n=1,\ldots,N$6

$n=1,\ldots,N$7Typically avoided for large $n=1,\ldots,N$8 by implicit computation.

Memory footprint:

• Data tensor $n=1,\ldots,N$9: $R_{N+1}=R_1$0
• Cores: $R_{N+1}=R_1$1
• Gram-tensors $R_{N+1}=R_1$2: $R_{N+1}=R_1$3
• Temporaries: $R_{N+1}=R_1$4 (baseline), $R_{N+1}=R_1$5 (QR).

7. Empirical Properties and Trade-Offs

Extensive experiments on synthetic and real data reveal the following characteristics:

• TR-ALS-SC reduces per-iteration cost by roughly half compared to baseline TR-ALS, converges in the same number of iterations, and can achieve 2–3$R_{N+1}=R_1$6 faster wall-clock time for large $R_{N+1}=R_1$7 and $R_{N+1}=R_1$8.
• TR-ALS-QR exhibits significant stability advantages on ill-conditioned or collinear core problems, where TR-ALS and TR-ALS-SC can stagnate or produce inaccurate solutions, while TR-ALS-QR maintains robust convergence and lower errors.
• On real imaging and video datasets (e.g., DC-Mall hyperspectral, "Park Bench," "Tabby Cat"), all methods yield identical reconstruction errors for $R_{N+1}=R_1$9, while TR-ALS-SC and TR-ALS-QRNE run 5–10$G_n(i_n) := G_n(:,i_n,:)$0 faster than TR-ALS; TR-ALS-QRNE is the fastest stable variant.

The selection of algorithmic variant is thus dictated by the desired balance of speed and numerical robustness:

• TR-ALS-SC is optimal for well-conditioned data and maximizes speed.
• TR-ALS-QR (and QRNE) ensure numerical stability, crucial when data are noisy or core collinearity is high, with only moderate computational overhead.
• The hybrid QRNE provides near-SC speed and QR stability, without formation of large intermediate tensors (Yu et al., 2022).