Tensor Random Projection (RFSF-TRP)

• Tensor Random Projection (RFSF-TRP) is a framework that compresses large-scale, high-order tensors by applying mode-wise random projections to preserve their multilinear structure.
• It uses QR decomposition on tensor unfoldings to form orthonormal bases, enabling efficient tensor ring (TR) decomposition with methods such as rTRSVD and rTRALS.
• The approach achieves significant speedups and memory savings in practical applications, balancing reconstruction error with computational efficiency.

Tensor Random Projection (RFSF-TRP) is a framework for efficient large-scale multilinear data compression and decomposition, combining mode-wise random projections with tensor ring (TR) factorization. It is motivated by the high computational and memory demands of traditional TR algorithms when applied to large dense tensors. By employing random projections in each tensor mode, RFSF-TRP reduces the tensor dimensions while preserving the underlying multilinear structure, enabling downstream tensor ring decomposition at a dramatically reduced cost without sacrificing accuracy. The approach exploits randomized linear algebra and leverages established properties of random projections in the high-dimensional regime (Yuan et al., 2019).

1. Foundational Principles and Pipeline

RFSF-TRP operates on a high-order tensor $$\mathcal X \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}$$ by applying the following two-stage pipeline:

1. Mode-wise Tensor Random Projection: For each mode $n = 1, \ldots, N$, construct a random projection matrix $$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$ (typically Gaussian or structured such as SRFT/Hadamard) and form the mode-$n$ unfolding $\mathcal X_{(n)}$. Project to a low-dimensional subspace, followed by QR decomposition, yielding an orthonormal basis $Q_n \in \mathbb{R}^{I_n \times K_n}$. The tensor is compressed as

$$\mathcal{P} = \mathcal{X} \times_1 Q_1^T \times_2 Q_2^T \cdots \times_N Q_N^T \in \mathbb{R}^{K_1 \times K_2 \times \cdots \times K_N},$$

with $K_n \ll I_n$.

2. TR Decomposition on Compressed Tensor: Standard TR decomposition algorithms (e.g. SVD-based rTRSVD, alternating least squares rTRALS) are run on $\mathcal{P}$, generating small core tensors $\{\mathcal{Z}_n\}_{n=1}^N$ with $n = 1, \ldots, N$0. The original space is recovered by lifting each core via

$n = 1, \ldots, N$1

This reduces the decomposition cost from operating directly on $n = 1, \ldots, N$2 ($n = 1, \ldots, N$3) to operating on $n = 1, \ldots, N$4 ($n = 1, \ldots, N$5), which is typically orders of magnitude smaller (Yuan et al., 2019).

2. Construction of Random Projection Matrices

The random projections in each mode can use fully dense Gaussian matrices or more efficient structured transforms:

• Dense Gaussian: $n = 1, \ldots, N$6 entries are i.i.d. $n = 1, \ldots, N$7.
• Fast Transforms: $n = 1, \ldots, N$8, where $n = 1, \ldots, N$9 is a diagonal Rademacher matrix, $$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$0 is a Hadamard or DFT, and $$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$1 is a subsampling operator.

The QR step ensures $$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$2 and that the span captures the principal subspace in the mode-$$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$3 unfolding, up to random error. Structured maps yield computational acceleration while maintaining the requisite isometry properties (Yuan et al., 2019).

3. Tensor Ring Decomposition Algorithms after Projection

3.1 SVD-based (rTRSVD)

Sequential SVD and truncation are performed via cyclic unfoldings of the small tensor $$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$4:

• At each step, unfold along the mode, perform economy SVD ($$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$5), truncate to rank $$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$6, and reshape $$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$7 to form the new core.
• Process continues cyclically for all modes; the ring constraint $$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$8 is enforced.

3.2 Alternating Least Squares (rTRALS)

ALS iterations solve, for each core,

$$M_n \in \mathbb{R}^{(\prod_{i \ne n} I_i) \times K_n}$$9

where $n$0 is the contracted product of all other cores, and $n$1 the mode-$n$2 unfolding. Factor updates iterate until convergence. The cores are then lifted to the original data space via the stored $n$3.

Both algorithms benefit from the dimension reduction. Empirically, rTRSVD requires $n$4, while rTRALS needs $n$5 to reach the plateau in relative squared error (RSE) (Yuan et al., 2019).

4. Computational Complexity and Memory Analysis

Method	Computation per step	Memory usage	Main scaling term
Standard TRSVD/TRALS	$n$6	$n$7 + cores	$n$8
RFSF-TRP	$n$9	$\mathcal X_{(n)}$0	$\mathcal X_{(n)}$1

With $\mathcal X_{(n)}$2 for a small $\mathcal X_{(n)}$3, the overall speedup is approximately $\mathcal X_{(n)}$4. For large-scale tensors where each mode is high-dimensional, this yields empirical speedups of 4–25× with no loss in accuracy when $\mathcal X_{(n)}$5 is properly tuned (Yuan et al., 2019).

5. Empirical Performance and Trade-offs

• Image Data (e.g. $\mathcal X_{(n)}$6 RGB): As $\mathcal X_{(n)}$7 increases and the RSE curve flattens, select the minimal $\mathcal X_{(n)}$8 achieving the plateau.
• Deep Learning Datasets (CIFAR-10, COIL100): Achieve compression ratios up to $\mathcal X_{(n)}$9 with RSE $Q_n \in \mathbb{R}^{I_n \times K_n}$0 in 1–20 s, outperforming rTucker, rCP, and TRSGD.
• Hyperspectral Images ($Q_n \in \mathbb{R}^{I_n \times K_n}$1): Using spectral mode prior knowledge, setting e.g. $Q_n \in \mathbb{R}^{I_n \times K_n}$2, rTRALS denoises data to RSE 0.015 in 60 s versus >200 s for TRSGD or CPALS.

Practical guidance is to increase $Q_n \in \mathbb{R}^{I_n \times K_n}$3 until diminishing returns are observed in RSE, balancing speed and reconstruction error (Yuan et al., 2019).

6. Theoretical Guarantees and Applicability

If each $Q_n \in \mathbb{R}^{I_n \times K_n}$4 captures the leading $Q_n \in \mathbb{R}^{I_n \times K_n}$5 subspace of the mode-$Q_n \in \mathbb{R}^{I_n \times K_n}$6 unfolding, the classical randomized linear algebra theory guarantees that

$Q_n \in \mathbb{R}^{I_n \times K_n}$7

is within a small multiple of the optimal TR approximation error. The accuracy is directly linked to the efficacy of the initial subspace estimation in the random projection step. The method is robust to various data types and remains compatible with any downstream TR solver (including SGD-based or sampling-based variants) (Yuan et al., 2019).

7. Context within the Tensor Random Projection Ecosystem

RFSF-TRP sits at the intersection of:

• Classical tensor random projection frameworks: Structures such as those in (Sun et al., 2021) and (Rakhshan et al., 2020) establish the biased-variance tradeoff and subspace preservation of tensor-based projections.
• Operator generality: The mode-wise projection approach is a special case of more general tensor random projection maps (including tensor-wise, block-wise, or combined investments as in Bayesian tensor regression (Casarin et al., 2 Oct 2025)).
• Algorithmic structure: It is distinct from earlier methods such as N-way Random Indexing (Sandin et al., 2011) which employs sparse ternary codebooks, but shares the central aim of reducing memory and compute while preserving underlying structure.
• Sketching-based alternatives: Approaches using subchain/Kronecker products and CountSketches, as in (Yu et al., 2022), provide alternative random projection and sketching formulations for TR-ALS, yielding similar order-of-magnitude gains.

The framework is maximally beneficial in large-scale, high-order tensor settings, where both data storage and the per-iteration complexity of traditional multilinear decompositions become prohibitive.

---

References:

• "Randomized Tensor Ring Decomposition and Its Application to Large-scale Data Reconstruction" (Yuan et al., 2019)
• "Tensor Random Projection for Low Memory Dimension Reduction" (Sun et al., 2021)
• "Tensorized Random Projections" (Rakhshan et al., 2020)
• "Incremental dimension reduction of tensors with random index" (Sandin et al., 2011)
• "Practical Sketching-Based Randomized Tensor Ring Decomposition" (Yu et al., 2022)
• "Compressed Bayesian Tensor Regression" (Casarin et al., 2 Oct 2025)