Generalized Tensor Random Projection

• The paper introduces a framework where tensor data is compressed using structured, multilinear projection operators that approximately preserve Frobenius geometry.
• It covers various methods—including TT/CP tensorized maps, Khatri–Rao sketches, and sparse random indexing—that balance trade-offs in storage, random-number generation, and contraction cost.
• The approach enables efficient tensor decompositions and regression by reducing memory and computational burdens, though challenges remain in addressing tight theoretical bounds for high-order tensors.

Generalized Tensor Random Projection denotes a family of randomized linear embeddings for tensor-valued data in which the projection operator is itself organized by multilinear structure rather than as a single dense matrix. In this literature, the term covers tensor-wise sketches applied to $\mathrm{vec}(X)$, mode-wise maps of the form $X \times_1 P_1 \times_2 \cdots \times_d P_d$, measurement maps $y_j=\langle X,W^{(j)}\rangle$ with low-rank TT or CP random tensors, Khatri–Rao or row-product random matrices, and sparse random indexing schemes based on outer products of index vectors (Rakhshan et al., 2020, Sun et al., 2021, Casarin et al., 2 Oct 2025, Sandin et al., 2011). Their common purpose is to compress very high-dimensional tensors while approximately preserving Frobenius or Euclidean geometry, and to do so with substantially lower storage, random-number generation, and contraction cost than dense Johnson–Lindenstrauss transforms.

1. Definition and relation to classical random projection

For vectors, the Johnson–Lindenstrauss lemma states that for any $\varepsilon \in (0,1)$ and any finite set $S \subset \mathbb{R}^n$ of $N$ points, there exists a linear map $f:\mathbb{R}^n \to \mathbb{R}^m$ with $m=O(\varepsilon^{-2}\log N)$ such that

$$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$

for all $x,y \in S$. A standard realization is the Gaussian map $X \times_1 P_1 \times_2 \cdots \times_d P_d$0 with $X \times_1 P_1 \times_2 \cdots \times_d P_d$1 having i.i.d. $X \times_1 P_1 \times_2 \cdots \times_d P_d$2 entries (Rakhshan et al., 2020).

Generalized Tensor Random Projection extends this paradigm from vectors to tensors by preserving Frobenius distances after multilinear compression. One general definition is

$X \times_1 P_1 \times_2 \cdots \times_d P_d$3

where the operator may reduce mode sizes, reduce the number of modes, or both. Special cases include a tensor-wise projection $X \times_1 P_1 \times_2 \cdots \times_d P_d$4, a pure mode-wise projection

$X \times_1 P_1 \times_2 \cdots \times_d P_d$5

and tensorized measurement maps obtained from random tensors through inner products (Casarin et al., 2 Oct 2025).

A second broad formulation treats the projection operator as a collection of structured random measurements. For tensors $X \times_1 P_1 \times_2 \cdots \times_d P_d$6, one defines

$X \times_1 P_1 \times_2 \cdots \times_d P_d$7

where each $X \times_1 P_1 \times_2 \cdots \times_d P_d$8 is not stored densely but implicitly represented in TT, CP, Khatri–Rao, or related compressed form (Rakhshan et al., 2020, Sun et al., 2021).

2. Principal construction families

The literature contains several non-equivalent but closely related operator classes. They differ mainly in how the random projector is factorized and in which tensor algebra the sketch is applied.

Family	Representative map	Distinctive property
TT/CP tensorized JL maps	$X \times_1 P_1 \times_2 \cdots \times_d P_d$9	Low-rank random tensors replace dense Gaussian matrices
Khatri–Rao / TRP maps	$y_j=\langle X,W^{(j)}\rangle$0	Low memory and cheap generation/application
Rank-one TT / TTRP	$y_j=\langle X,W^{(j)}\rangle$1	Storage $y_j=\langle X,W^{(j)}\rangle$2 rather than $y_j=\langle X,W^{(j)}\rangle$3
Mode-wise tensor sketches	$y_j=\langle X,W^{(j)}\rangle$4	Natural for Tucker, TR, and regression pipelines
Sparse random indexing	$y_j=\langle X,W^{(j)}\rangle$5	Fixed-size state tensor and online updates
t-product sketches	$y_j=\langle X,W^{(j)}\rangle$6	Preserves column tubal spaces in FFT-based tensor algebra

TT and CP tensorized random projections replace the dense Gaussian matrix in a JLT by a low-rank tensor implicitly represented in compressed form with random factors. In the TT case, each measurement tensor is specified by TT cores $y_j=\langle X,W^{(j)}\rangle$7; in the CP case, each measurement tensor has factor matrices $y_j=\langle X,W^{(j)}\rangle$8. The paper “Tensorized Random Projections” introduces both constructions and shows that they preserve Frobenius distances while cutting memory from $y_j=\langle X,W^{(j)}\rangle$9 to $\varepsilon \in (0,1)$0 for TT and $\varepsilon \in (0,1)$1 for CP (Rakhshan et al., 2020).

Khatri–Rao random projections build the sketch matrix as

$\varepsilon \in (0,1)$2

so each column is a rank-1 Kronecker vector. The related Tensor Random Projection map uses the Khatri–Rao product of several smaller random projections and stores only the factors rather than the full sketch matrix. These constructions are explicitly designed for low memory and for compatibility with sparse base maps, including implementations with no floating point operations in the inner loop (Sun et al., 2021, Saibaba et al., 31 Jul 2025).

A separate TT specialization is Tensor Train Random Projection, where all TT-ranks are set to one. In that case the projection matrix is exactly the Kronecker product

$\varepsilon \in (0,1)$3

with TT-cores taken by default from the Rademacher distribution. This is a concrete rank-one TT member of the wider generalized family (Feng et al., 2020).

Mode-wise random projections appear in randomized Tucker/HOSVD, randomized tensor ring decomposition, and compressed Bayesian tensor regression. The common pattern is to compute per-mode bases $\varepsilon \in (0,1)$4 or random maps $\varepsilon \in (0,1)$5, form a compressed tensor

$\varepsilon \in (0,1)$6

or its randomized analogue, then perform the downstream decomposition or inference in the compressed space (Che et al., 2019, Yuan et al., 2019, Casarin et al., 2 Oct 2025).

Sparse random indexing is structurally different but still fits the generalized projection viewpoint. A $\varepsilon \in (0,1)$7th-order tensor is encoded into a fixed-size state tensor by

$\varepsilon \in (0,1)$8

where the index vectors are sparse, balanced, and nearly orthogonal. This scheme emphasizes online encoding, decoding by inner products, and dynamic index growth rather than JL-style finite-set embedding (Sandin et al., 2011).

3. Isometry, variance, and Johnson–Lindenstrauss–type guarantees

A recurrent theoretical objective is expected isometry. For TT and CP tensorized maps, the squared norm of the projected tensor is unbiased: $\varepsilon \in (0,1)$9 The corresponding variance bounds are

$S \subset \mathbb{R}^n$0

and

$S \subset \mathbb{R}^n$1

These bounds imply that TT requires substantially smaller $S \subset \mathbb{R}^n$2 than CP to achieve the same distortion for high-order tensors (Rakhshan et al., 2020).

The corresponding JL-style finite-set dimensions differ sharply. For a set $S \subset \mathbb{R}^n$3 of tensors of order $S \subset \mathbb{R}^n$4, TT achieves

$S \subset \mathbb{R}^n$5

whereas CP requires

$S \subset \mathbb{R}^n$6

The dependence on $S \subset \mathbb{R}^n$7 is the main reason the TT format is described as substantially superior to CP in the high-order regime (Rakhshan et al., 2020).

TT-based tensorized maps also admit Rademacher cores. “Rademacher Random Projections with Tensor Networks” shows that TT-Gaussian cores can be replaced by TT-Rademacher cores with the same embedding size and the same variance rate,

$S \subset \mathbb{R}^n$8

while Matrix Product Operator random projections are not a Johnson–Lindenstrauss transform because their variance does not vanish with $S \subset \mathbb{R}^n$9 (Rakhshan et al., 2021).

Rank-one TTRP yields exact expected isometry under independent mean-zero, variance-one cores,

$N$0

together with the explicit variance bound

$N$1

Because the fourth moment $N$2 is minimized by the Rademacher distribution, the paper identifies Rademacher cores as an optimal default choice (Feng et al., 2020).

For Khatri–Rao random projections, the theory is stated in subspace-embedding and range-finding form. If $N$3 has orthonormal columns, then under subgaussian KRP factors the condition

$N$4

ensures that $N$5 is a $N$6 subspace embedding with probability at least $N$7. The same paper gives high-probability guarantees for randomized low-rank approximation and two-sided single-view sketches, while noting that current constants still scale exponentially in $N$8 (Saibaba et al., 31 Jul 2025).

Compressed Bayesian tensor regression adds tensor-wise and mode-wise JL inequalities for Achlioptas-type projections. Tensor-wise projection reduces directly to the Achlioptas JL lemma on $N$9, while the mode-wise theorem introduces a dependence on $f:\mathbb{R}^n \to \mathbb{R}^m$0 through fourth-moment bounds. This theory is used to support posterior consistency of inference on compressed tensor covariates (Casarin et al., 2 Oct 2025).

4. Storage, contraction cost, and algorithmic structure

The main practical motivation for generalized tensor random projection is the replacement of dense $f:\mathbb{R}^n \to \mathbb{R}^m$1 maps by structured operators whose parameters scale only with mode sizes and ranks. For TT and CP tensorized maps, the memory per measurement is

$f:\mathbb{R}^n \to \mathbb{R}^m$2

and the aggregated storage over $f:\mathbb{R}^n \to \mathbb{R}^m$3 measurements becomes $f:\mathbb{R}^n \to \mathbb{R}^m$4 for TT and $f:\mathbb{R}^n \to \mathbb{R}^m$5 for CP under uniform sizes (Rakhshan et al., 2020).

When the input is itself low-rank, contraction costs inherit the same multilinear structure. For TT inputs, the per-measurement contraction cost is

$f:\mathbb{R}^n \to \mathbb{R}^m$6

and for CP inputs the per-measurement cost is

$f:\mathbb{R}^n \to \mathbb{R}^m$7

This is the computational setting in which matching projector format to input format becomes especially advantageous (Rakhshan et al., 2020).

Rank-one TTRP gives even smaller asymptotic storage. The paper reports the following comparison: Gaussian RP requires storage $f:\mathbb{R}^n \to \mathbb{R}^m$8 and compute $f:\mathbb{R}^n \to \mathbb{R}^m$9; Very Sparse RP requires storage $m=O(\varepsilon^{-2}\log N)$0 and compute $m=O(\varepsilon^{-2}\log N)$1; Gaussian TRP requires storage $m=O(\varepsilon^{-2}\log N)$2 and compute $m=O(\varepsilon^{-2}\log N)$3; TTRP requires storage $m=O(\varepsilon^{-2}\log N)$4 and compute $m=O(\varepsilon^{-2}\log N)$5. For large $m=O(\varepsilon^{-2}\log N)$6, this translates into substantially lower memory and a slower growth of runtime than dense baselines (Feng et al., 2020).

Khatri–Rao sketches reduce random-number generation as well as storage. To form $m=O(\varepsilon^{-2}\log N)$7, only $m=O(\varepsilon^{-2}\log N)$8 random numbers are generated rather than $m=O(\varepsilon^{-2}\log N)$9. In Tucker pipelines this makes MTTKRP the natural application kernel, and memoization in RHOSVD-KRP can reduce multi-MTTKRP cost by approximately $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$0 (Saibaba et al., 31 Jul 2025).

Sparse random indexing emphasizes fixed-size storage and online updates. Encoding a single component update touches exactly $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$1 state entries, the state tensor has size $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$2, and index-code storage is $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$3. The method is explicitly incremental: new indices are added by generating new random codes without modifying the state tensor (Sandin et al., 2011).

5. Role in tensor decomposition, regression, and large-scale computation

Generalized tensor random projection is used both as a stand-alone embedding and as a front end for tensor algorithms. In randomized low multilinear rank approximation, mode-wise Kronecker sketches are used to construct orthonormal subspaces $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$4 for Tucker/HOSVD. The global error is then controlled by the tail singular values of the mode-$$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$5 unfoldings, and the incremental Tucker-SVD algorithm is reported as consistently the fastest across smooth, sparse, noisy, and MNIST tensor benchmarks, with classification accuracy comparable to deterministic methods (Che et al., 2019).

Randomized tensor ring decomposition uses mode-wise projection on every mode of a large-scale tensor, followed by TRALS or TRSVD on the compressed tensor and a lifting step $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$6. The paper reports that rTRALS is approximately $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$7 faster than TRALS at matched accuracy, with $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$8 s versus $$(1-\varepsilon)\|x-y\|_2^2 \le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2$$9 s, and that rTRSVD is approximately $x,y \in S$0 faster than TRSVD, with $x,y \in S$1 s versus $x,y \in S$2 s. Across broader tests it reports $x,y \in S$3–$x,y \in S$4 speedups (Yuan et al., 2019).

In CP factorization, random projections can contract a tensor along all but two modes, producing a family of matrices

$x,y \in S$5

that share the same left and right factors and differ only by diagonal scalings. Approximate joint diagonalization of these matrices reduces tensor factorization to simultaneous matrix diagonalization, and the method is stated to apply to non-orthogonal and asymmetric tensors of arbitrary order (Kuleshov et al., 2015).

Under the t-product, randomized one-sided sketches $x,y \in S$6 capture column tubal spaces and make small-core GTSVD possible. For randomized GTSVD II, the reported speedups over deterministic GTSVD are approximately $x,y \in S$7, $x,y \in S$8, and $x,y \in S$9 for $X \times_1 P_1 \times_2 \cdots \times_d P_d$00, $X \times_1 P_1 \times_2 \cdots \times_d P_d$01, and $X \times_1 P_1 \times_2 \cdots \times_d P_d$02, while relative errors remain on the order of $X \times_1 P_1 \times_2 \cdots \times_d P_d$03 (Ahmadi-Asl et al., 2023).

Khatri–Rao projections are used beyond tensor decompositions. In block Hankel system identification, single-view R-KRP achieves approximately $X \times_1 P_1 \times_2 \cdots \times_d P_d$04 speedup over R-Gauss and approximately $X \times_1 P_1 \times_2 \cdots \times_d P_d$05 over RandERA, while using approximately $X \times_1 P_1 \times_2 \cdots \times_d P_d$06 of the random numbers of RandERA and approximately $X \times_1 P_1 \times_2 \cdots \times_d P_d$07 of R-Gauss. In flow sensor placement, RHOSVD-KRP-MEMO achieves approximately $X \times_1 P_1 \times_2 \cdots \times_d P_d$08 speedup over HOSVD at $X \times_1 P_1 \times_2 \cdots \times_d P_d$09, and RHOSVD-KRP approximately $X \times_1 P_1 \times_2 \cdots \times_d P_d$10 (Saibaba et al., 31 Jul 2025).

Compressed Bayesian tensor regression uses GTRP to embed tensor-valued covariates before fitting low-rank Bayesian models. The method supports tensor-wise, mode-wise, and combined projections, employs Gibbs sampling with hierarchical priors, and uses Bayesian model averaging over multiple random projections. Simulations and a real S&P 500 return prediction task are reported to show better out-of-sample prediction with significantly reduced computational cost compared to standard Bayesian tensor regression (Casarin et al., 2 Oct 2025).

6. Limitations, exclusions, and open directions

The main negative result in this area is structural rather than statistical. MPO-format random projections are shown not to be Johnson–Lindenstrauss transforms, because the variance of the squared norm contains a term that does not decay with $X \times_1 P_1 \times_2 \cdots \times_d P_d$11. This sharply distinguishes TT-row constructions from MPO-parameterized operators (Rakhshan et al., 2021).

Format dependence also creates a strong separation between TT and CP. For tensorized JL maps, CP requires a projection dimension with a dominant $X \times_1 P_1 \times_2 \cdots \times_d P_d$12 factor, making it unsuitable for very high-order tensors unless $X \times_1 P_1 \times_2 \cdots \times_d P_d$13 becomes very large. TT mitigates the high-order penalty through rank control, but its runtime and memory still increase as TT ranks increase, so projector-rank tuning remains a genuine trade-off (Rakhshan et al., 2020).

Khatri–Rao theory remains more pessimistic than practice. The improved analysis removes explicit $X \times_1 P_1 \times_2 \cdots \times_d P_d$14-dependence except in logarithms and covers broad subgaussian families, but constants and $X \times_1 P_1 \times_2 \cdots \times_d P_d$15 terms still scale exponentially in $X \times_1 P_1 \times_2 \cdots \times_d P_d$16. The paper explicitly identifies the removal of this $X \times_1 P_1 \times_2 \cdots \times_d P_d$17-dependence as an open problem (Saibaba et al., 31 Jul 2025).

Several frameworks also have theory gaps of their own. The TTRP concentration bound is stated to be not tight because it still involves the data dimension $X \times_1 P_1 \times_2 \cdots \times_d P_d$18 (Feng et al., 2020). For randomized tensor ring decomposition, the paper relies on standard randomized range-finding intuition and empirical results but does not provide new formal multi-mode TRP error theorems (Yuan et al., 2019). In compressed Bayesian tensor regression, the concentration bounds are described as conservative in high $X \times_1 P_1 \times_2 \cdots \times_d P_d$19, and CP or Tucker rank misspecification can degrade performance (Casarin et al., 2 Oct 2025).

Finally, not all generalized tensor projections are designed for the same regime. Sparse random indexing is explicitly said to perform well at high dimensionality only, with $X \times_1 P_1 \times_2 \cdots \times_d P_d$20, and its goal is low cross-talk and incremental storage rather than finite-set JL embedding. This suggests that “generalized tensor random projection” is best viewed as a research umbrella rather than as a single canonical map: the unifying theme is structured random compression of multilinear data, but the relevant guarantees, costs, and target applications depend strongly on the choice of tensor algebra, factorization format, and downstream task (Sandin et al., 2011).