Tensor-Train Reformulation

• Tensor-Train Reformulation is a method that expresses high-dimensional tensors as chains of low-order cores, reducing storage from a product of dimensions to a summation.
• It employs techniques like TT-SVD, TT rounding, and randomized sketching to achieve quasi-optimal approximations and efficient multilinear operations.
• Applications range from numerical PDEs and operator inference to statistical learning, offering significant computational and memory savings.

Searching arXiv for recent and foundational papers on tensor-train reformulations. Tensor-train reformulation denotes the recasting of a high-dimensional tensor, operator, or computational pipeline into a chain of low-order cores $$G^{(k)} \in \mathbb{R}^{r_{k-1}\times n_k\times r_k}$$ with boundary ranks $r_0=r_d=1$, so that

$$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$

In this representation, storage scales as $\sum_{k=1}^d n_k r_{k-1}r_k$ rather than $\prod_{k=1}^d n_k$, and many high-dimensional operations can be rewritten as corewise contractions, orthogonalizations, and local truncations (Lee et al., 2014, Psenka et al., 2020).

1. Algebraic structure of the reformulation

For a $d$-way tensor $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$, the TT rank vector is $(r_1,\ldots,r_{d-1})$, with each $r_k$ equal to the rank of the $k$-th TT unfolding. In the notation of partial matricization,

$r_0=r_d=1$0

This connects the chain structure of the cores to a sequence of balanced bipartitions of the modes, rather than to mode-wise unfoldings alone (Cazeaux et al., 11 Mar 2026, Psenka et al., 2020).

Two standard reformulation primitives are TT-SVD and TT rounding. TT-SVD performs a sequence of SVDs on partial unfoldings to construct a TT representation with prescribed ranks, while TT rounding takes a TT representation with ranks $r_0=r_d=1$1 and reduces them to target ranks $r_0=r_d=1$2 by orthogonalization and local truncation via SVD (Cazeaux et al., 11 Mar 2026). In the large-scale analysis literature, TT-SVD is quasi-optimal in the sense that

$r_0=r_d=1$3

and TT-rounding costs $r_0=r_d=1$4 under uniform mode size $r_0=r_d=1$5 and rank bound $r_0=r_d=1$6 (Lee et al., 2014).

The same reformulation extends to operators. A matrix $r_0=r_d=1$7 can be reshaped as a $r_0=r_d=1$8-way tensor with TT-matrix cores $r_0=r_d=1$9. This permits matrix–vector products, inner products, Hadamard products, Kronecker products, and contracted products to be carried out through local core algebra instead of dense ambient-space operations (Lee et al., 2014).

2. Reformulating multilinear operations and inaccessible tensors

A direct instance of tensor-train reformulation is the Tensor Train Contraction Product. For a contraction $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$0, the reformulation first permutes the contracting modes to the front, decomposes the permuted tensors into TT form, contracts the first TT cores by

$$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$1

and then assembles the result as a TT before permuting back. Under uniform mode size $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$2 and equal first TT ranks $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$3, the dominant contraction cost becomes $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$4, whereas the paper states the standard TCP complexity as $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$5 for the corresponding full computation (Kisil et al., 2021).

A second pattern arises when tensor entries are not directly accessible. In that setting, a tensor is viewed as a vector-valued multilinear map, and TT cores are extracted from tensor actions rather than from array entries. The construction proceeds by randomized range finding on action-induced unfoldings, combined with a “peeling” mechanism that exposes the remainder tensor after the first few cores are fixed. This permits TT compression of high-order derivative tensors for implicitly defined nonlinear mappings, including derivative tensors of the noise-whitened parameter-to-output map for a stochastic PDE, without forming the full tensor explicitly (Alger et al., 2020).

These two cases illustrate a general point: TT reformulation is not limited to storage compression. It can replace an ambient multilinear object by a sequence of low-order maps whose interfaces are controlled by TT ranks. This suggests that the essential operation is often not “approximating a tensor” but “changing the algebra in which the problem is expressed.”

3. Randomized and sketch-based reformulations

Randomized TT reformulation has recently been sharpened by the Block Sparse Tensor Train sketch. The BSTT sketch $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$6, with $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$7, is built by stacking $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$8 independent TT row-blocks of TT-rank $$X(i_1,\ldots,i_d)=G^{(1)}(:,i_1,:)\,G^{(2)}(:,i_2,:)\cdots G^{(d)}(:,i_d,:).$$9 and normalizing by $\sum_{k=1}^d n_k r_{k-1}r_k$0. By varying the two integer parameters $\sum_{k=1}^d n_k r_{k-1}r_k$1 and $\sum_{k=1}^d n_k r_{k-1}r_k$2, BSTT interpolates between the Khatri–Rao sketch $\sum_{k=1}^d n_k r_{k-1}r_k$3 and the Gaussian TT sketch $\sum_{k=1}^d n_k r_{k-1}r_k$4 (Cazeaux et al., 11 Mar 2026).

The principal reformulation result is that sketching guarantees can be made linear in tensor order $\sum_{k=1}^d n_k r_{k-1}r_k$5 and subspace dimension $\sum_{k=1}^d n_k r_{k-1}r_k$6. BSTT is an $\sum_{k=1}^d n_k r_{k-1}r_k$7-oblivious subspace embedding if

$\sum_{k=1}^d n_k r_{k-1}r_k$8

and it is an $\sum_{k=1}^d n_k r_{k-1}r_k$9-oblivious subspace injection if

$\prod_{k=1}^d n_k$0

The cost of applying $\prod_{k=1}^d n_k$1 to a TT input with maximum TT rank $\prod_{k=1}^d n_k$2 and maximum mode size $\prod_{k=1}^d n_k$3 is

$\prod_{k=1}^d n_k$4

with memory

$\prod_{k=1}^d n_k$5

so the sketch itself remains in TT form (Cazeaux et al., 11 Mar 2026).

These guarantees feed directly into randomized QB factorization and randomized TT rounding. In the OSE regime, the QB approximation obeys

$\prod_{k=1}^d n_k$6

and randomized TT rounding inherits quasi-optimality with linear scaling in $\prod_{k=1}^d n_k$7 and the target ranks. The same paper also records a limitation of the underlying product structure: dilation can be heavy-tailed, because the upper singular values depend on products of chi-square factors across cores (Cazeaux et al., 11 Mar 2026).

4. Optimization and learning in TT coordinates

When the TT ranks are fixed, the set

$\prod_{k=1}^d n_k$8

is a smooth embedded submanifold. Its dimension is

$\prod_{k=1}^d n_k$9

On this manifold, the tangent projector $d$0, the Riemannian gradient

$d$1

and the exact Riemannian Hessian

$d$2

can all be written in TT-native form. In the formulation of second-order optimization, projector applications cost $d$3, the diagonal curvature sum has the same order, and the cross-term sum costs $d$4, enabling a Riemannian trust-region method on the fixed-rank TT manifold (Psenka et al., 2020).

In statistical learning, TT reformulation replaces dense weight objects by TT cores. In TT networks for predictive modeling, the regression weight tensor or TT-operator for a fully connected layer is learned directly by ALS with ridge regularization and GSVD-based updates. The paper reports “a reduction of up to 95% of the coefficients” in the MLP recovery experiment, and states that TT models in the Mackey–Glass and NASDAQ experiments “converges in 2–10 sweeps,” whereas comparable MLP baselines required “150–250 epochs” or “>2000 epochs” (Costa et al., 2021).

Several supervised and unsupervised variants specialize the same principle. The Support Tensor Train Machine replaces the rank-one weight of a support tensor machine by a TT, turning each core update into a standard linear SVM subproblem in a mixed-canonical gauge (Chen et al., 2018). Tensor-train discriminant analysis represents discriminant projections by TT or multi-branch TT networks, so that projected features become matrices or third-order tensors rather than single vectors (Sofuoglu et al., 2019). Graph Regularized Tensor Train decomposition adds the Laplacian penalty

$d$5

to the TT objective, preserving local neighborhood structure while constraining TT cores to Stiefel manifolds (Sofuoglu et al., 2019).

5. Completion, factorization, and structured recovery

A major completion reformulation replaces Tucker-style mode unfoldings by TT-balanced unfoldings. In TT nuclear-norm completion,

$d$6

where $d$7 is the balanced unfolding that places the first $d$8 modes in the rows and the remaining modes in the columns. This yields the SiLRTC-TT convex surrogate and the TMac-TT multilinear factorization model. For equal mode size $d$9 and uniform TT rank $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$0, the reported one-iteration complexities are $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$1 for SiLRTC-TT and $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$2 for TMac-TT (Phien et al., 2016).

An alternating-minimization variant under the TT/MPS model updates one core at a time by slice-wise least-squares problems. Under equal TT ranks, the per-iteration complexity is $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$3, where $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$4 is the number of observed entries. This TT completion by alternating minimization is reported to be superior to existing TT-based baselines in several synthetic and real settings (Wang et al., 2016).

For third-order visual data, TT rank minimization can be combined with nonlocal self-similarity by stacking similar cubes into a fourth-order “group” tensor and minimizing a TT nuclear norm on each group. The resulting NL-TT method uses an ADMM scheme tailored to the grouped structure and, according to the paper, is “superior to several existing state-of-the-art methods in terms of both qualitative and quantitative measures” (Ding et al., 2020). A different branch of factorization enforces non-negativity: distributed NN-TT decomposes a dense tensor into non-negative TT cores through successive NMF subproblems on distributed unfoldings, using MPI collectives together with Dask and Zarr to support out-of-core reshaping and 16–256 core scaling (Bhattarai et al., 2020).

The observation pattern itself can also be reformulated. In fiberwise completion along a single mode, some fibers are fully observed and others entirely missing. The 2025 paper shows that, for this observation model, one can “use only standard linear algebra operations to compute the tensor train decomposition” and obtain deterministic recovery guarantees under conditions on the observation pattern, rather than relying on random entrywise sampling and incoherence assumptions (Sofi et al., 16 Sep 2025).

6. Scientific computing, canonical forms, and current boundaries

In numerical PDEs, TT reformulation converts classical discretizations into low-rank tensor algebra. A recent finite-difference framework for three-dimensional parabolic equations rewrites explicit Euler, implicit Euler, and Crank–Nicolson time stepping directly on TT cores, and solves the implicit systems by a matrix-free preconditioned conjugate gradient method that exploits separability in TT form. The reported time savings versus full-grid methods range from $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$5 to $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$6, and memory savings from $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$7 to $X \in \mathbb{R}^{n_1\times\cdots\times n_d}$8, while preserving essentially the same accuracy (Manzini et al., 12 Sep 2025).

A related nonintrusive reformulation appears in Tensor-Train Operator Inference. There, snapshot data are stored in TT or QTT form, least-squares operator learning is executed through TT pseudoinverses and TT contractions, and a reduced-order variant uses the last TT core as the reduced snapshot matrix, thereby replacing POD/SVD by TT conversion (Danis et al., 9 Sep 2025). In random field modeling, TT-Karhunen–Loève expansion rewrites second- and third-order cumulant functions in TT form on NURBS-parametric domains, thereby removing the need to choose basis functions, collocation points, and quadrature points, and avoiding large-scale eigenvalue problems (Bu et al., 2019). In high-order tensor denoising and SVD, Tensor-Train Orthogonal Iteration combines TT-SVD initialization with backward/forward updates and achieves the minimax-optimal rate

$X \in \mathbb{R}^{n_1\times\cdots\times n_d}$9

under the spiked tensor model (Zhou et al., 2020).

Recent work has also moved TT reformulation beyond the real and complex fields. A 2026 paper introduces a “unique normal form and associated polynomial-time reduction strategy for tensor trains over arbitrary fields,” based on an LDPU decomposition rather than on SVD. The reduction runs in $(r_1,\ldots,r_{d-1})$0, where $(r_1,\ldots,r_{d-1})$1 is the maximum TT rank and $(r_1,\ldots,r_{d-1})$2 is the TT size, and the resulting normal form minimizes TT ranks and size (Vilmart, 7 Jul 2026). By contrast, TT-derived rank-1 series such as TTr1SVD provide an orthogonal rank-1 decomposition that is unique for a fixed order of indices, but the number of SVDs can be exponential in $(r_1,\ldots,r_{d-1})$3 in the worst case, and the decomposition depends on index ordering (Batselier et al., 2014).

Across these settings, several limitations recur. TT reformulations remain sensitive to mode ordering, rank selection, and conditioning; rank growth after nonlinear operations often requires repeated rounding; and some guarantees are probabilistic rather than uniform. A plausible implication is that TT reformulation is best understood not as a single algorithm, but as a family of structure-preserving translations from ambient multilinear problems into chained low-rank coordinates, with the specific success of the translation determined by whether the target tensor, operator, or observation pattern is genuinely compatible with low TT ranks.