---
title: Recursive Sketched Interpolation (RSI)
url: https://www.emergentmind.com/topics/recursive-sketched-interpolation-rsi
type: topic
---

# Recursive Sketched Interpolation (RSI)

Recursive Sketched Interpolation (RSI) is a “scale product” algorithm for computing Hadamard products of tensor trains (TT) and related element-wise mappings with cubic complexity in the TT-rank $χ$. It combines randomized tensor-train sketching with interpolative decomposition (ID) and a recursive re-interpolation strategy that maintains ranks and avoids forming $χ^2$-sized bond spaces. In the formulation introduced in "Recursive Sketched Interpolation: Efficient Hadamard Products of Tensor Trains" [2602.17974], RSI is intended for settings such as TT-based function multiplication for nonlinear differential equations, convolutions, Hadamard products of multiple TTs, and other element-wise nonlinear mappings.

## 1. Tensor-train formulation and the Hadamard-product bottleneck

A $d$-dimensional TT tensor $X \in \mathbb{R}^{n_1 \times \cdots \times n_d}$ is represented by cores $G_k \in \mathbb{R}^{r_{k-1}\times n_k \times r_k}$ with $r_0=r_d=1$ and TT-ranks $\{r_k\}$. Its expansion is
$$
X(i_1,\ldots,i_d)=G_1[:,i_1,:]\,G_2[:,i_2,:]\cdots G_d[:,i_d,:].
$$
For two tensors $X$ and $Y$ with identical modes, the Hadamard product is defined entrywise by
$$
(X\circ Y)(i_1,\ldots,i_d)=X(i_1,\ldots,i_d)\cdot Y(i_1,\ldots,i_d).
$$
If the input cores are $A_k$ and $B_k$, the naive TT construction forms
$$
G_k(:,i_k,:)=A_k(:,i_k,:)\otimes B_k(:,i_k,:),
$$
so the ranks multiply as $r_k^G=r_k^A r_k^B$. Under uniform ranks $r^A \approx r^B \approx χ$, forming these Kronecker slices costs $\Omega(\sum_k n_k χ^4)$, and TT-rounding with output rank $R=χ^2$ scales like $\Omega(\sum_k n_k χ^6)$, typically dominating the total cost. Conventional approaches therefore scale at least as $\mathcal{O}(χ^4)$ and often worse due to rounding, even though the output ranks $χ^2$ are usually redundant [2602.17974].

This bottleneck is the immediate problem RSI addresses. Its key premise is that the full $χ^2$ bond space need not be formed explicitly if one can approximate the relevant scale information of the right-hand segment and then interpolate only representative slices.

## 2. Core mechanism: sketching, slice selection, and recursive re-interpolation

RSI adopts a “scale product” viewpoint: it never constructs $χ^2$ bond spaces. Instead, it compresses scale information from the right-hand segment via sketching and then uses ID to interpolate representative slices, recursively updating cores left-to-right. The randomized sketching step fixes two adjacent external indices open and sketches the remaining right segment with a random separable map. For TT $A$, one generates independent Gaussian matrices $\Omega_{s_j}\in\mathbb{R}^{n_j\times k}$ for $j\ge 3$, forms sketched cores $\Omega_{s_j}^{\top}A_j$, and builds a Khatri–Rao product sketch matrix $S_A\in\mathbb{R}^{r_2^A\times k}$; the same construction is used for $B$. Contracting $S_A$ and $S_B$ with the first two cores yields sketched tensors
$$
\tilde T_1\in\mathbb{R}^{n_1\times n_2\times k},\qquad
\tilde T_2\in\mathbb{R}^{n_1\times n_2\times k},
$$
and the sketched product is
$$
\tilde G^{i_1 i_2 k}=\tilde T_1^{i_1 i_2 k}\cdot \tilde T_2^{i_1 i_2 k}.
$$

The interpolation step is ID-based slice selection. After matricizing $\tilde G$ with rows $i_1$ and columns $(i_2,k)$, RSI computes a rank-$χ$ row ID,
$$
\tilde G^{i_1 [i_2 k]} \approx X^{i_1 \alpha_1} R^{\alpha_1 [i_2 k]},
$$
where $X\in\mathbb{R}^{n_1\times χ}$ contains an identity submatrix and $R=\tilde G(\mathbb{I}_1,:)$ are pivot rows indexed by $\mathbb{I}_1$. The interpolation matrix $X$ becomes the first TT core $G_1(:,i_1,:)=X^{i_1\alpha_1}$.

The recursive component enters through re-interpolation. RSI re-interpolates the input TTs $A$ and $B$ onto the selected pivot slices $\mathbb{I}_1$ by slicing the contraction $A_1A_2$ over $i_1\in\mathbb{I}_1$ so that the left bond aligns with $\alpha_1$ of size $χ$, and then repeats the procedure on the next pair of open indices. At each stage, pivot sets are nested, the same rank-$χ$ structure is preserved, and the algorithm proceeds left-to-right. In the last iterations, when the remaining right segment is smaller than $k$, sketching is skipped; the Hadamard product of the contracted tails is computed directly, and successive IDs extract the remaining cores. The final core is taken from the skeleton $R$ of the last ID. Throughout, the intermediate bonds remain size $χ$, never $χ^2$ [2602.17974].

## 3. Parameterization and implementation structure

The sketch size is chosen so that the row ID can attain the target rank. The paper states the condition as $k \ge χ/d_{\min}+p$, where $d_{\min}$ is a representative mode size and $p\ge 0$ is oversampling; the practical guidance writes this as $k=\lceil χ/d_{\mathrm{phys}}\rceil+p$, with small $p$—typically $0$ to $10$—usually sufficient. At iteration $j$, the sketched tensor is matricized to shape $(n_j\cdot χ)\times(n_{j+1}\cdot k)$, and ID is applied via partial rank-revealing LU (prrLU) until the error is at most $\varepsilon_{\mathrm{ID}}$ or the rank reaches $χ$ [2602.17974].

The implementation has a backward-sweep and a left-to-right phase. In the backward sweep, RSI precomputes sketched cores and Khatri–Rao product sketch matrices $S_A^{(1)},\ldots,S_A^{(d-2)}$ and their analogues for $B$. In the main loop, it contracts two neighboring TT cores with the appropriate cached sketch, forms the sketched Hadamard product, performs row ID, records pivots, and re-interpolates the inputs so that all TTs share aligned nested interpolation sets. If the remaining tail dimension satisfies $\prod_{i=j+2}^d n_i < k$, sketching is skipped and exact tail contractions are used instead.

The procedure is single-pass: unlike TT-cross, it does not require back-and-forth sweeps. It is also compatible with TT libraries such as ITensor. TT-rounding or TT-SVD may be applied after RSI if additional compression is desired, but the intended output is already rank-$χ$.

## 4. Complexity, memory, and error model

The asymptotic motivation for RSI is most transparent when it is compared with standard alternatives.

| Method | Asymptotic scaling | Characteristic issue |
|---|---:|---|
| RSI | $\mathcal{O}(d\,m^2χ^3+dχ^3)$ | fixed-rank construction |
| Naive direct | at least $\Omega(\sum n_k χ^4)$ | rank multiplication |
| Direct with rounding | up to $\Omega(\sum n_k χ^6)$ | TT-rounding dominates |
| TT-cross | $\mathcal{O}(χ^4)$ overall | multiple sweeps |

For uniform mode size $m$, the paper gives the overall RSI cost as $\mathcal{O}(d\,m^2χ^3+dχ^3)$, which reduces to $\mathcal{O}(χ^3)$ for fixed $d$ and moderate $m\ll χ$. Per-iteration costs are dominated by sketch contractions of two TT cores, row ID on a matrix of size roughly $(mχ)\times χ$, and re-interpolation contractions. Sketch precomputation contributes $\mathcal{O}(dχ^3)$.

The memory footprint is likewise controlled by keeping the bond dimension at $χ$. Storing Khatri–Rao sketch matrices requires $\mathcal{O}(dχk)=\mathcal{O}(dχ^2/m)$, storing sketched cores requires $\mathcal{O}(dχ^3/m)$ per TT, and the intermediate sketched tensors have size $m\times m\times k$, namely $\mathcal{O}(mχ)$. This is the principal memory contrast with naive constructions, whose intermediate objects reflect $χ^2$ ranks.

Accuracy is organized around the ID step and the randomized sketch. The local ID errors $\|\tilde G-XR\|\le \varepsilon_{\mathrm{ID}}$ serve as effective proxies for global TT error because of the nested interpolation structure. The Gaussian sketch with Khatri–Rao composition is described as preserving the column space of the right-segment matricization with high probability, but the paper does not provide a formal probabilistic theorem. Instead, it presents the heuristic summary that, for appropriate $k$ and $\varepsilon_{\mathrm{ID}}$,
$$
\|(X\circ Y)-\mathrm{RSI}(X,Y)\|_F \le \varepsilon
$$
with high probability determined by the randomness in $\Omega$ and the conditioning and coherence of the TT cores; oversampling $p$ reduces failure probability and improves stability [2602.17974].

## 5. Extensions, benchmarks, and application domains

RSI is generalized beyond pairwise Hadamard products. For multiple TTs, the sketched product step becomes
$$
\tilde G=\prod_{\ell=1}^L \tilde T^{(\ell)},
$$
and the paper states that the complexity remains $\mathcal{O}(χ^3)$ because the maintained ranks stay at $χ$ and the sketching and ID costs scale only by a constant factor in the number of operands. Element-wise nonlinear mappings are treated by replacing the product with
$$
\tilde G^{i_j i_{j+1} k}=f\!\left(\tilde T^{i_j i_{j+1} k}\right),
$$
with the remainder of the algorithm unchanged; the paper explicitly reports a demonstration for $\mathrm{ReLU}$, $g(x)=\max(0,x)$ [2602.17974].

The empirical study spans several TT and QTT regimes. On the quantum MPS diagonal problem for spin-1 Heisenberg chains with $20$ sites and $\chi_{\max}(\psi)\in\{10,20,30\}$, direct Kronecker-plus-rounding gives the best accuracy, TCI is second, and RSI is slightly worse but close; runtime scaling is reported as significantly lower-order for RSI than a $χ^3$ reference, whereas TCI and direct lie between $χ^3$ and $χ^4$. For $50$ sites and $\chi_{\max}(\psi)\in\{20,40,60,80,100\}$, RSI’s deviation in the diagonal observable $H_{zz}$ is about an order of magnitude larger than direct but acceptable, while TT-rounding dominates the direct method and RSI grows slower than both the full direct method and even the Kronecker-only phase.

On $25$-bit QTT Gaussians with $\chi_{\max}(f)=10$, RSI reaches $\varepsilon_r\approx 10^{-13}$ at $\chi_{\max}(g)=12$ for two-Gaussian products with various separations, comparable to TCI and direct. For “spiked” Gaussians, RSI and direct converge similarly, while TCI often fails to converge from random initialization because of the near-zero structure. In multiple-product tests at $\chi_{\max}(g)=15$ for $f_1f_2$, $f_1f_2^2$, and $f_1^2f_2^2$, RSI runtime varies mildly across cases, with reported speedups up to $8904\times$ over direct and $300\times$ over TCI for $f_1^2f_2^2$.

The paper also identifies regimes where RSI is less favorable. For oscillatory QTT functions involving rapid oscillations and mixed exponentials and polynomials, RSI converges more slowly; oversampling with $p=5$ or $10$ yields slight improvement, but an accuracy gap relative to TCI and direct remains. In application settings, the method is used for a nonlinear PDE active-matter example in which a QTT representation of $D_{xx}$ on a $256\times 256$ grid, vectorized to $2^{16}$ with $\chi_{\max}=20$, yields $\varepsilon_r\approx 2.15\times 10^{-3}$; RSI approximates $D_{xx}\cdot D_{xx}$ with improving fidelity as $\chi_{\max}$ increases, matching input QTT accuracy around $\chi=30$. In convolution via TT Fourier transforms with an MPO-based QTT FFT, the overall convolution error is reported as $\varepsilon_r\approx 1.91\times 10^{-7}$, and the multiplication stage is not the bottleneck because of RSI’s $\mathcal{O}(χ^3)$ scaling.

## 6. Assumptions, limitations, and broader numerical context

The stated assumptions are that the inputs are well approximated by rank-$χ$ TTs and that the sketch captures enough of the right-segment column space for successful ID at each iteration. Under these assumptions, RSI provides rank-controlled Hadamard products and element-wise mappings without explicit passage through $χ^2$ bond spaces. A plausible implication is that RSI is most effective when the right-segment matricizations have sufficiently low effective dimension for randomized range capture and subsequent pivot interpolation to remain faithful.

The principal limitation emphasized in the paper is an accuracy gap on highly oscillatory inputs. The current random Khatri–Rao sketches may be insufficiently expressive in such geometries; increasing $k$ or opening more indices improves accuracy but increases cost. The paper also explicitly notes that no formal probabilistic error bounds are provided: its guarantees are heuristic and empirical rather than theorem-level.

Future directions are stated in concrete terms: adaptive or structured sketching, including multi-cluster sketches, SRHT, and higher TT-ranks in sketches; deterministic variants and alternative slice-selection strategies such as nuclear-score maximization; expanded theory for probabilistic guarantees and error propagation across nested IDs; and broader applications to tensorized neural networks, quantum information measures, and MPO–MPS operations. A plausible contextual link is that RSI belongs to a wider movement in numerical linear algebra toward sketch-based reduction and short recursive constructions, as seen in sketched Arnoldi-type relations for matrix functions and structured sketch-and-project methods for linear systems [2306.06481], [2407.00746].

Source: https://www.emergentmind.com/topics/recursive-sketched-interpolation-rsi