Tensor Search (TenS): Optimizing Tensor Structures

• Tensor Search (TenS) is a methodology for automated structural decision-making in tensor decompositions, optimizing compression and approximation accuracy.
• It leverages diverse algorithmic paradigms—including genetic algorithms, program synthesis, and local search—to efficiently navigate combinatorial tensor configurations.
• Applications span image compression, quantum many-body systems, and data simulation, often yielding significant improvements in compression ratios and processing speed.

Tensor Search (TenS) is not a single universally fixed technical object; rather, recent literature uses the term for several distinct search problems centered on tensorized representations. In the tensor-decomposition and tensor-network literature, TenS most often denotes automated search over tensor shape, tensor-network topology, mode-to-core permutation, ranks, or even decomposition family, with the aim of maximizing compression or minimizing approximation or variational error under explicit constraints (Solgi et al., 2022, Watanabe et al., 2024, Guo et al., 4 Feb 2025). Separate literatures use the same label for late-interaction multi-vector retrieval and, more loosely, for searches involving tensor sectors of physical signals, so the term is best understood through its problem formulation rather than through its name alone (Wang et al., 2 Aug 2025).

1. Problem formulations and scope

Across tensor-network compression work, the central TenS problem is a constrained model-selection problem over structured low-rank representations. In "Tensor Network Structure Search with Program Synthesis" the tensor network is defined as an undirected graph $G=(V,E)$, where each vertex is a tensor and each edge represents a shared index, with the represented tensor written as the contraction $N(G)$. The stated objective is to find the network $G$ that minimizes storage while staying within a prescribed approximation error, reflecting the joint dependence of compression quality on both topology and internal ranks (Guo et al., 4 Feb 2025). In "Domain-Aware Tensor Network Structure Search" the same family of problems is formalized as minimization over $(G,\mathbf{r})$, where $G=(V,E)$ is the TN graph and $\mathbf{r}\in\mathbb{Z}_+^E$ are TN ranks, under an objective combining a structural complexity term $\phi(G,\mathbf{r})$ and normalized Frobenius reconstruction error; that paper further states that TN-SS is NP-hard (Iacovides et al., 29 May 2025).

The literature also contains narrower and broader variants of this basic search problem. "Tensor Shape Search for Optimum Data Compression" treats TenS as optimization over the reshape $\theta=(n_1,\ldots,n_d)$ applied before TT decomposition, observing that the tensor shape affects both the TT ranks and the compression ratio (Solgi et al., 2022). "Permutation Search of Tensor Network Structures via Local Sampling" restricts attention to TN permutation search (TN-PS), in which the template graph is fixed up to isomorphism and the search concerns mode-to-vertex mappings plus TN-ranks (Li et al., 2022). At the opposite extreme, "Hierarchical Tensor Network Structure Search for High-Dimensional Data" generalizes the problem to Tensor Network Structural Rounding (TNSR), where an existing tensor network $N$ is transformed into an optimal network $N^*$ that minimizes parameter count subject to

$N(G)$0

That formulation subsumes the single-tensor-input case as a special instance (Guo et al., 29 Mar 2026).

A concise way to organize these formulations is to distinguish the search variable from the fixed computational substrate.

Formulation	Search variables	Stated criterion
Tensor shape search	$N(G)$1	Maximize TT compression ratio under error bound
TN permutation search	Isomorphic $N(G)$2 and ranks $N(G)$3	Compact TN representation of $N(G)$4
TN structure search	Topology $N(G)$5 and internal ranks	Minimize size under approximation constraint
TNSR	New tree network $N(G)$6 and reshaped indices	Minimize parameter count under relative error

This variety suggests that TenS is best treated as an umbrella for automated structural decisions in tensor methods, rather than as a single algorithmic primitive.

2. Search dimensions: shape, permutation, topology, and decomposition family

One axis of TenS concerns tensorization itself. In the TT setting, the original data $N(G)$7 is reshaped to $N(G)$8, and the objective is explicitly written as maximizing

$N(G)$9

with TT-SVD enforcing the reconstruction bound. Because different reshapes induce different unfoldings and therefore different singular spectra, TenS at this level is a search over mode factorizations before any network topology is even chosen (Solgi et al., 2022).

A second axis concerns permutations within a fixed TN format. TN-PS shows that even if the graph template is fixed, different assignments of tensor modes to TN vertices can induce different feasible ranks and therefore different parameter counts. The paper studies the search space through graph automorphisms and gives exact counts for common formats: for $G$0, a TT path has $G$1 and $G$2, while a TR cycle has $G$3 and $G$4. It then endows the search space with a semi-metric derived from adjacent transpositions, yielding a neighborhood structure for local search (Li et al., 2022).

A third axis is full topology-and-rank search. The program-synthesis approach of (Guo et al., 4 Feb 2025) recasts TN-SS as synthesis of a program over tensor transformations. The search variable is no longer a graph directly, but a DSL program composed of output-directed splits $G$5, where each split corresponds to introducing an internal edge and each rank choice corresponds to its bond dimension. This representation makes topology and rank search compositional: a completed program is equivalent to a tree tensor network.

A fourth axis extends beyond tensor-contraction families altogether. "TenExp: Mixture-of-Experts-Based Tensor Decomposition Structure Search Framework" argues that previous structure-search methods are confined by a fixed factor-interaction family and cannot deliver a mixture of decompositions. TenExp therefore searches across Tucker, FCTN, and TF for third-order tensors, and across Tucker and FCTN for higher-order tensors, with a top-$G$6 mixture-of-experts gate that can either select a single decomposition or combine several (Zhou et al., 3 Mar 2026). This substantially broadens the meaning of TenS: the search object may be not only the structure within a family, but the decomposition family itself.

3. Algorithmic paradigms

The simplest TenS workflow in the surveyed literature is evolutionary search over reshapes. In (Solgi et al., 2022), a GA samples tensor shapes $G$7, evaluates each by TT-SVD, and uses compression ratio as fitness. TT-SVD supplies the approximation guarantee

$G$8

so the optimizer can focus on compression rather than on a separate explicit error-constraint mechanism. Chromosomes are mode-size tuples, and elitism, crossover, and mutation drive the search over feasible reshapes.

For ER-TN and MERA-like states, TenS becomes a local structural reconstruction problem tied to variational energy. "Automatic Structural Search of Tensor Network States including Entanglement Renormalization" defines TenS as repeated local reconstruction of adjacent tensor pairs, local optimization of each admissible structure, stochastic structure selection by a Gibbs rule,

$G$9

and subsequent global tensor re-optimization. Replica exchange is added to improve exploration, and the algorithm operates directly with isometric tensors $(G,\mathbf{r})$0, $(G,\mathbf{r})$1, and $(G,\mathbf{r})$2 under the MERA constraints (Watanabe et al., 2024).

Constraint-based search appears in the program-synthesis line. The key move in (Guo et al., 4 Feb 2025) is to separate sketch generation from sketch completion. First, semantically valid output-directed split sketches are enumerated. Then the rank holes $(G,\mathbf{r})$3 are filled by solving an ILP/CSP built from singular values of matricizations of the original tensor. The stated technical lever is an upper-bound property: singular values of later splits after truncations are bounded by those of corresponding matricizations of the original tensor. This allows sound rank completion without running full tensor decomposition for every candidate. The paper also proves a completeness statement for output-directed splits: if $(G,\mathbf{r})$4 is the optimal tree tensor network for $(G,\mathbf{r})$5 and error bound $(G,\mathbf{r})$6, there exists a program $(G,\mathbf{r})$7 with output-directed splits such that $(G,\mathbf{r})$8 (Guo et al., 4 Feb 2025).

Recent work introduces increasingly heterogeneous search machinery. tnLLM replaces large portions of neighborhood sampling with domain-aware prompting to a LLM that directly proposes TN structures and explanations from mode semantics (Iacovides et al., 29 May 2025). RGTN uses multi-scale renormalization group flows, learnable edge gates, node tension, and edge information flow to support coarse-to-fine continuous structure evolution rather than fixed-scale discrete search (Wang et al., 31 Dec 2025). HISS integrates stochastic sub-network sampling, entropy-guided index clustering based on

$(G,\mathbf{r})$9

recursive refinement, and targeted index reshaping for arbitrary tree networks (Guo et al., 29 Mar 2026). Collectively, these methods show TenS evolving from heuristic enumeration toward structured search spaces, symbolic surrogates, and hierarchical optimization.

4. Representative results and application domains

The empirical literature is diverse, spanning image compression, quantum many-body states, remote-sensing tensors, scientific simulation data, and tensor completion. The reported results are heterogeneous because the underlying objectives differ: some papers optimize compression ratio under a fixed error bound, some optimize variational energy, and some optimize completion quality.

Work	Setting	Reported result
(Solgi et al., 2022)	RGB image TT compression	Optimized shape always improves compression ratio
(Watanabe et al., 2024)	Random XY models	Improvements in variational energy, fidelity, entanglement entropy
(Guo et al., 4 Feb 2025)	TN-SS via program synthesis	Up to $G=(V,E)$0 speedup; $G=(V,E)$1 to $G=(V,E)$2 better compression
(Iacovides et al., 29 May 2025)	Domain-aware TN-SS	Comparable objective values with much fewer evaluations
(Zhou et al., 3 Mar 2026)	Cross-family search / mixture	Best or near-best CR/RE tradeoff on MSIs; strong completion results
(Guo et al., 29 Mar 2026)	Tree-network structural rounding	$G=(V,E)$3 to $G=(V,E)$4 higher compression than TT/HT, peaking at $G=(V,E)$5

In the TT shape-search setting, the COCO image study uses $G=(V,E)$6, population $G=(V,E)$7, parent size $G=(V,E)$8, and $G=(V,E)$9, reporting that the optimized shape $\mathbf{r}\in\mathbb{Z}_+^E$0 always improves compression ratio over the original shape while keeping error below the bound. The reported examples include Image 6 improving from about $\mathbf{r}\in\mathbb{Z}_+^E$1 to $\mathbf{r}\in\mathbb{Z}_+^E$2, Image 7 from about $\mathbf{r}\in\mathbb{Z}_+^E$3 to $\mathbf{r}\in\mathbb{Z}_+^E$4, and Image 1 from about $\mathbf{r}\in\mathbb{Z}_+^E$5 to $\mathbf{r}\in\mathbb{Z}_+^E$6 (Solgi et al., 2022).

In ER-TN structural search, the tetramer-singlet benchmark shows that the reconstructed structure can represent the tetramer singlet state exactly in numerical precision, and that replica exchange with heat-bath sampling converges much better than greedy lowest-energy selection alone. For the frustrated regime $\mathbf{r}\in\mathbb{Z}_+^E$7, the energy error is reported to decrease from $\mathbf{r}\in\mathbb{Z}_+^E$8 to $\mathbf{r}\in\mathbb{Z}_+^E$9. On random XY chains, for 8-site systems the mean relative-energy reduction rates are $\phi(G,\mathbf{r})$0 from MERA initialization and $\phi(G,\mathbf{r})$1 from ER-SDRG initialization; for 16-site systems they are $\phi(G,\mathbf{r})$2 and $\phi(G,\mathbf{r})$3, respectively. Infidelity-per-site improvements are similarly stronger for ER-SDRG than for MERA (Watanabe et al., 2024).

The program-synthesis literature emphasizes scalability and pruning efficiency. The abstract of (Guo et al., 4 Feb 2025) reports up to $\phi(G,\mathbf{r})$4 speed improvement and compression ratios $\phi(G,\mathbf{r})$5 to $\phi(G,\mathbf{r})$6 better than state of the art, together with scalability to larger tensors unattainable by prior work. Its ablation on BigEarthNet $\phi(G,\mathbf{r})$7 is especially sharp: output-directed splits plus constraint solving use only 63 sketches, whereas input-directed splits plus constraint solving produce 35,727 sketches; at $\phi(G,\mathbf{r})$8 the runtimes are $\phi(G,\mathbf{r})$9 s versus $\theta=(n_1,\ldots,n_d)$0 s, and at $\theta=(n_1,\ldots,n_d)$1 they are $\theta=(n_1,\ldots,n_d)$2 s versus $\theta=(n_1,\ldots,n_d)$3 s. Generalization experiments further report train/test CR pairs of $\theta=(n_1,\ldots,n_d)$4 on BigEarthNet and $\theta=(n_1,\ldots,n_d)$5 on PDEBench, with runtimes around $\theta=(n_1,\ldots,n_d)$6 s and $\theta=(n_1,\ldots,n_d)$7 s (Guo et al., 4 Feb 2025).

The domain-aware tnLLM line measures success primarily by evaluation efficiency. On images, videos, and time-series tensors, tnLLM achieves train objective values $\theta=(n_1,\ldots,n_d)$8, $\theta=(n_1,\ldots,n_d)$9, and $N$0 with only $N$1, $N$2, and $N$3 evaluations, respectively. The paper reports up to $N$4 fewer evaluations than TNLS, $N$5 fewer than TnALE, and $N$6 fewer than tnGPS; as an initializer, it yields up to $N$7 fewer evaluations than vanilla TNLS and up to $N$8 fewer than vanilla TnALE (Iacovides et al., 29 May 2025).

The broadest compression claims appear in HISS and TenExp. HISS reports empirical polynomial scaling with dimensionality relative to the sampling budget and compression ratios $N$9 to $N^*$0 higher than TT and HT, peaking at $N^*$1. In the thermal line-source problem at $N^*$2, TT-round yields about $N^*$3, $N^*$4, and $N^*$5, while HISS with reshaping yields about $N^*$6, $N^*$7, and $N^*$8 at representative timesteps (Guo et al., 29 Mar 2026). TenExp, by contrast, targets family selection and mixtures: on synthetic single-family cases it matches or improves on the best baselines, and on mixture-of-decompositions data it attains much smaller RE than uniformly weighted mixtures or single-family search methods; on MSIs, videos, and light-field completion it is reported to achieve the best quantitative performance across PSNR, SSIM, and RE, while top-$N^*$9 and top-$N(G)$00 variants improve further over top-$N(G)$01 (Zhou et al., 3 Mar 2026).

5. Distinct uses of the term outside tensor-network compression

In information retrieval, "Balancing the Blend" defines Tensor Search (TenS) as a multi-vector, late-interaction semantic retrieval paradigm in which each token retains its own contextual embedding. Query-document scoring is given by the MaxSim rule

$N(G)$02

That paper places TenS alongside FTS, SVS, and DVS in hybrid search, and introduces Tensor-based Re-ranking Fusion (TRF) as a way to obtain TenS-like token-level semantics without paying the full cost of running TenS as a primary retrieval path. It reports that TenS is often the most accurate single path but also the most expensive in memory and indexing cost; on MLDR(en), indexing tensors for fewer than half the documents could require up to $N(G)$03 TB, while EMVB reduces TenS memory use by up to $N(G)$04 relative to brute force (Wang et al., 2 Aug 2025).

In robotics, the related notion is not named TenS directly but is explicitly connected to earlier Tensor Search / TTGO-style methods. "Unifying Robot Optimization: Monte Carlo Tree Search with Tensor Factorization" represents a decision tree as a high-dimensional tensor, compresses it in TT form, and then combines this global tensor heuristic with MCTS. The paper characterizes TTTS as a hybrid of Tensor Search and Monte Carlo Tree Search, states the naive tree complexity as $N(G)$05, and argues that TT factorization yields a low-rank linear-complexity representation. It also gives a TT-based retrieval cost of $N(G)$06 for the TT model’s optimum and proves a bounded-error guarantee of the form $N(G)$07 (Xue et al., 7 Jul 2025).

At the level of program optimization, Prism is described as directly related to Tensor Search because it reformulates tensor-program search from concrete candidate enumeration into symbolic superoptimization over families of programs. Its sGraph representation leaves mapping and parallelization choices symbolic, verifies equivalence by e-graph rewriting, and then instantiates surviving candidates through auto-tuning. On five LLM workloads it reports up to $N(G)$08 speedup over the best superoptimizers, up to $N(G)$09 over compiler-based approaches, and up to $N(G)$10 lower end-to-end optimization time (Wu et al., 16 Apr 2026).

The phrase "tensor search" also appears in physics in a literal sense, referring to searches for tensor or mixed tensor-scalar physical signals rather than to tensorized representations. Examples include direct waveform-based searches for mixed scalar-tensor GW polarization content, yielding $N(G)$11 for GW170814 and $N(G)$12 for GW170817 (Takeda et al., 2021), and Planck PR4 searches for tensor and mixed tensor-scalar bispectra that find no significant detection, with a maximum single-template significance of about $N(G)$13 and improvements over previous bounds of roughly $N(G)$14–$N(G)$15 (Philcox et al., 2024). These usages are terminologically adjacent but methodologically separate.

6. Significance, limitations, and emerging directions

A consistent conclusion across the tensor-network literature is that structural choice matters as much as numerical optimization. The surveyed papers repeatedly show that fixing topology or tensorization a priori can leave substantial compression or accuracy on the table, while automated search over shape, permutation, topology, or decomposition family can materially change the attainable CR/RE or variational-energy frontier (Solgi et al., 2022, Watanabe et al., 2024, Zhou et al., 3 Mar 2026). This suggests that TenS should be regarded not as an optional preprocessing step but as a first-class component of tensor modeling.

At the same time, the literature is explicit about the associated difficulties. Search spaces are combinatorial; topology and rank interact strongly; local search can stall in poor basins; and practical evaluation is expensive because candidate validation often requires decomposition, contraction, or iterative fitting. These limitations motivate output-directed split pruning and singular-value upper bounds in program synthesis (Guo et al., 4 Feb 2025), mode-semantic prompting in tnLLM (Iacovides et al., 29 May 2025), multi-scale flows in RGTN (Wang et al., 31 Dec 2025), and stochastic sub-network sampling with entropy-guided clustering and reshaping in HISS (Guo et al., 29 Mar 2026). In the ER-TN setting, performance depends strongly on the initial TN structure, and the current demonstrations are at small bond dimension $N(G)$16 (Watanabe et al., 2024).

Another recurrent theme is transferability. Several papers report that discovered structures generalize to related data: the synthesis-based method obtains test compression ratios close to train ratios on BigEarthNet and PDEBench (Guo et al., 4 Feb 2025); HISS reports that a structure optimized for one data instance typically maintains compression performance within $N(G)$17 of target-specific search on related instances (Guo et al., 29 Mar 2026). A plausible implication is that TenS can often be amortized: expensive search may be performed on a representative sample and then reused across a family of similar tensors.

The current frontier is therefore moving in three directions. One is broader search spaces, including arbitrary tree networks, index reshaping, and mixtures of decomposition families (Guo et al., 29 Mar 2026, Zhou et al., 3 Mar 2026). A second is cheaper and more structured search, via symbolic reasoning, hierarchical refinement, or LLM-assisted initialization (Guo et al., 4 Feb 2025, Iacovides et al., 29 May 2025, Wu et al., 16 Apr 2026). A third is domain coupling: in disordered quantum systems, ER-SDRG preprocessing improves TenS; in real-world tensor datasets, mode semantics improve search efficiency and explanation quality (Watanabe et al., 2024, Iacovides et al., 29 May 2025). Taken together, these developments indicate that TenS has become a general methodology for automated structural adaptation in tensor methods, while remaining a term whose precise meaning is inseparable from the domain in which it is used.