Tensor-Train Interior Point Method

• TT-IPM is a scalable optimization framework that leverages tensor-train decompositions to represent high-dimensional convex programs with linear complexity.
• It integrates interior-point methods with adaptive rank management, using inexact TT-based Newton systems to maintain efficiency and convergence.
• TT-IPMs are applied in semidefinite programming, optimal transport, and high-dimensional PDEs, delivering high accuracy under moderate TT-rank assumptions.

The Tensor-Train Interior Point Method (TT-IPM) refers to a specialized class of optimization algorithms that leverage tensor-train (TT) decompositions within interior-point frameworks to solve high-dimensional or large-scale convex programs. TT-IPMs are particularly effective in settings where domains lack traditional sparsity or low-rank matrix structure but instead admit low TT-rank approximations, enabling scalability for problems otherwise hindered by the curse of dimensionality.

1. Mathematical Foundations of Tensor-Train Decomposition

The TT format compresses a d-dimensional tensor $$x \in \mathbb{R}^{n_1\times n_2\times\cdots\times n_d}$$ using a sequence of third-order "cores" $\{ X^{(k)} \}_{k=1}^d$, with TT-ranks $\{ r_k \}_{k=0}^{d}$ and boundary conditions $r_0=r_d=1$. The canonical decomposition writes

$$x(i_1, \ldots, i_d) = X^{(1)}_{1,\alpha_1}(i_1) X^{(2)}_{\alpha_1, \alpha_2}(i_2) \cdots X^{(d)}_{\alpha_{d-1},1}(i_d)$$

for multi-indices $i_k=1,\ldots,n_k$ and intermediate contraction indices $\alpha_k=1,\ldots,r_k$. This representation reduces the number of parameters from $\prod_k n_k$ to $O(d n r^2)$ for uniform mode sizes $n$ and ranks $r$, yielding linear scaling in both $d$ and $n$.

TT decompositions are central to TT-IPMs as they allow efficient representation and manipulation of optimization variables and constraint matrices/tensors in high-dimensional spaces; for instance, in semidefinite programming (SDP), the primal and dual variables, as well as barrier and residual computations, become tractable by TT-format manipulations (Kelbel et al., 15 Sep 2025).

2. Interior Point Method Architecture in the TT Format

Interior point methods solve constrained convex programs by maintaining iterates strictly inside the feasible set, typically by adding a barrier function to the objective and following the so-called central path. TT-IPMs adapt this paradigm to tensor variables and constraints:

• Barrier formulation: For a TT-represented variable $X$, the logarithmic barrier takes the form $$\sigma(X) = -\sum_{i_1,\ldots,i_d} \log x_{i_1,\ldots,i_d}$$, ensuring strictly positive iterates. The barrier gradient and Hessian can be calculated via TT-algebra.
• Primal-dual framework: TT-IPMs solve systems arising from perturbed KKT conditions, such as

$\nabla \sigma(X) + \mu \nabla F(X) = 0$

where $F$ encodes cost and constraints, and $\mu$ is reduced each iteration. For SDPs, both primal and dual variables are maintained and projected via TT-format operations.

• Step computation: Unlike traditional approaches, all Newton systems (for both primal and dual updates) are solved approximately within the TT-format, using inexact tensor operations to preserve efficiency and scalability.
• Feasibility and residuals: Constraints (e.g., semidefinite cones, marginal conditions in optimal transport) are formulated and checked using TT-matrices/tensors, with infeasibility and duality gaps monitored in tensor-train representation.

3. Rank Adaptivity and Approximate Subspace Enrichment

A haLLMark of advanced TT-IPM variants is their adaptive rank management—the TT-ranks of iterates may be increased ("enrichment") or decreased via TT-rounding, allowing the method to maintain approximation quality while controlling computational cost.

• Rank enrichment via steepest descent: As detailed in (Dolgov et al., 2013), after an ALS or coordinate update, the TT subspace is enriched not by merging superblocks (as DMRG would do, with $O(n^3)$ cost), but by injecting a direction closely aligned with the global residual—approximated in TT–format—into one selected TT-core. This allows "steering" each iterate toward the descent direction while keeping the computational cost $O(d n r^2)$, nearly linear.
• Riemannian rank adaptivity: Modern rank-adaptive Riemannian approaches (Vermeylen et al., 19 Feb 2024) combine tangent cone analysis and SVD-based projections to select effective rank increments, further reducing wall-clock time and improving convergence by avoiding unnecessary rank inflation.
• Inexact computations: TT-IPMs accept inexact solves, e.g., approximate Newton systems in TT-format, as long as superlinear convergence is preserved. This is critical for scalability: experiments show duality gaps below $10^{-6}$ for problems with $2^{12}$ variables in $\sim$1.5h and $<2$GB RAM (Kelbel et al., 15 Sep 2025).

4. Complexity, Scalability, and Convergence Analysis

TT-IPMs exploit the linearly scaling complexity inherent in TT representations:

• Per-iteration cost: Operations such as contraction, addition, and matrix/tensor multiplication all scale linearly with $d$ and polynomially with $n$, $r$ for typical TT-ranks; e.g., solving update equations requires $O(d n r^2)$ arithmetic operations.
• Convergence behavior: Despite inexact TT computations, the interior-point trajectory maintains moderate TT-ranks, with empirical evidence that ranks remain bounded across iterations, avoiding exponential growth even when solutions are not classically low-rank.
• Comparison to DMRG and traditional IPMs:
  • DMRG (density matrix renormalization group) improves convergence by superblock merging but at $O(n^3)$ cost; TT-IPMs, via TT-format enrichment, achieve similar contraction rates with much lower cost (Dolgov et al., 2013).
  • ALS updates are linear but may converge slowly; TT-IPMs combine ALS-style local minimization with global residual–informed enrichment for rapid error decay.
• Barriers and iterations: For tensor optimal transport, the number of iterations to $\varepsilon$-precision is $$O(\sqrt{\prod_{k=1}^d n_k} \log((\prod_k n_k)/(\varepsilon \prod_k \min_i p^{(k)}_i)))$$, showing scalability under uniform marginals (Friedland, 2023).

5. Applications in Semidefinite Programming, Optimal Transport, and Benchmark Problems

TT-IPMs have enabled scalable solutions to convex programs previously inaccessible to classical solvers:

• Semidefinite Programming: In combinatorial problems (Maximum Cut, Maximum Stable Set, Correlation Clustering), TT-IPMs solve SDP relaxations for problems up to $2^{12}$ variables, achieving duality gaps of $10^{-6}$ in practical memory and time (Kelbel et al., 15 Sep 2025). The Lovász theta function, a foundational SDP relaxation, and its extensions motivate such large-scale applications.
• Tensor Optimal Transport: For multi-marginal transfer and assignment problems, TT-IPMs operate directly on $d$-tensor constraint sets, following central-path IPM steps within TT-format and leveraging barrier functions for tractable convergence analysis (Friedland, 2023).
• Function Optimization and Completion: For general function optimization, TT-IPMs incorporating deterministic candidate selection ("beam search") find near-exact optima for up to $d=100$ dimensions and $2^{10}$ modes, with sub-minute runtime and errors below $10^{-12}$ (Chertkov et al., 2022).
• High-Dimensional PDEs: Discretized elliptic operators (e.g., Laplacians in many dimensions) can be solved efficiently by TT-IPMs, demonstrating up to $100\times$ speedup over DMRG at comparable error (Dolgov et al., 2013).

6. Contextual Significance and Theoretical Insights

TT-IPMs represent a paradigm shift in scalable convex optimization for high-dimensional settings:

• Overcoming the curse of dimensionality: By encoding optimization variables in TT-format, TT-IPMs break the exponential scaling barrier present in full-matrix/tensor approaches. This is essential for problems lacking classical sparsity.
• Latent tensor structure exploitation: Even when solution manifolds lack pure low-rank structure, moderate TT-ranks of iterates enable practical optimization as empirically verified.
• Convergence guarantees: Superlinear convergence is maintained along the central path despite inexact TT computations; theoretical contraction rates in A-norm (the energy) are comparable to steepest descent and DMRG, with additional ALS sweeps providing enhanced error decay.
• Algorithmic progression: From coordinate ALS and superblock DMRG to residual-informed enrichment and Riemannian rank-adaptivity, the TT-IPM landscape reflects methodological advances in both numerical algebra and optimization geometry.

7. Comparative Analysis, Limitations, and Future Directions

TT-IPMs are distinct from other tensor optimization approaches:

Approach	Scalability	Rank Adaptation	Applicability
ALS	Linear in $d,n$, slow	Fixed rank	General TT functions
DMRG	Cubic in $n$, fast	Adaptive (block)	Quantum/physics, low
TT-IPM	Linear/poly in $d,n,r$	Adaptive, global	SDP, OT, PDE, general

TT-IPMs are not universally optimal:

• They rely on moderate TT-rank approximability of the domain and solution; structureless or full-rank tensors may overwhelm computational resources.
• Barrier function and interior path design must account for tensor positivity and constraint enforcement.
• Extensions to fully non-convex or discrete combinatorial settings require further theoretical work.

Research continues on integration of second-order methods on TT-manifolds (Psenka et al., 2020), Riemannian trust-region algorithms, and automatic rank selection, with applications expanding in machine learning (tensorized neural networks) and scientific computing.

---

The Tensor-Train Interior Point Method synthesizes tensor decompositions and nonlinear interior-point theory for scalable, robust, and rank-adaptive optimization. It enables the solution of high-dimensional convex programs—including semidefinite relaxations and transport problems—where classical matrix-based techniques are infeasible, provided that TT-rank approximability holds across the feasible trajectory.