Tensor Train–Karhunen–Loève Framework

• The Tensor Train–Karhunen–Loève framework is a computational method that combines the classical Karhunen–Loève expansion with tensor-train decompositions for efficient analysis of continuous-indexed random fields.
• It eliminates computational bottlenecks by avoiding mesh, basis selection, and large-scale eigenvalue problems, enabling adaptive computation of both second- and third-order cumulants.
• The method employs an adaptive TT decomposition algorithm to achieve significant compression and scalability, outperforming traditional Galerkin and collocation approaches in high-dimensional settings.

The Tensor Train–Karhunen–Loève (TT–KL) framework is a theoretical and computational architecture for the representation of continuous-indexed random fields over multidimensional domains. It unifies the classical Karhunen–Loève (K–L) expansion with modern tensor-train (TT) decompositions of higher-order cumulant functions, enabling efficient, adaptive, and mesh-free computation of both second- and third-order statistics for Gaussian and non-Gaussian random fields. The TT–KL framework eliminates basis selection and large-scale eigenvalue problems, exhibiting robust adaptivity and offering significant computational advantages over traditional Galerkin or collocation methods, especially in high dimensions (Bu et al., 2019).

1. Classical Karhunen–Loève Expansion and Its Limitations

Given a second-order random field $\omega(x, \theta)$ on domain $D \subset \mathbb{R}^d$ with zero mean and covariance $$C(x, x') = \mathbb{E}[\omega(x, \theta)\omega(x', \theta)]$$, the K–L expansion represents the field as

$$\omega(x, \theta) = \sum_{n=1}^\infty \sqrt{\lambda_n}\,\phi_n(x)\,\xi_n(\theta),$$

where $\{ \lambda_n, \phi_n \}$ are the eigenvalue and eigenfunction pairs of the covariance integral operator, and $\{\xi_n\}$ are uncorrelated random coefficients with unit variance. In multiple dimensions ($d \geq 2$), this formulation requires solving extremely large-scale eigenvalue problems, typically via basis selection (e.g., Galerkin) or collocation and assembly of large stiffness matrices. Furthermore, the K–L expansion is limited to second-order (covariance) statistics and cannot natively represent non-Gaussian structure that relies on higher-order cumulants (Bu et al., 2019).

2. Tensor-Train Representation of Cumulant Functions

To address the computational obstacles of the K–L approach and to capture higher-order statistics, the TT–KL framework reformulates cumulant functions in a structured parametric space $[0,1]^m$, mapped from physical coordinates by an isogeometric mapping $x = \mathcal{X}(\xi)$. The $K$th-order cumulant $C_K(\xi_1,\ldots,\xi_K)$ is a smooth real function on $[0,1]^{mK}$, which is reordered as $C_K(u_1,\ldots,u_a)$ with $a = mK$.

A low-rank tensor-train approximation of $C_K$ takes the form

$$C_K(u_1,\ldots,u_a) \approx \sum_{\alpha_0,\ldots,\alpha_a} G^{(1)}_{\alpha_0,i_1,\alpha_1} \cdots G^{(a)}_{\alpha_{a-1},i_a,\alpha_a},$$

where each TT core $G^{(k)}$ is of dimension $r_{k-1} \times I_k \times r_k$ ($r_0 = r_a = 1$) and $i_k$ indexes discretization in the corresponding direction. This structure yields significant compression and enables efficient manipulation of the high-dimensional cumulant function (Bu et al., 2019).

3. Adaptive Rank-Revealing TT Decomposition Algorithm

The TT–KL framework utilizes an adaptive cross-approximation algorithm, in line with the Savostyanov–Oseledets approach, for constructing the TT decomposition. The algorithm operates by adaptively selecting interpolation sets in each TT core direction and iteratively refining these sets based on the maximal residual error until a prescribed tolerance is achieved.

Algorithmic steps:

1. Initialization: Choose the starting pivot $u^{(0)} = \arg\max_u |G(u)|$ from quasi-random samples.
2. Sweep Phases: Alternate left-to-right and right-to-left sweeps across core indices, expanding the interpolation sets for dimensions where the maximum error exceeds tolerance.
3. Reconstruction: For each $k=1,\ldots,a$, compute the TT core by appropriate conditional matrix inversions on the selected interpolation sets.
4. Convergence: Terminate when the maximum core error falls below the tolerance.

The global TT approximation error is measured by

$$\epsilon_g = \frac{\sqrt{\frac{1}{N} \sum_{i=1}^N (C(u_i) - C_{\text{TT}}(u_i))^2}}{\sqrt{\frac{1}{N} \sum_{i=1}^N C(u_i)^2}}.$$

(Bu et al., 2019)

4. Computation of Second- and Third-Order Cumulants

Second-Order (Covariance) TT Decomposition

For the second-order cumulant ($K=2$), the TT decomposition reduces to computing two TT cores, yielding an approximation

$$C_2(\xi,\xi') \approx M_1(\xi)_{1 \times r} M_2(\xi')_{r \times 1}.$$

The univariate spectral problem is solved using QR decompositions and a small SVD on matrices of size $O(r)$, producing mode functions $f(\xi)$ and eigenvalues $\lambda$.

Third-Order (Non-Gaussian) Cumulant TT Decomposition

For $K=3$, following TT decomposition,

$$C_3(\xi, \xi', \xi'') \approx \hat{G}_1(\xi) \cdots \hat{G}_{3m}(\xi''),$$

the third cumulant of the latent factors $\gamma_i(\theta)$ is expressed as an integral over TT-compressed cores,

$$\breve{C}_3(i,j,k) = \int_{[0,1]^{3m}} F_i(\xi)F_j(\xi')F_k(\xi'') \,C_3(\xi,\xi',\xi'')\,d\xi\,d\xi'\,d\xi''.$$

The result is a highly compressed representation $\breve{C}_3 = A_1 \times^1 A_2 \times^1 A_3$, reducing storage and computation costs.

Dimension reduction of latent variables $\{\gamma_i\}$ uses HOSVD on the super-symmetric $\breve{C}_3$, with subsequent eigenvalue problem and transformation to new latent factors $\gamma_{[3]}$ ensuring the compressed cumulant structure is preserved (Bu et al., 2019).

5. Computational Comparison with Galerkin and Collocation Schemes

The TT–KL framework eliminates several key bottlenecks of classical methods:

• No mesh, basis, or quadrature selection: Modes and interpolation sets are chosen adaptively, leveraging Chebfun and TT interpolation.
• No large-scale eigenproblems: Only SVDs of size $O(r)$ are solved; $r \approx 10$–100 in typical cases.
• Automatic adaptivity: Interpolation sets and TT ranks self-tune to meet accuracy requirements.
• Computational complexity: TT decomposition scales as $O(a\,r^3 + S\,m_k\,r^2)$, with each SVD $O(r^3)$.
• Eigenpair extraction: Reduced to moment computations and low-rank algebra.

This methodology allows for user-prescribed accuracy control and typically exhibits computational costs orders of magnitude below finite-element K–L expansions (Bu et al., 2019).

6. Numerical Results and Performance

Three test cases illustrate the scalability and accuracy of the TT–KL framework.

Example	Domain/Dim.	TT-Ranks	Second-Order Error $\epsilon_{g2}$	Third-Order TT-Rank	Latent Dim. Reduction	Computational Times
1	$d=1$	80	$\sim 10^{-12}$	(80, 80)	80 $\to$ 5	TT-cov: 100s, modes: 440s
2	$d=2$	(8, 37, 8)	$2 \times 10^{-7}$	(8, 41, 129, 38, 8)	37 $\to$ 11	TT-cov: 30s, modes: 300s
3	$d=3$	(4,21,60,21,4)	$3 \times 10^{-6}$	(3,20,60,110,164,56,35,8)	60 $\to$ 17	TT-cov: 200s, modes: 2000s

These results confirm automatic accuracy control relative to prescribed tolerances and compressions, with costs far below classical strategies. For example, in $d=1$, exact TT-rank 80 achieves error $\epsilon_g \approx 10^{-12}$ and third cumulant latent dimension is reduced $80 \to 5$, while higher dimensions show similar gains (Bu et al., 2019).

7. Significance and Prospects

The TT–KL framework offers a rigorous, scalable approach to the representation and analysis of high-dimensional non-Gaussian random fields. By combining the spectral compactness of the K–L expansion with modern low-rank tensor techniques applied directly to cumulant functions, it enables new directions in stochastic modeling, uncertainty quantification, and data-driven partial differential equation input analysis. The elimination of mesh and basis selection, together with demonstrated scalability in three-dimensional test cases, suggests strong potential for large-scale engineering and scientific computations where higher-order statistics are relevant (Bu et al., 2019).