Functional Tensor-Based Models

• Functional Tensor-Based Model is a low-rank representation that expresses multivariate functions via continuously parameterized tensor cores.
• It leverages various decompositions such as tensor trains, Tucker, CP, and tree tensor networks to capture complex multivariate interactions efficiently.
• Optimization and inference techniques, including gradient-based, Bayesian, and Riemannian methods, ensure scalable training and accurate low-rank approximations.

A functional tensor-based model is a low-rank representation of a multivariate function, tensor-valued field, or statistical signal in which tensor-network structure is combined with mode-wise functional parameterizations, continuous indices, or functional bases rather than only discrete array entries. In the functional tensor-train formulation, for example, a scalar-valued function on a product domain is written as $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\mathcal{F}_2(x_2)\cdots \mathcal{F}_d(x_d)$$, where each $\mathcal{F}_k$ is a matrix-valued function whose entries are univariate functions learned from data (Gorodetsky et al., 2018). Related constructions replace discrete factors by Gaussian-process latent functions, Tucker cores coupled to continuous mode functions, CP components indexed by real-valued coordinates, or tree tensor networks acting on functional bases, but they share the same structural aim: exploit low multilinear complexity while preserving a genuinely functional description of the target object (Strössner et al., 2022, Fang et al., 2023, Hong et al., 2022).

1. Conceptual scope and defining principles

Across the literature, the term denotes a family rather than a single architecture. One axis of variation concerns the object being modeled. In regression and surrogate modeling, the target is a scalar-valued multivariate function $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$. In continuous-index tensor analysis, the target is a tensor-valued function whose modes include real coordinates such as latitude, longitude, time, or other continuous covariates. In PDE and quantum settings, the target is a solution field or wavefunction in continuous variables. In each case, a full tensor-product representation is replaced by a low-rank tensor network whose local factors remain functions, basis expansions, or latent processes.

A second axis concerns what “low rank” means. In functional tensor-train and functional tensor network models, the analogue of tensor-train or tree-tensor ranks controls the number of interaction channels between neighboring or hierarchically related variables. In functional Tucker and CP models, multilinear ranks or CP rank determine the size of the latent core or the number of separable components. In Bayesian formulations, rank may be fixed, adapted, or inferred during learning. A plausible unifying interpretation is that functional tensor-based models impose a structural prior on multivariate dependence: high-dimensional interactions are permitted, but only through a compressed network of lower-dimensional factors.

A third axis concerns continuity. Some models begin from continuous functional objects and only discretize for computation; others start from function evaluations on grids and recover a continuous surrogate afterward. This distinction matters because several papers explicitly separate low-rank functions from low-rank coefficient tensors, and because the ability to interpolate, impose smoothness, or operate on irregular domains depends on whether continuity is built into the model itself rather than introduced after discretization.

2. Canonical representation families

The main representation families can be organized by the underlying tensor-network topology and by how each mode is parameterized.

Family	Representative form	Representative paper
Functional tensor train (FT)	$$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$	(Gorodetsky et al., 2018)
Extended functional tensor train (EFTT)	Tucker-compressed FTT with basis functions $u_j^{(\ell)}$ and TT cores $\mathcal{H}^{(\ell)}$	(Strössner et al., 2022)
Functional Tucker	$$f(\mathbf{i})\approx \operatorname{vec}(\mathcal{W})^\top(\mathbf{U}^1(i_1)\otimes\cdots\otimes\mathbf{U}^K(i_K))$$	(Fang et al., 2023)
Functional CP	$$x_{\mathbf{i}}=\sum_{r=1}^R \prod_{k=1}^K u_r^k(i_k)$$	(Li et al., 25 Dec 2025)
Functional tree tensor network	coefficient tensor represented by an MPS or TTN over functional bases	(Hong et al., 2022, Klug et al., 10 Apr 2026)

In FT, each core $$\mathcal{F}_k(x_k)\in\mathbb{R}^{r_{k-1}\times r_k}$$ is a matrix of univariate functions $f_k^{ij}(x_k)$, so the model is a chain of matrix-valued functions rather than a chain of discrete slices. The paper distinguishes this from the more restrictive FT-c regime, where one learns a TT decomposition of a tensor of basis coefficients under a common basis per dimension. That distinction is substantive: FT allows different parameterizations in different core entries and can exploit sparsity patterns inside cores that a dense TT of coefficients cannot exploit (Gorodetsky et al., 2018).

EFTT starts from tensorized Chebyshev interpolation of a function on $\mathcal{F}_k$0, compresses the evaluation tensor by a Tucker decomposition, then compresses the Tucker core by TT. The resulting model stores only $\mathcal{F}_k$1 basis functions per mode and TT cores of size $\mathcal{F}_k$2, rather than storing all mode-wise univariate functions that a direct FTT would carry. Function evaluation then decomposes into evaluating mode bases $\mathcal{F}_k$3 and contracting the TT core chain (Strössner et al., 2022).

Functional Tucker models replace discrete factor matrices by vector-valued functions over continuous indices. In FunBaT, each mode $\mathcal{F}_k$4 has a latent function field $\mathcal{F}_k$5, and a fixed core tensor $\mathcal{F}_k$6 couples these mode-wise functions through a Tucker interaction. This formulation preserves Tucker’s multilinear core–factor semantics while making every mode evaluable at unseen real-valued coordinates (Gelfert et al., 2023). Closely related Tucker structures appear when one mode is functional rather than continuous-indexed in all modes. In functional-edged network modeling, the adjacency object is $\mathcal{F}_k$7, and the decomposition $\mathcal{F}_k$8 uses a functional basis in the third mode together with a shared node basis $\mathcal{F}_k$9 in the first two modes (Xu et al., 2024).

Functional CP models make the same transition from discrete factors to univariate latent functions. In RR-FBTC, the core representation is $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$0, where each $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$1 is a function on a real-valued index set and is modeled through a multioutput Gaussian process. In the fMRI setting, CP is used somewhat differently: the response tensor of subject-level maps is decomposed into a spatial tensor basis and subject scores, and the subject scores are then regressed on scalar covariates in a Bayesian multivariate regression (Miranda, 2024, Li et al., 25 Dec 2025).

Functional tree tensor networks generalize the same idea to hierarchical tensor-network topologies. In the many-body Schrödinger setting, a continuous wavefunction is expanded in an orthonormal product basis, and the coefficient tensor of that expansion is represented as an MPS. In the functional TTN learning framework, the coefficient tensor belongs to a tree tensor network manifold and is mapped to functions through a tensor-product feature map. The model is therefore functional in its ambient hypothesis space and low-rank in its coefficient geometry (Hong et al., 2022, Klug et al., 10 Apr 2026).

3. Training, inference, and optimization

Optimization strategies vary sharply with the loss function and with whether uncertainty is modeled explicitly. In FT regression, the empirical risk is least squares, and the principal algorithmic contribution is an efficient gradient computation based on forward and backward sweeps through the cores. Because each parameter affects only one entry of one functional core, the derivative structure is sparse. The cost of evaluating $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$2 and its gradient scales as $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$3, and for $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$4 samples the least-squares gradient costs $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$5. This supports stochastic gradient descent, ADAM, and quasi-Newton methods such as L-BFGS. On synthetic regression tasks, AAO and SGD outperform alternating least squares in the low-sample regime, often by several orders of magnitude in relative squared error (Gorodetsky et al., 2018).

Bayesian functional tensor models replace direct empirical-risk minimization by posterior approximation. FunBaT places Gaussian-process priors on the scalar latent functions $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$6, converts Matérn GPs to state-space priors through equivalent stochastic differential equations, and performs approximate inference via mean-field factorization, Conditional Expectation Propagation, Kalman filtering, and RTS smoothing. The stated overall time complexity is $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$7, with space complexity $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$8, which is the principal mechanism that makes continuous-index Tucker inference scalable (Gelfert et al., 2023).

Riemannian optimization becomes central when orthogonality or manifold constraints define the model class. Functional-edged network modeling constrains $$f:\mathcal{X}_1\times\cdots\times\mathcal{X}_d\to\mathbb{R}$$9 and $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$0 to Stiefel manifolds and formulates estimation as masked tensor completion with a roughness penalty on the functional basis. The search direction is a Riemannian conjugate gradient, vector transport is implemented by tangent-space projection, and retraction is performed by a Symmetry-HOSVD step that restores orthogonality and the shared node basis (Xu et al., 2024).

For general functional TTNs and arbitrary losses, the most explicit geometric development is the natural Riemannian gradient framework. The natural direction $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$1 is defined by $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$2, so the induced motion in hypothesis space matches the intrinsic Riemannian gradient. The paper emphasizes that this direction is independent of the choice of basis of the underlying functional tensor product space. Because the exact operator $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$3 is too expensive to invert, a hierarchy of approximations is introduced: block-diagonal natural gradients, a rank-one approximation for the multinomial logistic Fisher term, and a fully diagonal approximation with spectral scaling. Numerical experiments show that natural Riemannian gradient descent considerably improves convergence behavior compared with standard Riemannian gradient methods (Klug et al., 10 Apr 2026).

4. Rank control, regularization, and approximation theory

Rank is the principal capacity parameter in almost every formulation, but the mechanisms for handling it differ. In FT regression, the paper introduces both a rank-adaptation procedure and a group-sparsity regularizer. The regularizer acts on entire univariate functions,

$$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$4

so sparsity is imposed at the level of core entries rather than individual basis coefficients. Rank adaptation combines cross-validation with FT rounding: ranks are increased until cross-validation error worsens or rounding consistently reduces the representation. The same paper also shows that nonlinear parameterizations of the univariate functions, such as Gaussian kernels with moving centers, can achieve roughly an order of magnitude lower error than linear parameterizations on the OTL circuit problem (Gorodetsky et al., 2018).

EFTT treats rank reduction as a two-stage compression problem. Its storage complexity is $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$5, compared with $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$6 for a direct TT approximation at comparable TT ranks, and its black-box construction adapts polynomial degrees $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$7, multilinear Tucker ranks $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$8, and TT ranks $$f(x_1,\ldots,x_d)=\mathcal{F}_1(x_1)\cdots\mathcal{F}_d(x_d)$$9. The reported headline result is that the number of function evaluations required to achieve a prescribed accuracy is reduced by up to over $u_j^{(\ell)}$0 compared to the 2019 algorithm used as baseline. The same paper also notes a caveat: for some Genz functions in very high dimensions, specifically $u_j^{(\ell)}$1, the direct TT approach can be slightly more accurate at the same tolerance (Strössner et al., 2022).

Bayesian rank learning appears most explicitly in RR-FBTC. There the CP representation is overparameterized initially, each rank component is assigned a Gamma hyperprior through a diagonal precision matrix $u_j^{(\ell)}$2, and posterior shrinkage drives redundant functional components toward zero. The paper interprets the effective rank as the number of active components after inference and proves a universal approximation property for continuous multidimensional signals when universal tensor-product kernels are used in the multioutput Gaussian-process prior (Li et al., 25 Dec 2025).

Approximation theory is also explicit in FTSVD and FTTNN. FTSVD studies a third-order tensor with one functional mode and gives non-asymptotic contractive error bounds for its RKHS-based constrained power iteration under assumptions on incoherence, grid density, and signal-to-noise ratio. The error has a geometric contraction term $u_j^{(\ell)}$3 plus a statistical floor driven by the functional remainder, discretization parameter, and noise level (Han et al., 2021). FTTNN introduces a functional TT-rank defined as the supremum of discrete TT ranks over all samplings of the tensor function, then uses this rank notion inside a physics-informed loss. The accompanying approximation theorem states that for any $u_j^{(\ell)}$4 and any $u_j^{(\ell)}$5, there exist FTT-ranks and an FTTNN whose $u_j^{(\ell)}$6 error is at most $u_j^{(\ell)}$7 (Feng et al., 15 Oct 2025).

5. Application domains

The regression and surrogate-modeling literature uses functional tensor-based models as parametric alternatives to both full tensor-product expansions and nonparametric methods. FT regression compares against 22 other regression methods on 10 real-world data sets, achieves top-five accuracy for seven of the data sets, and achieves best accuracy for two of the data sets. The same model is reported as the best among the parametric models in that benchmark and competitive with the best non-parametric methods. EFTT is positioned as a black-box surrogate for high-dimensional functions arising in benchmark optimization functions, engineering models such as borehole and piston, and quantity-of-interest maps from parametric elliptic PDEs (Gorodetsky et al., 2018, Strössner et al., 2022).

Continuous-index tensor analysis is another major domain. FunBaT targets continuous-indexed tensor data such as BeijingAir pollutant measurements and US-TEMP temperature observations, where discrete Tucker decompositions are sensitive to binning and do not natively interpolate to unseen coordinates. The model learns continuous mode-wise latent factors whose posterior means and variances can be queried at arbitrary coordinates. A related but structurally different development is the functional tensor model for dynamic multilayer networks, where the logit edge-probability tensor is written as $u_j^{(\ell)}$8, with shared time-invariant vertex embeddings and RKHS-smooth, layer-specific interaction kernels. The paper emphasizes downstream uses such as vertex community detection, periodicity analysis, visualization of dynamic network evolution patterns, and evaluation of inter-layer similarity (Gelfert et al., 2023, Tang et al., 5 Sep 2025).

Network-structured and imaging applications use functional tensor decompositions to encode domain-specific invariances. Functional-edged network modeling represents metro origin–destination flows as a Tucker functional decomposition of an adjacency tensor with a continuous time mode and shared node basis, enabling tensor completion under irregular observations and extraction of interpretable temporal basis functions such as morning and evening peaks. In multi-subject fMRI, a CP decomposition of the tensor of subject-specific contrast maps yields a tensor basis over the three spatial modes; subject scores in that basis are then regressed on scalar covariates in a Bayesian model, permitting rapid MCMC and multiplicity-controlled activation inference (Xu et al., 2024, Miranda, 2024).

Scientific computing supplies several further variants. Functional tensor networks and FTTNN solve high-dimensional Schrödinger, Poisson, Helmholtz, and eigenvalue problems by representing the solution itself as a tensor network over continuous variables, with observables or physics-informed losses computed through tensor contractions and one-dimensional quadrature (Hong et al., 2022, Feng et al., 15 Oct 2025). In open quantum dynamics, the bath influence functional in the inchworm method is discretized and approximated by a tensor train, converting high-dimensional time integrals into sequential tensor contractions and yielding a deterministic alternative to Monte Carlo evaluation (Wang et al., 14 Jun 2025). For the discretized Dean–Kawasaki model, the density solving the associated Fokker–Planck equation is estimated from particle trajectories and represented by a functional hierarchical tensor on transformed simplex coordinates, allowing entropy and correlation observables to be computed by tensor contraction (Tang et al., 28 Mar 2025).

6. Misconceptions, limitations, and directions

A common misconception is that a functional tensor-based model is merely a discrete tensor decomposition applied after gridding. The surveyed formulations contradict that view in different ways. FT treats each core entry as a univariate function rather than a static coefficient. FunBaT replaces finite factor matrices by Gaussian-process latent functions over continuous indices. FTSVD models one mode directly in an RKHS. FTTNN and FTN regard the coefficient tensor as a compressed representation inside a functional hypothesis space rather than the function itself. These constructions indicate that continuity is often part of the model, not merely a preprocessing or interpolation step.

A second misconception is that optimization and rank selection are secondary engineering details. The literature shows the opposite. FT reports that ALS can be inferior to all-at-once gradient-based optimization in low-sample regression, but the 2026 functional TTN work also makes clear that least-squares-specific alternating optimization does not transfer directly to losses such as multinomial logistic regression (Gorodetsky et al., 2018, Klug et al., 10 Apr 2026). EFTT shows that additional Tucker compression can reduce storage and evaluations dramatically, yet also records cases in which direct TT can be slightly more accurate in extreme dimensions (Strössner et al., 2022). Functional hierarchical tensor density estimation for simplex-supported processes requires nontrivial coordinate transformations before tensor methods can be applied at all (Tang et al., 28 Mar 2025).

Several limitations recur. Some frameworks have not yet developed a full algebra of operations such as addition, multiplication, differentiation, and integration in their compressed functional format. Some depend strongly on basis choice, kernel choice, or initialization. Some are naturally tied to tensor-product domains, whereas others address irregular domains only through decomposition into unions of boxes or through continuous latent-function priors. This suggests a common research direction: combine geometry-aware optimization, adaptive rank control, and richer tensor-network topologies with functional representations that remain valid under irregular sampling, missingness, non-Gaussian likelihoods, and complex domains.