Compressing multivariate functions with tree tensor networks (2410.03572v1)

Abstract: Tensor networks are a compressed format for multi-dimensional data. One-dimensional tensor networks -- often referred to as tensor trains (TT) or matrix product states (MPS) -- are increasingly being used as a numerical ansatz for continuum functions by "quantizing" the inputs into discrete binary digits. Here we demonstrate the power of more general tree tensor networks for this purpose. We provide direct constructions of a number of elementary functions as generic tree tensor networks and interpolative constructions for more complicated functions via a generalization of the tensor cross interpolation algorithm. For a range of multi-dimensional functions we show how more structured tree tensor networks offer a significantly more efficient ansatz than the commonly used tensor train. We demonstrate an application of our methods to solving multi-dimensional, non-linear Fredholm equations, providing a rigorous bound on the rank of the solution which, in turn, guarantees exponentially scaling accuracy with the size of the tree tensor network for certain problems.

• The paper introduces tree tensor networks as an efficient ansatz for compressing high-dimensional functions, surpassing traditional tensor trains.
• It presents direct and interpolative constructions, including tensor cross interpolation, to accurately represent polynomials and complex multivariate functions.
• It shows significant numerical improvements and enables solving non-linear Fredholm equations, highlighting TTNs' practical potential in computational sciences.

Overview of "Compressing Multivariate Functions with Tree Tensor Networks"

The paper "Compressing Multivariate Functions with Tree Tensor Networks" by Joseph Tindall, E. Miles Stoudenmire, and Ryan Levy explores the application of tree tensor networks (TTNs) to represent multivariate functions efficiently. The authors propose that TTNs, having a more complex topology than conventional tensor trains, can provide a more effective ansatz for certain high-dimensional functions, thus facilitating problem-solving tasks in computational sciences.

Key Concepts and Methods

Tensor networks, particularly tensor trains or matrix product states (MPS), are a compressed format for high-dimensional data. The authors generalize the use of tensor networks beyond these one-dimensional constructs to tree tensor networks, enabling the representation of multi-dimensional functions.

Direct and Interpolative Construction

The paper introduces methodologies for constructing TTNs:

• Direct Construction: The authors provide explicit constructions for elementary functions such as polynomials, with a guaranteed bound on the rank irrespective of the network's topology. This construction allows for the exact representation of one-dimensional polynomials using a TTN.
• Interpolative Construction: The tensor cross interpolation (TCI) algorithm is extended beyond tensor trains to TTNs. This algorithm allows for the adaptive approximation of a target function by querying function values at certain pivot points, efficiently interpolating the function's values into the TTN format.

Numerical Results and Comparisons

The potential of TTNs was demonstrated through comparison with tensor trains on various test functions:

1. One-Dimensional Functions: Functions like the Laguerre polynomial and the Weierstrass function were used to show that TTNs can achieve lower errors and better compression in representing complex functions compared to tensor trains.
2. Three-Dimensional Functions: TTNs were shown to significantly outperform tensor trains when representing multivariate functions, particularly in cases where inter-dimensional correlations were strong. Structured TTNs like comb tree tensor networks (CTTN) and coupled binary tree tensor networks (BTTN) emerged as superior ansatzes.
3. Tensor Cross Interpolation of Multinormal Distribution: The paper applies the TCI algorithm to learn a trivariate Gaussian distribution, demonstrating TTNs' effectiveness in handling functions with complex inter-variable correlations.

Application to Non-Linear Fredholm Equations

An innovative application of TTNs is presented in solving multi-dimensional, non-linear Fredholm integral equations. By representing both the kernel and the solution as TTNs, the proposed iterative solver can guarantee exponential scaling accuracy with manageable computational resources. This is contingent on the TTN representation of the kernel and additional functions involved, emphasizing the approach's efficiency over brute force methods.

Implications and Future Directions

The research presented has broad implications for computational methods dealing with high-dimensional data. Tree tensor networks offer a viable alternative to existing methods for function representation and problem-solving.

Future work might focus on developing heuristics for selecting optimal tree structures based on problem-specific correlations, potentially leveraging mutual information measures. Additionally, the application of TTNs may extend to more complex domains, like turbulence modeling in fluid dynamics, where traditional methods face scalability challenges.

In summary, the paper lays the groundwork for broader application and enhancement of tensor network-based methods in representing and computing high-dimensional functions, extending both theoretical and practical capabilities in computational quantum physics and beyond.