Open-Source Infinite-Tensor Framework

• Open-Source Infinite-Tensor Framework is a system that enables scalable, infinite-dimensional tensor computations through rigorous discretization and composable abstractions.
• It leverages advanced symmetry exploitation and block-sparse algorithms to significantly reduce computational complexity in simulations of quantum many-body systems and tensor networks.
• The framework integrates analytic inference methods and distributed parallelism, ensuring high performance for applications in field theory, infinite-width neural networks, and quantum simulations.

An open-source infinite-tensor framework is a software system that enables scalable calculations with tensors in the infinite limit or in settings where tensors encode infinite or continuum degrees of freedom. Such frameworks support applications ranging from field-theoretic inference and infinite-width neural networks to the simulation of quantum many-body systems with tensor networks on infinite lattices. They provide rigorously defined and composable abstractions for representing, manipulating, and inferring over high-dimensional or infinite-dimensional data, while leveraging advanced parallelism and symmetry reduction techniques for computational efficiency. The term "infinite-tensor" covers both literal mathematical objects (e.g., fields on manifolds, infinite tensor networks) and infinite-width limits of parametric families (e.g., neural networks as Gaussian processes).

1. Architectural Principles and Abstractions

Open-source infinite-tensor frameworks are unified by their abstraction of tensors as objects on infinite or large discrete domains, supporting both general field theory and network-based approaches.

Field-Theoretic Frameworks:

NIFTy 3 ("Numerical Information Field Theory") is constructed around a layered abstraction of domains, fields, and operators, enabling the discretization and manipulation of continuous fields independently of geometry.

• DomainObjects: Provide discretization logic and carry attributes like shape, dimension, and discrete volume weights $V_q$.
• Spaces: Subclasses such as RGSpace, HPSpace, and PowerSpace model specific geometries, each capable of harmonic transforms and mutual composition (Cartesian products).
• Fields and FieldTypes: Bundle the array data (typically distributed for HPC) with a tuple of domains. Non-scalar fields are represented via additional FieldType objects and arbitrary nesting (Steininger et al., 2017).

Neural Network Infinite-Limit Frameworks:

Neural Tangents constructs infinite-width neural networks by providing high-level, JAX-backed layer APIs whose analytical kernels represent the function-space behavior.

• Layer Factories: Each returns an initializer, apply function, and a kernel_fn for NNGP/NTK calculation.
• Compositionality: Network architectures are constructed as compositions, identical for both finite and infinite-width regimes.
• Analytic Training: Weight-space sampling is replaced by GP-based or NTK-based inference methods (Novak et al., 2019).

Tensor Network Frameworks:

YASTN targets abelian-symmetric tensor network calculations, especially for infinite projected entangled pair states (iPEPS).

• Logical Layering: Dense tensor backends (NumPy, PyTorch) are augmented by block-sparse logic encoding charge sectors.
• Legs and Tensors: Each tensor 'leg' (mode) carries symmetry charge information; the logical layer manages fusion, block-indexing, and block-sparse allocation.
• Fusion History and Symmetry Constraints: The fusion/tree-history and block-conservation logic support efficient contraction and environment computation for infinite tensor networks (Rams et al., 20 May 2024).

2. Discretization, Representation, and Geometries

A central challenge when modeling infinite or continuum objects is rigorous, geometry-agnostic discretization.

Continuous Fields:

NIFTy 3 approximates a continuum field $\varphi(x)$ by a collection of values $\varphi_q$ on a discretized grid with volumes $V_q$, so that:

$\varphi(x) \approx \sum_{q=1}^Q \varphi_q B_q(x)$

with scalar products computed with proper volume-weighted sums. The geometry is agnostic—by redefining Spaces, the same logic applies to 1D, 2D grids, spheres, or product spaces. Harmonic transforms are intrinsic, with specific transforms (FFT, spherical harmonics) linked to each space type. The framework manages all pixel volumes and normalization automatically (Steininger et al., 2017).

Infinite Tensor Networks (iPEPS):

In YASTN, each lattice site hosts a rank-5 tensor with four virtual legs (bond dimension $D$) and a physical leg. Translation invariance is achieved with a repeating unit cell. In the infinite-lattice limit, the tensor network contraction is handled via the corner transfer matrix (CTM) method, with each contraction step handled efficiently through symmetry-based block-sparse logic (Rams et al., 20 May 2024).

Infinite-Width Neural Networks:

Neural Tangents analytically constructs the Gaussian Process kernel of the infinite-width limit by defining layer-wise recurrences of covariance and tangent kernels, entirely removing the need for weight-space simulations (Novak et al., 2019).

3. Support for Symmetries and High-Rank Data

Efficient simulation and inference with infinite tensors relies crucially on exploiting symmetries and supporting arbitrary tensorial rank.

Abelian Symmetries (YASTN): Each leg of a tensor can carry an abelian charge (e.g., for $U(1)$ or $\mathbb{Z}_n$ symmetries), leading to a large reduction in computational and memory complexity. Block-sparse representation replaces a dense tensor of size $D^N$ by much smaller active charge sectors, with sector-fusion, block-lookup, and conservation rules applied for every contraction step. The environment contraction and SVD cost drops from $O(D^{12})$ to a sum over many small block-cubes $O(L_\alpha^3)$, yielding up to $30$-$100\times$ speedups in practical applications (Rams et al., 20 May 2024).

Non-Scalar or Multi-Component Fields (NIFTy 3): Any field, including vectors and tensors of arbitrary rank, is handled via the domain tuple abstraction. All operators, minimization routines, and covariance analyzers are written to be agnostic with respect to the scalar/vector distinction, supporting a wide variety of physical models in a unified interface (Steininger et al., 2017).

4. Analytic and Numerical Inference Algorithms

Frameworks provide direct support for both analytic infinite-tensor methods and large-scale numerical inference on discretized and/or symmetrized representations.

Analytic Inference (Neural Tangents):

The function-space kernels governing both Bayesian inference (NNGP) and gradient-descent (NTK) dynamics are analytically derived for arbitrary architectural depth, via recurrence relations for covariance and tangent kernels. For inputs $x,x'$, the recursion is:

• GP covariance:

$$K^{(\ell)}(x, x') = \mathbb{E}_{(u,v)\sim\mathcal{N}(0,\,\Sigma^{(\ell-1)})} [\phi(u)\phi(v)]$$

• NTK recursion:

$$\Theta^{(\ell)}(x, x') = \Theta^{(\ell-1)}(x, x')\,\mathbb{E}[\phi'(u)\phi'(v)] + K^{(\ell)}(x, x')$$

Closed-form predictors and gradient-flow trajectories are available without explicit weight manipulation (Novak et al., 2019).

Variational and Environment Algorithms (YASTN iPEPS):

Ground-state energies and thermal states for infinite 2D tensor networks are computed via:

• CTM-based environment construction, where each iteration involves contraction of corners and edges into block-diagonal matrices, SVD blockwise truncation, and update steps.
• Energy functional $$E(\{A\}) = \langle \Psi(\{A\})|H|\Psi(\{A\})\rangle/\langle \Psi(\{A\})|\Psi(\{A\})\rangle$$ is differentiated via backend (e.g., PyTorch) autograd, enabling gradient-based minimization to machine precision.
• Imaginary-time evolution leverages Suzuki–Trotter gates and re-compression, followed by standard iPEPS routines (Rams et al., 20 May 2024).

Implicit Operator Inference (NIFTy 3):

For signal inference tasks, linear and implicit operators (e.g., data response, Wiener filter curvature) are represented abstractly, supporting methods such as matrix-free conjugate-gradient inversion and stochastic Hutchinson probing for uncertainty quantification (Steininger et al., 2017).

5. Parallelism, Scalability, and Computational Backends

Maximal domain size and realistic problem complexity are addressed by integrating distributed and accelerator-aware computation into the core design.

Distributing Arrays and Operations (NIFTy 3): Numerical arrays are stored as D2O ("distributed_data_object"), partitioned across MPI processes. Local operations require no communication when the distribution is aligned; global operations invoke collective MPI calls only where needed. This structure allows seamless scaling from laptop-scale tests to HPC clusters, with companion tools (keepers) providing checkpointing and restart without explicit MPI management (Steininger et al., 2017).

Accelerator and Backend Integration (YASTN): Tensor backends can be switched, e.g., to NumPy (CPU, no AD) or PyTorch (CPU/GPU, full AD). Block-bookkeeping logic in Cython is combined with backend dispatch for dense operations (matmul, SVD, reshapes). This modularity enables advanced use cases such as AD-based optimization or GPU-accelerated contractions (Rams et al., 20 May 2024).

JAX/XLA-Powered Computation (Neural Tangents): All kernel recursions and inference steps leverage JAX's JIT compilation and vectorization, as well as out-of-the-box multi-device dispatch. Batching and sharding strategies allow near-linear scaling with the number of available GPU/TPU devices; block-diagonal methods further reduce memory and computational bottlenecks (Novak et al., 2019).

6. Performance Metrics and Benchmarks

Concrete performance metrics are documented across frameworks:

Framework	Key Scaling/Speedup	Example Benchmark
NIFTy 3	Parallel scaling via MPI, D2O	Scales to terabyte 3D reconstructions
Neural Tangents	Linear scaling with devices, $\sim$100 samples saturate device, block-diag GP: reduces $\mathcal{O}(C^3)$ to $\mathcal{O}(C)$ solves	21-layer CNN NTK on TPU: linear accel with number of GPUs/TPUs
YASTN	$10$–$100\times$ speed-up via abelian block-sparsity; allows D$\sim$36 iPEPS	30$\times$ speedup for D=8 Heisenberg (U(1)), $>100\times$ for SU(3) Kagome (Rams et al., 20 May 2024)

These improvements, particularly through symmetry exploitation and parallel/backends integration, enable applications that were previously intractable at comparable system sizes or bond dimensions.

7. Installation, Licensing, and Extensibility

All referenced frameworks are released as open-source projects under permissive licenses and are actively developed by academic research groups:

• NIFTy 3: GNU GPLv3, repository at https://gitlab.mpcdf.mpg.de/ift/NIFTy/; Python, NumPy/SciPy/healpy/mpi4py/FFTW stack (Steininger et al., 2017).
• Neural Tangents: Publicly available at https://github.com/google/neural-tangents, JAX/XLA backend with comprehensive notebook collection (Novak et al., 2019).
• YASTN: pip-installable (pip install yastn torch), hosted at https://github.com/… (see paper), leverages PyTorch/NumPy, supports Cython-extended Python (Rams et al., 20 May 2024).

Each system provides minimal working examples and maximal composability, supporting both rapid prototyping and large-scale production deployments. The layered design ensures that domain-specific algorithms (e.g., Wiener filtering, iPEPS ground states, infinite-width learning dynamics) can be reliably expressed in code closely aligned with their mathematical formulation.

8. Context and Comparative Landscape

YASTN's symmetry-preserving block logic matches or exceeds open-source alternatives such as ITensor, TenPy, Block2, and ad-peps, especially for large D, high-order tensor networks, and gradient-based optimization. NIFTy 3 enables universal manifold-agnostic Bayesian field inference. Neural Tangents stands as the reference implementation for infinite-width neural network analyticity and kernel-based training emulation. Each of these frameworks occupies a critical position in the contemporary open-source ecosystem for infinite-tensor computations, shaping both methodological development and scalable research practice in mathematical physics, quantum simulation, and machine learning.