ND-TNN: Tensor-Neural-Network Approximation for High-Dimensional Nonlocal Diffusion Models

Abstract: We study a numerical method, built on the tensor neural network (TNN) architecture introduced in \cite{wang2022tensor}, for solving nonlocal diffusion models in high-dimensional spaces. The tensor-product structure of the TNN ansatz, combined with the separability of the Gaussian kernel, reduces the high-dimensional integrals in the nonlocal energy to products of low-dimensional integrals, which are evaluated by Gauss--Legendre quadrature; nonseparable source and boundary data are handled by a TNN-based preconditioning step. For the Dirichlet boundary condition, we establish the asymptotically compatible $L<sup>2$ error estimate [ |u_{\mathrm{loc}}-u_{δ,p}|{L^2(Ω)} \le C!\left(\frac{\varepsilon_f}{\sqrtδ} +\frac{\varepsilon_g}δ +\frac{\varepsilon_u}{\sqrtδ} +η{\mathrm{opt}}\right) +C\sqrtδ, ] where $\varepsilon_f$, $\varepsilon_g$ and $\varepsilon_u$ are the data and trial-class approximation errors and $η<em>{\mathrm{opt}}$ is the optimization residual. For the Neumann boundary condition, the $L<sup>2$ estimate is improved to $$O(\varepsilon_f+\varepsilon_g/\sqrtδ+\varepsilon_u +η</em>{\mathrm{opt}}+δ)$$, and an $H<sup>1$ gradient estimate is further obtained through a smoothing post-processing step. Numerical experiments on tensor-product domains up to $d=20$ support the theoretical results, and additional tests on two- and three-dimensional $L$-shaped domains demonstrate the practical robustness of the method beyond the smooth-domain setting covered by the analysis.

• The paper introduces a TNN variational solver that accurately decomposes high-dimensional nonlocal integrals into products of low-dimensional computations.
• It provides rigorous error analysis with precise L2 and H1 bounds, demonstrating nearly first-order convergence and superconvergence in experiments.
• Extensive numerical experiments confirm the method's scalability up to 20 dimensions and its robustness for both Dirichlet and Neumann boundary conditions.

ND-TNN: Tensor-Neural-Network Approximation for High-Dimensional Nonlocal Diffusion Models

Introduction

This paper presents a variational approximation scheme for high-dimensional nonlocal diffusion models leveraging a tensor neural network (TNN) architecture. Nonlocal diffusion models, which generalize classical elliptic PDEs by using integral operators to capture long-range interactions and singular phenomena, are increasingly relevant in fields such as fracture mechanics, image processing, fractional Laplacian analysis, and multiscale modeling. The key computational challenge is the evaluation of double integrals over high-dimensional product domains, which becomes computationally infeasible via direct quadrature for large dimensions.

The tensor neural network approach builds on the separation structure of the Gaussian kernel, combined with TNN's efficient representation of high-dimensional functions, enabling a reduction of these integrals to products of low-dimensional computations tractable for $d \gg 1$. The paper introduces preconditioning steps for general (nonseparable) source and boundary data through TNN surrogates, ensuring the workflow is robust beyond tensor-product scenarios.

Tensor Neural Network Architecture and Variational Workflow

The TNN architecture represents a $d$-variate function as a sum of tensor products of one-dimensional fully connected subnetworks, parameterized by separation rank $p$. It inherits the separable structure of the rescaled Gaussian kernel, which allows integrals in the variational energy functional to be decomposed into products of two-dimensional Gauss-Legendre quadrature calculations, irrespective of dimensionality:

$$u(x; \Theta) = \sum_{j=1}^p c_j \prod_{i=1}^d D_{i,j}(x_i; \theta_{i,j})$$

The variational workflow replaces the classical finite-element trial space with the TNN class and formulates energy minimization tasks for Dirichlet and Neumann boundary conditions. For nonseparable source term $f$ and boundary data $g$, TNN surrogates are obtained via empirical $L^2$ minimization before solving the variational problem.

Error Analysis and Theoretical Results

The paper provides rigorous asymptotically compatible error estimates decomposing total error into four contributions: preconditioning errors for $f$ and $g$ ($E_f$, $d$0), trial-class approximation error ($d$1), and optimization residual ($d$2). For the Dirichlet problem, the $d$3 error bound is:

$d$4

For the Neumann problem, the estimates are sharper, reflecting improved coercivity:

$d$5

An additional $d$6-gradient estimate is obtained in the Neumann case through smoothing post-processing. The error bounds are shown to be asymptotically compatible: as the errors and nonlocal horizon $d$7 tend to zero, the TNN output converges to the local PDE solution. Notably, the Neumann boundary achieves half-order better convergence in $d$8 than the Dirichlet case.

Numerical Experiments

Extensive experiments validate the theoretical results. Tensor-product domains up to $d$9 demonstrate successful minimization with residuals and errors at small values, and convergence rates for $p$0 and $p$1 errors empirically reach nearly first order in $p$2, exceeding the predicted $p$3 rate and indicating superconvergence.

For non-tensor-product data, hyperparameter studies demonstrate that TNN surrogate errors remain $p$4 RMSE for dimensions up to $p$5, confirming sustained expressiveness and scalability. Experiments on 2D and 3D L-shaped domains (beyond the covered theory) evidence practical robustness. Various architectural parameter scans reveal stable error behavior with respect to separation rank, depth, and width for moderate values, with the cost growing with dimension.

Implications and Future Directions

This work establishes TNNs as an expressive and computationally feasible trial class for high-dimensional nonlocal diffusion modeling, circumventing the curse of dimensionality for separable kernels. Theoretical error bounds and empirical evidence support practical robustness, even for irregular domains and non-separable data.

On the practical side, the ND-TNN approach is pertinent for a range of high-dimensional integral-operator PDEs relevant to computational materials science, image analysis, and machine learning. Theoretically, the observed superconvergence and robust $p$6 performance in the Dirichlet case suggest further investigation into stronger error bounds and sharper regularity assumptions.

Potential developments include: removal of the separable kernel constraint, extension to general domain topologies, rigorous analysis of observed convergence behavior, and applications to time-dependent and multiscale nonlocal models. The ND-TNN methodology also enables future integration with deep learning for operator learning and surrogate modeling in scientific computing.

Conclusion

The paper rigorously develops and numerically validates a TNN-based variational solver for high-dimensional nonlocal diffusion equations, offering sharp error bounds, efficient quadrature schemes, and practical scalability up to $p$7. The scheme is robust for both Dirichlet and Neumann boundary conditions, with empirical convergence outperforming theoretical predictions. The ND-TNN approach is significant for both computational mathematics and applied fields requiring high-dimensional nonlocal modeling, and opens avenues for further analytical and computational advancements in scientific machine learning and nonlocal PDE numerics.

Reference: "ND-TNN: Tensor-Neural-Network Approximation for High-Dimensional Nonlocal Diffusion Models" (2606.08685)