Parallel Tensor Basis Neural Network

• Parallel Tensor Basis Neural Networks are a family of models that decompose outputs into a linear combination of fixed tensor bases and learned scalar coefficients, enabling explicit invariance and parallel computation.
• They are applied in diverse settings such as PDE solution via tensor-product subnetworks, hyperelastic constitutive modeling using analytic tensor bases, and tensorized networks that factorize neural weights.
• The design allows independent evaluation of invariants, basis tensors, and coefficient functions that are recombined through short linear contractions, enhancing computational parallelism.

Searching arXiv for the most relevant papers on tensor-basis neural networks, TPNet, and related parallel/tensorized architectures. First, I’ll retrieve the specific TPNet paper and closely related TBNN papers by title/arXiv id. The available literature suggests that a Parallel Tensor Basis Neural Network is not a single canonical architecture but a family of structured neural models in which the target quantity is written as a linear combination of basis objects and neural components provide scalar coefficients or low-dimensional features. In current arXiv work, this pattern appears in tensor-product basis networks for PDE solution, in representation-theoretic tensor basis neural networks for constitutive modeling, and in tensorized neural networks whose bond indices define latent basis channels (Yang et al., 28 May 2026, Fuhg et al., 2022, Hamreras et al., 26 May 2025). The common architectural principle is to replace a monolithic map by an expansion such as $y=\sum_a c_a(\mathcal{I})B_a$ or $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$, so that invariance, expressivity, and computational parallelism are expressed explicitly in the model form.

1. Conceptual scope and main lineages

Three research lineages dominate the current technical meaning of the topic. The first is the PDE-oriented tensor-product network, where two subnetworks generate feature vectors whose pairwise products form a basis, and the final coefficients are obtained by least squares rather than gradient descent. The second is the classical tensor basis neural network of constitutive modeling, where invariant-dependent scalar functions are learned while tensor basis generators are fixed analytically by representation theory. The third is the broader tensorized-neural-network literature, where dense layers are reshaped into higher-order tensors and factorized into tensor networks; there the basis-like role is played by factor matrices, cores, and bond-index channels.

Lineage	Basis objects	Parallel structure
TPNet for PDEs	Tensor-product basis from two subnetworks	Parallel subnetworks, batched outer products, row-wise matrix assembly
Hyperelastic or stress TBNN	Analytic tensor bases with neural coefficient functions	Parallel invariant evaluation, basis construction, coefficient evaluation
Tensorized neural networks	MPO/TT, Tucker, CP, or related tensor-network cores	Parallel rank channels, contractions, and bond-index feature paths

A central misconception is to treat these lineages as interchangeable. They are related by a shared structural decomposition, but they solve different problems and impose different constraints. In TPNet, the basis is formed from random or frozen subnetwork outputs and the coefficients are determined by a deterministic least-squares solve. In hyperelastic and turbulence TBNNs, the basis is usually prescribed by classical representation theorems and the coefficients are trained by gradient-based optimization. In tensorized neural networks, the primary object being factorized is the weight operator itself rather than a constitutive or PDE residual representation.

2. Algebraic structure of tensor-basis representations

The most general tensor-basis formulation in the cited literature writes the target tensor or field as a sum of known basis generators weighted by unknown scalar functions. In hyperelasticity, the second Piola–Kirchhoff stress is represented as

$S(C,A)=\sum_i c_i(I)\,B_i(C,A),$

where $I=\{I_k(C,A)\}$ are scalar invariants, $B_i=\partial_C I_i$ form a tensor basis, and $c_i(I)=\partial_{I_i}\Psi(I)$ are scalar coefficient functions derived from a potential. For isotropy this basis can be $\{I,C,C^2\}$; for transverse isotropy and orthotropy it is augmented by structure-tensor terms such as $N_1$, $\operatorname{sym}(CN_1)$, $N_2$, and $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$0 (Fuhg et al., 2022).

The same separation appears in turbulence modeling. Pope’s General Eddy Viscosity Model writes the Reynolds-stress anisotropy tensor as

$$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$1

where $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$2 are tensor polynomials in normalized strain and rotation and the neural network approximates the scalar functions $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$3 of the invariants $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$4. In the revisited TBNN framework for plane channel flow and square duct flow, this decomposition is retained while additional scalar inputs such as $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$5, $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$6, and $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$7 are introduced to reduce multi-valued mappings and capture non-local effects (Cai et al., 2024).

A parallel interpretation follows directly from this algebra. Basis generators are independent objects at fixed inputs; invariant evaluation is likewise separable; and the final contraction is a short sum of scalar–tensor products. This suggests that the expensive part of the computation can be split across basis elements, spatial points, or dedicated coefficient branches. In the hyperelastic literature, the potential-based model computes $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$8 by automatic differentiation of a scalar network, whereas a proposed extension replaces this with explicit parallel coefficient subnetworks $$u(\boldsymbol{x})=\sum_{m=1}^p\sum_{n=1}^p w_{mn}\phi_{1m}(\boldsymbol{x})\phi_{2n}(\boldsymbol{x})$$9, subject to compatibility constraints if hyperelasticity is to be preserved (Fuhg et al., 2022).

3. Tensor-product basis networks for PDEs

In the PDE setting, the clearest realization of a parallel tensor-basis design is TPNet. Its neural feature space is

$S(C,A)=\sum_i c_i(I)\,B_i(C,A),$0

and the approximate solution is

$S(C,A)=\sum_i c_i(I)\,B_i(C,A),$1

The distinguishing step is that the basis itself is constructed as a tensor product of two subnetwork outputs: $S(C,A)=\sum_i c_i(I)\,B_i(C,A),$2 so that $S(C,A)=\sum_i c_i(I)\,B_i(C,A),$3 and

$S(C,A)=\sum_i c_i(I)\,B_i(C,A),$4

In the randomized setting, all parameters of the two subnetworks are randomly initialized and fixed; only the linear coefficients are solved for (Yang et al., 28 May 2026).

This formulation is explicitly parallel-friendly. The two subnetworks can be evaluated independently. Basis construction reduces to batched outer products. Derivatives required by PDE operators follow the tensor-product rule

$S(C,A)=\sum_i c_i(I)\,B_i(C,A),$5

so automatic differentiation is applied only to the small subnetworks rather than to a single network with $S(C,A)=\sum_i c_i(I)\,B_i(C,A),$6 outputs. The least-squares system for collocation,

$S(C,A)=\sum_i c_i(I)\,B_i(C,A),$7

is assembled row-wise from interior and boundary collocation points; each row is independent once the basis evaluations are available.

The paper positions this design against PINNs and large random-feature baselines. For 2D Helmholtz with DoF $S(C,A)=\sum_i c_i(I)\,B_i(C,A),$8, TP-ResNet achieves $S(C,A)=\sum_i c_i(I)\,B_i(C,A),$9 and $I=\{I_k(C,A)\}$0 in about $I=\{I_k(C,A)\}$1 s, while FEM with about $I=\{I_k(C,A)\}$2k DoF reports $I=\{I_k(C,A)\}$3 in $I=\{I_k(C,A)\}$4 s, HLConcELM reports $I=\{I_k(C,A)\}$5 in $I=\{I_k(C,A)\}$6 s, RFM reports $I=\{I_k(C,A)\}$7 in about $I=\{I_k(C,A)\}$8 s, and RNN-PG is less accurate with $I=\{I_k(C,A)\}$9 s. For long-time diffusion with block time-marching, TP-ResNet plus BTM reaches $B_i=\partial_C I_i$0 in about $B_i=\partial_C I_i$1 s, whereas HLConcELM plus BTM requires about $B_i=\partial_C I_i$2 s with larger errors (Yang et al., 28 May 2026).

The same work extends the architecture to nonlinear and long-time problems. Nonlinear PDEs are handled by Picard iteration, treating the nonlinear term as a known source at each step; the matrix $B_i=\partial_C I_i$3 is assembled once and only the right-hand side is updated. Long-time evolution can be handled either by including time as an input or by block time-marching, where the solution at the end of one block supplies the initial condition for the next. The paper does not implement an explicit distributed algorithm, but it identifies parallel evaluation of the subnetworks, tensor-product assembly, pointwise nonlinear-source evaluation, and accelerated least-squares solves as natural computational pathways.

4. Tensor basis neural networks for hyperelastic constitutive modeling

In hyperelasticity, the tensor basis neural network is grounded in objective representation theory rather than in tensor-product random features. Material symmetry is encoded by structure tensors, and the constitutive law is reduced to scalar invariants. The free energy is written as $B_i=\partial_C I_i$4, with $B_i=\partial_C I_i$5 the mutual invariants of strain and structure tensors. The stress then takes the representation

$B_i=\partial_C I_i$6

This splits the constitutive problem into an analytic part, carried by the tensor basis $B_i=\partial_C I_i$7, and a data-driven part, carried by the scalar coefficient functions $B_i=\partial_C I_i$8 (Fuhg et al., 2022).

The architecture described in the literature contains a deterministic tensor-basis preprocessing layer, a dense feed-forward potential network, and an automatic-differentiation step that extracts the coefficients as derivatives of the scalar potential. For orthotropy, the full basis is

$B_i=\partial_C I_i$9

Two trainable scalars $c_i(I)=\partial_{I_i}\Psi(I)$0 weight the anisotropic blocks,

$c_i(I)=\partial_{I_i}\Psi(I)$1

and an $c_i(I)=\partial_{I_i}\Psi(I)$2 penalty on $c_i(I)=\partial_{I_i}\Psi(I)$3 promotes sparse activation of anisotropy modes. Rotation parameters $c_i(I)=\partial_{I_i}\Psi(I)$4 make the structure tensors learnable, so the model can identify both anisotropy type and anisotropy orientation from stress–strain data (Fuhg et al., 2022).

A polyconvex extension replaces the potential network by an input-convex neural network and uses a polyconvex invariant set together with the analytic growth term

$c_i(I)=\partial_{I_i}\Psi(I)$5

The cited work states that, by satisfying the growth condition, this can guarantee the existence of boundary value problem solutions. That guarantee is one of the main technical differences between the potential-based hyperelastic TBNN and unconstrained coefficient regression (Fuhg et al., 2022).

The architecture is structurally parallel even when trained as a single scalar-potential network. Invariant computation is independent per invariant, basis construction is independent per basis tensor, and the final tensor sum is a small reduction. The paper explicitly notes a possible extension in which separate coefficient subnetworks $c_i(I)=\partial_{I_i}\Psi(I)$6 are evaluated in parallel. A parallel design of that type would be straightforward computationally, but hyperelastic consistency would then require additional integrability constraints so that the learned coefficients still derive from a scalar potential.

5. Stress representations, turbulence closures, and physically optimal bases

The constitutive-model literature stresses that not every tensor-basis representation is equally effective numerically, even when the representations are theoretically equivalent. A systematic comparison of nine TBNN variants for isotropic hyperelasticity examines Rivlin-type, Criscione-type, orthogonalized, monotone, convex-potential, and St. Venant-like formulations. The main conclusion is that Convex-Pot and Mono-Coeff show the lowest test RMSE across materials and noise levels, Rivlin-Pot is close behind, orthogonal bases generally underperform the Rivlin variants on Mooney–Rivlin and modified Carroll but do relatively better on the Gent model, and StV-Coeff performs very poorly because the coefficient functions become too complex for the basis choice (Fuhg et al., 2023). This is a direct warning against equating “more orthogonal” or “more geometric” with “better learnability.”

A related issue arises in turbulence modeling, where the physically admissible tensor basis depends on flow class. For plane channel flow, the revisited TBNN framework shows that the proper model is not the full 3D GEVM but a reduced 2D GEVM with effectively one independent invariant and a small tensor basis. The paper further introduces a generalized constant tensor

$c_i(I)=\partial_{I_i}\Psi(I)$7

and reduces the plane-channel basis to $c_i(I)=\partial_{I_i}\Psi(I)$8 and $c_i(I)=\partial_{I_i}\Psi(I)$9, with scalar inputs $\{I,C,C^2\}$0 (Cai et al., 2024). For square duct flow, the same work uses the 3D GEVM augmented by a constant tensor and finds numerically that five basis tensors plus one constant tensor are sufficient for the studied case.

These turbulence TBNNs are explicitly designed for deployment across many grid points. At each point, one computes local gradients, forms $\{I,C,C^2\}$1 and $\{I,C,C^2\}$2, evaluates invariants, constructs the basis tensors, feeds a small MLP to obtain the coefficients, and assembles

$\{I,C,C^2\}$3

For plane channel flow, a shallow architecture with three hidden layers of ten neurons gives accurate predictions and generalizes to unobserved $\{I,C,C^2\}$4 in both interpolation and extrapolation scenarios. For square duct flow, a deeper architecture with ten hidden layers of fifty neurons is used; GELU and batch size $\{I,C,C^2\}$5 are identified as the best performance–cost trade-off. In the transfer-learning experiment at $\{I,C,C^2\}$6, fine-tuning from a source model with only $\{I,C,C^2\}$7 of the target training data reaches global $\{I,C,C^2\}$8, compared with about $\{I,C,C^2\}$9 without transfer learning (Cai et al., 2024).

6. Parallelization mechanisms, broader tensorized architectures, and limitations

Across the cited literature, parallelization is enabled by structural decomposition rather than by a single universal training algorithm. In TPNet, the two subnetworks can be placed on different devices, tensor-product basis formation is a batched outer-product operation, nonlinear-source evaluation in Picard iteration is embarrassingly parallel, and least-squares assembly is row-wise independent. In hyperelastic and turbulence TBNNs, invariant evaluation, basis-tensor construction, and coefficient inference are pointwise local and therefore naturally batched over quadrature points, material points, or CFD cells (Yang et al., 28 May 2026, Cai et al., 2024).

A broader but distinct architectural interpretation comes from tensorized neural networks. There, tensorization means reshaping dense weight matrices into higher-order tensors and approximating them by tensor-network decompositions such as MPO/TT, Tucker, or CP. In this setting, the basis-like entities are factor matrices, local cores, and especially bond indices, which the literature describes as new latent spaces not found in conventional networks. The paper’s stack view interprets an MPO layer as a composition of sparse fully connected layers, and bond-index values become internal feature channels that can be processed in parallel or analyzed as basis trajectories through the layer (Hamreras et al., 26 May 2025). This does not make tensorized networks identical to classical TBNNs, but it does provide a rigorous architectural vocabulary for “parallel tensor basis” computation.

Several limitations recur. TPNet reports possible ill-conditioning of large least-squares systems, accuracy plateaus as basis size grows, architecture sensitivity, and degraded performance in higher dimensions; it also notes that parallelization and advanced tensor structures are well suited to the method but are not implemented explicitly (Yang et al., 28 May 2026). Hyperelastic and stress-representation TBNNs show that theoretical equivalence of bases does not imply equal numerical behavior: basis choice changes coefficient complexity, conditioning, and generalization, and coefficient extraction can become ill-conditioned near repeated eigenvalues (Fuhg et al., 2023). Tensorized neural networks face separate challenges, including hardware and software support for general contractions, large hyperparameter spaces involving topology and bond dimensions, and incomplete understanding of their inductive bias in mainstream deep learning workloads (Hamreras et al., 26 May 2025).

A final misconception is to reduce “parallel” to multi-GPU execution alone. In this literature, parallelism is more fundamental: it refers to decompositions in which invariants, basis generators, coefficient functions, tensor-product feature blocks, or bond-index channels are independent enough to be evaluated concurrently and then recombined by a short linear contraction. The topic is therefore best understood as a structured modeling paradigm in which tensor bases organize both physics and computation.