Spectral Normalized GP for SPINNs

• The paper introduces a spectral normalized GP module embedded within SPINNs to provide Bayesian uncertainty quantification without the cost of explicit spectral norm enforcement.
• Quantum orthogonal neural layers replace traditional spectral normalization, ensuring 1-Lipschitz continuity and scalability through inherent orthogonality.
• Numerical results demonstrate significant improvements in accuracy and calibration for high-dimensional PDE problems compared to classical methods.

A Spectral Normalized Gaussian Process (SNGP) for Separable Physics-Informed Neural Networks (SPINNs) is a recently introduced approach for embedding Bayesian uncertainty quantification into deep learning surrogates for Partial Differential Equations (PDEs). The QO-SPINN framework integrates quantum orthogonal neural layers with a SNGP module, leveraging quantum computing to achieve both computational efficiency and provable regularity. This methodology provides distance-aware uncertainty quantification with inherent Lipschitz guarantees and removes the need for explicit spectral normalization, enabling efficient and theoretically grounded application to high-dimensional PDE problems (Zanotta et al., 16 Nov 2025).

1. SPINN Architecture and Separable Ansatz

The SPINN paradigm approximates scalar PDE solutions by decomposing the network into a set of $K$ one-dimensional subnetworks and combining their outputs by a rank-$r$ canonical polyadic (CP) decomposition. For a $K$-dimensional problem, the solution representation is: $$u(x_1,\dots,x_K) = \sum_{i=1}^r \prod_{j=1}^K \phi_{i,j}(x_j),$$ where each $\phi_{i,j}:\mathbb{R}\to\mathbb{R}$ is an MLP subnetwork. For $K=2$, this reduces to: $$u(x_1,x_2) = \sum_{i=1}^r \phi_i(x_1)\,\psi_i(x_2).$$ In QO-SPINNs, each subnetwork is realized as a Quantum Orthogonal MLP, ensuring orthogonality of the linear weights. The overall CP rank $r$ governs expressivity and computational complexity, while separation across input dimensions facilitates scalable collocation ($O(N \times K)$ vs $O(N^K)$ for dense PINN collocation).

2. Spectral Normalized Gaussian Process Layer

Within the SNGP framework, the conventional final dense layer of the (residual) SPINN is replaced by a Gaussian Process prior $g \sim \mathcal{N}(0,K)$. Here, the covariance $K$ is constructed over latent representations $H = \{h_i\}_{i=1}^N$ with a kernel $k(h,h')$, typically chosen as the RBF: $k(h,h') = \exp(-\gamma \|h-h'\|^2).$ By Bochner’s theorem, this allows for spectral (random Fourier) feature approximations: $$\phi(h) = \sqrt{\frac{2}{D_L}} \cos(\sqrt{2\gamma} W_L h + b_L) \in \mathbb{R}^{D_L},$$ with $W_L$ sampled from $\mathcal{N}(0,2\gamma I)$ and $b_L$ uniform over $[0,2\pi]$. The kernel matrix is approximated as $K(H,H) \approx \Phi(H)\Phi(H)^T$ with $\Phi_{i,\ell} = \phi_\ell(h_i)$.

3. Spectral Norm Enforcement: Classical and Quantum Orthogonal Layers

In standard SNGP, spectral norm constraints ($\|W\|_2 \le 1$) on all hidden layers are necessary to guarantee 1-Lipschitzness, enforced via power iteration or SVD, incurring $O(k\,mn)$ or $O(\min(m^2 n, n^2 m))$ cost, respectively. This normalization step ensures stability and well-calibrated uncertainty.

QO-SPINNs replace each linear operation with networks of Hamming weight-preserving quantum gates (notably, real beam splitter (RBS) circuits), yielding strictly orthogonal weight matrices $W \in SO(d)$ with

$W^T W = I,\quad \|W\|_2 = 1.$

This ensures spectral norm regularization is inherent and cost-free, removing the bottleneck of explicit normalization and directly supporting distance awareness for the SNGP posterior.

4. Bayesian Inference and Predictive Uncertainty

The final SNGP module admits closed-form Bayesian linear regression over the projected random features. The posterior covariance on the GP regression weights $\beta \in \mathbb{R}^{D_L}$ is: $$\Sigma = I_{D_L} - \Phi(H) [\Phi(H)^T\Phi(H)]^{-1} \Phi(H)^T.$$ The predictive mean and variance at a test feature point $\phi(h_*)$ are: $\mu_{u_*} = \phi(h_*)^T\,\hat{\beta}$

$$\sigma_{u_*}^2 = \phi(h_*)^T \Sigma\,\phi(h_*) = k(h_*,h_*) - k(h_*,H)\,K(H,H)^{-1}k(H,h_*),$$

representing distance-aware, pointwise posterior uncertainty for the predicted PDE solution. This approach provides a built-in, theoretically motivated calibration tool absent in standard PINNs or MC Dropout methods.

5. Training, Losses, and Optimization

The training objective aggregates three terms:

• PDE residual loss: $$L_{\rm PDE} = \sum_{x\in\Omega_{\rm coll}} \|R[u(x)]\|^2$$ (using forward-mode automatic differentiation)
• Boundary/data loss: $$L_{\rm data} = \sum_{x\in\partial\Omega} \|u(x) - u_{\rm bc}(x)\|^2$$
• GP posterior (negative log marginal likelihood): $$L_{\rm GP} = \frac{1}{2} \beta^T\beta + \frac{1}{2} \|\Phi(H)\beta - y\|^2$$

The total loss is $$L = L_{\rm PDE} + L_{\rm data} + \lambda L_{\rm GP}$$. Both SPINN and SNGP parameters ($\{\theta,\beta\}$) are trained jointly using gradient descent (Adam optimizer). Kernel hyperparameters (e.g., $\gamma$) can be tuned by maximizing the marginal likelihood or via cross-validation.

Quantum acceleration arises through quantum tomography of the orthogonal layers: the forward pass in each $d$-dimensional subnet involves $O(d \log d/\epsilon^2)$ complexity versus $O(d^2)$ for classical dense layers.

6. Computational Complexity and Scaling

The following table summarizes the key computational aspects, comparing classical and quantum SNGP for SPINNs:

Architecture	Forward Pass Complexity	Spectral Norm Enforcement
Classical SNGP	$O(d^2)$	$O(k\,mn)$ (power iteration)
QO-SPINN SNGP	$O(d \log d/\epsilon^2)$	None (built-in by orthogonality)

Weight update remains $O(d^2)$ in all settings. Collocation scaling for SPINN is $O(N \times K)$, substantially improved over $O(N^K)$ in non-separable PINNs. This demonstrates the scalability and efficiency delivered by the quantum orthogonal architecture.

7. Uncertainty Quantification and Numerical Results

The SNGP posterior provides access to a pointwise standard deviation $\sigma_{u_*}$, reflecting the epistemic uncertainty of the model. To assess calibration, the Error-Aware Correlation (EAC) metric is used: $$\mathrm{EAC} = \frac{\operatorname{Cov}(\sigma, e)}{\sqrt{\operatorname{Var}(\sigma)}\sqrt{\operatorname{Var}(e)}},$$ where $e(x) = |u(x) - \hat{u}(x)|$ is the absolute error and $\sigma(x)$ is the predicted uncertainty. Higher EAC values indicate concordance between model error and predicted uncertainty.

Numerical benchmarks on the forward and inverse solution of PDEs provided the following highlights:

• 2D advection–diffusion: QO-SPINN MSE $=1.233\times 10^{-2}$ vs. SPINN $=2.264\times 10^{-1}$.
• 1D Burgers (at $t=0.25$): QO-SPINN MSE $=8.42\times 10^{-7}$, EAC $=0.7642$ versus MC Dropout EAC $=-0.0462$.
• 3D advection–diffusion: QO-SPINN MSE $=3.348\times 10^{-1}$ vs. SPINN $=1.070$.

These results indicate an order-of-magnitude improvement in accuracy and calibration using QO-SPINN SNGP over both classical SPINN and MC Dropout methods in tested regimes.

In summary, embedding spectral normalized Gaussian process layers in quantum-orthogonal SPINN architectures yields a framework that is provably 1-Lipschitz, avoids costly spectral normalization, supports theoretically grounded uncertainty quantification, and achieves quantum-accelerated scaling for PDE surrogate modeling (Zanotta et al., 16 Nov 2025).