---
title: Hilbert–Galerkin Neural Operators
url: https://www.emergentmind.com/topics/hilbert-galerkin-neural-operators-hgnos
type: topic
---

# Hilbert–Galerkin Neural Operators

Hilbert--Galerkin Neural Operators (HGNOs) are neural-operator ansatzes for separable Hilbert spaces in which an input is represented through finitely many basis coordinates, processed by a finite-dimensional neural network, and re-embedded into a Hilbert-space codomain. In the formulation developed for infinite-dimensional PDEs and optimal control, HGNOs are used to solve a single infinite-dimensional PDE directly on a separable Hilbert space \(H\), rather than first discretizing the PDE into a finite-dimensional projected equation. The architecture is proved to approximate not only the target function but also its Fréchet derivative, second Fréchet derivative, and unbounded-operator-composed first-derivative terms required by fully nonlinear second-order PDEs and HJB equations on \(H\) [2603.19463]. The mathematical significance of HGNOs is tied to a broader discretization problem: when an operator \(F:X\to X\) on an infinite-dimensional Hilbert space can be approximated by finite-dimensional operators \(F_V:V\to V\) in a convergent and representation-stable Galerkin sense, and when such a program is obstructed by infinite-dimensional topology [2412.03393].

## 1. Architectural definition and Galerkin form

Let \(H_1,H_2\) be separable Hilbert spaces with orthonormal bases \(\{e_i\}_{i=1}^\infty\subset H_1\) and \(\{g_i\}_{i=1}^\infty\subset H_2\). For \(d,p\in\mathbb N\), the coordinate and embedding maps are
\[
\mathcal E_d^{H_1}(x)=\big(\langle x,e_i\rangle\big)_{i=1}^d\in \mathbb R^d,
\qquad
\widehat{\mathcal E}_p^{H_2}(y_1,\dots,y_p)=\sum_{j=1}^p y_j g_j \in H_2.
\]
The associated orthogonal projection is
\[
P_d^H x=\sum_{i=1}^d \langle x,e_i\rangle e_i
=\widehat{\mathcal E}_d^H(\mathcal E_d^H x).
\]

An HGNO is then defined by
\[
f^{d,\theta,p}:H_1\to H_2,
\qquad
f^{d,\theta,p}(x)
=
\big(\widehat{\mathcal E}_p^{H_2}\circ \tilde f^{d,\theta,p}\circ \mathcal E_d^{H_1}\big)(x)
=
\sum_{j=1}^p
\tilde f_j^{d,\theta,p}\!\Big(\big(\langle x,e_i\rangle\big)_{i=1}^d\Big) g_j,
\]
where \(\tilde f^{d,\theta,p}:\mathbb R^d\to\mathbb R^p\) is a trainable finite-dimensional neural network with parameters \(\theta\). This is explicitly a Galerkin-type representation: the input is truncated to the first \(d\) basis coordinates, and the output is expanded in the first \(p\) basis elements of the codomain. The finite-dimensional core is typically written as
\[
\tilde f^{d,\theta,p}(y)=A_{\mathfrak L}\circ \mathfrak m\circ A_{\mathfrak L-1}\circ \cdots \circ \mathfrak m\circ A_1(y),
\]
with activation \(\mathfrak m\), affine layers \(A_j\), and parameter set \(\Theta\) [2603.19463].

Several specializations are central. A scalar-valued value-function or PDE-solution HGNO has the form
\[
v^{d,\theta}\equiv v^{d,\theta,1}:H\to \mathbb R,
\qquad
v^{d,\theta}(x)=\tilde v^{d,\theta}(\mathcal E_d(x)).
\]
A control or operator HGNO is written
\[
u^{d,\phi,p}:H\to \tilde U,
\qquad
u^{d,\phi,p}(x)=\sum_{j=1}^p \tilde u_j^{d,\phi,p}(\mathcal E_d(x))\,g_j.
\]
When \(U\subsetneq \tilde U\), the output may be postprocessed by a map \(\Pi_{\mathbb R^p}\) to enforce \(u^{d,\phi,p}(x)\in U\); for \(U=B_{\tilde U}(0,1)\), one example is \(\Pi_{\mathbb R^p}(x)=x/(1+|x|)\).

The architectural interpretation is therefore fixed throughout: analysis into finitely many Hilbert coordinates, nonlinear computation in Euclidean latent space, and synthesis back into the Hilbert codomain. A plausible implication is that the term “Galerkin” in HGNO refers primarily to basis truncation and reconstruction, rather than to a weak-form layer by layer finite-element assembly.

## 2. Infinite-dimensional PDEs, controls, and derivative representation

The principal PDE class treated with HGNOs is the fully nonlinear second-order equation on a separable Hilbert space \(H\),
\[
\langle L Dv(x),x\rangle + F(x,v(x),Dv(x),D^2v(x))=0,
\qquad x\in H,
\]
where \(L:D(L)\subset H\to H\) is a closed, densely defined, possibly unbounded linear operator, \(Dv(x)\in H\) is the Fréchet gradient, and \(D^2v(x)\in S(H)\) is the bounded self-adjoint Hessian. The controlled setting is the stochastic evolution equation
\[
dX_t=[AX_t+b(X_t,u_t)]dt+\sigma(X_t,u_t)dW_t^Q,
\qquad X_0=x\in H,
\]
with value function \(V(x)=\inf_u J(x;u)\) associated with the infinite-dimensional HJB equation
\[
-\gamma v + \langle A^*Dv(x),x\rangle +\inf_{u\in U} \left\{ \langle Dv(x),b(x,u)\rangle +\frac12 \operatorname{Tr}\!\big[\sigma(x,u)Q\sigma^*(x,u)D^2v(x)\big] +l(x,u) \right\} =0.
\]
When \(Q=0\), this reduces to a deterministic first-order HJB equation; when the control is fixed or the admissible-control set is a singleton, one gets the Kolmogorov PDE [2603.19463].

The derivative calculus of scalar HGNOs is explicit. For
\[
v^{d,\theta}(x)=\tilde v^{d,\theta}(\mathcal E_d(x)),
\]
one has
\[
Dv^{d,\theta}(x) = \sum_{i=1}^d \partial_i \tilde v^{d,\theta}(\mathcal E_d(x))\,e_i,
\]
\[
D^2v^{d,\theta}(x)h
=
\sum_{i=1}^d
\left[
\sum_{j=1}^d
\partial_{ij}^2 \tilde v^{d,\theta}(\mathcal E_d(x))\,h_j
\right] e_i.
\]
If \(\{e_i\}\subset D(L)\), then \(Dv^{d,\theta}(x)\in D(L)\) and
\[
LDv^{d,\theta}(x)=\sum_{i=1}^d \partial_i \tilde v^{d,\theta}(\mathcal E_d(x))Le_i.
\]
These formulas are the basis of the “derivative-informed” character of HGNOs: once the Euclidean core is differentiable, the Hilbert-space derivatives needed by the PDE can be recovered by lifting finite-dimensional derivatives through the chosen basis.

A central obstruction is that cylindrical Hessian truncations generally do not converge in operator norm because
\[
\|I-P_d\|=1 \quad \forall d.
\]
Accordingly, the relevant topology for Hessians is weakened. On \(S(H)\), the compact-open topology is generated by seminorms
\[
s_K^{co}(Z)=\sup_{h\in K}|Zh|,
\qquad K\subset H \text{ compact},
\]
and on \(C(H;S(H))\) by
\[
p^{COCO}_{K,K'}(Z)=\sup_{x\in K,\ h\in K'} |Z(x)h|.
\]
For \(C^2(H)\), the weakened compact-open topology is generated by
\[
\mathbf p^{COCO}_{K,K'}(v) = \max\Big\{ \sup_{x\in K}|v(x)|,\  \sup_{x\in K}|Dv(x)|,\  \sup_{x\in K,\ h\in K'}|D^2v(x)h| \Big\}.
\]

Within these topologies, HGNOs satisfy several universal approximation theorems. On compact sets they are dense in \(C^2(H)\) with simultaneous approximation of \(v\), \(Dv\), and \(D^2v\). In weighted Sobolev-type norms they are dense in \(C_{w,q}^2(H)\). Under additional consistency assumptions they also approximate \(\langle LDv(x),x\rangle\). These results feed directly into residual approximation theorems for the full PDE operator
\[
(\mathcal Fv)(x):=\langle LDv(x),x\rangle + F(x,v(x),Dv(x),D^2v(x)),
\]
yielding HGNOs for which either
\[
\sup_{x\in K}|(\mathcal Fv^{d,\theta})(x)|<\epsilon
\]
or
\[
\|\mathcal Fv^{d,\theta}\|_{L^2(H;\mu)}^2
=
\int_H |(\mathcal Fv^{d,\theta})(x)|^2\,\mu(dx) <\epsilon
\]
holds. The paper further proves a bounded-inverse estimate for certain dissipative Kolmogorov-type PDEs, so that a small residual implies a small \(L^2(H;\mu)\) solution error [2603.19463].

## 3. Discretization theory, no-go phenomena, and monotone resolution

The Hilbert--Galerkin aspect of HGNOs is not exhausted by basis truncation in the ansatz; it also raises a structural discretization question. In a general Hilbert-space setting one considers a directed family \(S_0(X)\) of finite-dimensional subspaces with dense union, and a discretization family
\[
(F_V)_{V\in S_0(X)},
\qquad F_V:V\to V,
\]
subject to the approximation requirement
\[
\lim_{V\to X}\sup_{x\in \overline B_X(0,r)\cap V}\|F_V(x)-F(x)\|_X=0.
\]
A stronger form requires
\[
\sup_{x\in \overline B_V(0,r)} \|F_V(x)-P_V(F(x))\|_X \le M\epsilon_V.
\]
The canonical linear discretization is
\[
F_V=P_V\circ (F|_V):V\to V,
\]
with \(P_V\) the orthogonal projector onto \(V\). This is precisely the Hilbert-space Galerkin pipeline of restriction to a trial space, application of the continuum operator, and projection back to the trial space [2412.03393].

For bijective neural operators framed as \(C^1\)-diffeomorphisms, there is a fundamental obstruction: there exists no discretization scheme into finite-dimensional diffeomorphisms that is simultaneously convergent and continuous in the natural sense that \(F^{(j)}\to F\) implies \(F^{(j)}_V\to F_V\) for every fixed finite-dimensional \(V\). The source of the obstruction is topological. In finite dimensions, diffeomorphism groups split into orientation-preserving and orientation-reversing components; in infinite-dimensional Hilbert spaces, relevant invertible operator groups are path-connected. The proof uses a path joining identity to a reflection in infinite dimension together with finite-dimensional degree arguments. For HGNOs, the consequence is exact: invertibility or diffeomorphicity of a continuum operator does not by itself imply a well-posed discretization theory.

The positive resolution is strong monotonicity. A map \(F:X\to X\) is strongly monotone if there exists \(\alpha>0\) such that
\[
\langle F(x_1)-F(x_2),\,x_1-x_2\rangle_X \ge \alpha \|x_1-x_2\|_X^2
\qquad \forall x_1,x_2\in X.
\]
For a strongly monotone \(C^1\)-diffeomorphism, each finite-dimensional restriction
\[
P_VF|_V:V\to V
\]
remains strongly monotone and is a \(C^1\)-diffeomorphism. In finite dimensions the derivative matrices are then strictly positive definite, so determinants stay strictly positive and the orientation-switching obstruction disappears.

The neural-operator layer studied in this framework is
\[
F(x)=x+T_2G(T_1x),
\]
where \(T_1,T_2:X\to X\) are compact linear operators and \(G\in C^1(X)\). If
\[
\|DG|_x\|_{X\to X}\le \frac12 \|T_1\|_{X\to X}^{-1}\|T_2\|_{X\to X}^{-1}
\qquad \forall x\in X,
\]
then \(F\) is strongly monotone, hence a diffeomorphism. On this class the projection discretization \(F\mapsto P_VF|_V\) satisfies the approximation property and is continuous. The same paper further shows that a bilipschitz neural-operator layer can be written on bounded balls as
\[
F(x)=H_J\circ \cdots \circ H_1\circ A_0(x),
\qquad x\in B_X(0,r_1),
\]
where \(A_0\) is either \(Id\) or the reflection
\[
x\mapsto x-2\langle x,e\rangle_X e,
\]
and each factor has the form
\[
H_k(x)=x+B_k(x),
\qquad \mathrm{Lip}(B_k)<\epsilon.
\]
If \(F\in C^2(X,X)\), then \(J=\mathcal O(\epsilon^{-2})\).

This decomposition yields an architecture-level finite-rank realization. In a separable Hilbert space with orthonormal basis \(\varphi=\{\varphi_n\}\),
\[
E_Nu=(\langle u,\varphi_1\rangle_X,\dots,\langle u,\varphi_N\rangle_X),
\qquad
D_N\alpha=\sum_{n\le N}\alpha_n\varphi_n,
\]
and a residual network on \(X\) is
\[
\mathcal R_{T,N,\varphi,\sigma}(X)
=
\left\{
G=\bigcirc_{t=1}^T \bigl(Id_X + D_N\circ NN_t\circ E_N\bigr)
\right\}.
\]
Each block is a finite-rank residual perturbation of identity. The inverse of a block is obtained by contraction iteration:
\[
x_{n+1}=y+D_N\circ NN_t\circ E_N(x_n),
\qquad x_0=0.
\]
This provides a rigorous discretization-safe class of invertible Hilbert-space neural operators and a local inversion mechanism compatible with Galerkin truncation.

## 4. Whole-space residual minimization and numerical realization

The numerical methodology built around HGNOs is explicitly formulated on the whole Hilbert space rather than on a projected PDE. The target residual norm is
\[
\|\mathcal Fv^{d,\theta}\|_{L^2(H;\mu)}^2
=
\int_H |(\mathcal Fv^{d,\theta})(x)|^2\,\mu(dx),
\]
and the basic Monte Carlo loss for samples \(x_m\sim\mu\) is
\[
\widehat{\mathcal L}(\theta)
=
\frac1M\sum_{m=1}^M |(\mathcal Fv^{d,\theta})(x_m)|^2.
\]
The defining point is that the ansatz is finite-dimensional, but the residual is evaluated at points \(x\in H\), not merely at \(P_dx\). In practice, \(x\) is sampled up to a larger truncation \(N\gg d\), so that terms such as \(\langle LDv^{d,\theta}(x),x\rangle\) are evaluated using the full sampled state rather than the critic’s input truncation [2603.19463].

Two training gradients are emphasized. For Deep Hilbert--Galerkin Methods (DHGM),
\[
G_{\mathrm{DHGM}(\theta;\{x_m\}_{m=1}^M)}
=
-\frac1M\sum_{m=1}^M
\mathcal Fv^{d,\theta}(x_m)\,\nabla_\theta \mathcal Fv^{d,\theta}(x_m).
\]
For the QHPDE variant,
\[
G_{\mathrm{QHPDE}(\theta;\{x_m\}_{m=1}^M)}
=
-\frac1M\sum_{m=1}^M
\mathcal Fv^{d,\theta}(x_m)\,\nabla_\theta(-v^{d,\theta}(x_m)).
\]
For HJB equations the paper introduces Hilbert Actor--Critic, with critic \(v^{d,\theta}:H\to\mathbb R\) and actor \(u^{d,\phi,p}:H\to U\subset \tilde U\). The controlled residual is
\[
\mathcal F^{cv}(v,u)(x)
=
-\gamma v(x)+\langle A^*Dv(x),x\rangle
+\langle Dv(x),b(x,u(x))\rangle
+\frac12\operatorname{Tr}\!\big[\sigma(x,u(x))Q\sigma^*(x,u(x))D^2v(x)\big]
+l(x,u(x)),
\]
and the actor objective minimizes the current-value Hamiltonian
\[
\int_H F^{cv}(x,Dv^{d,\theta}(x),D^2v^{d,\theta}(x),u^{d,\phi,p}(x))\,\mu(dx).
\]

The numerical experiments reported in the paper concern heat and Burgers control problems on \(L^2([0,2\pi])\). For deterministic and stochastic heat equation control, the critic HGNO uses \(d=25\), one hidden layer, and 600 neurons; the actor uses \(p=d=25\), one hidden layer, and 600 neurons; training uses \(T=2\times 10^6\) iterations, \(M=2000\) Monte Carlo samples per iteration, and state samples represented to \(N=250\) basis modes, with PyTorch + CUDA. Metrics are
\[
\mathrm{ME},\ \mathrm{RMSE},\ \mathrm{RE1},\ \mathrm{RE2}
\]
computed over \(K=10^6\) sampled points. For Kolmogorov problems, both DHGM and QHPDE perform well; for HJB actor-critic, QHPDE dramatically outperforms DHGM. In one trace-class covariance-noise example, QHPDE gives critic RMSE \(9.042\times10^{-3}\) and actor RMSE \(6.146\times10^{-3}\), whereas DHGM actor-critic errors are reported around order \(1\). The derivative errors under QHPDE satisfy
\[
\|D(v^{d,\theta}-v)\|_{L^4(H;\mu)}\approx 10^{-2},
\qquad
\|D^2(v^{d,\theta}-v)\|_{(4;\mu,\mu)}\approx 10^{-2},
\]
while operator-norm Hessian errors remain much larger, consistent with the COCO-based theory.

For the controlled stochastic Burgers equation, the setup uses critic input dimension \(d=100\), state samples up to \(N=500\), an LSTM-like architecture from DGM rather than a simple feedforward network, about \(3.5\times\) more parameters than in the heat examples, QHPDE only, \(T=2\times10^6\), \(M=2000\), and \(\mu=\mathcal N(0,\operatorname{diag}(1/n^4))\). A Monte Carlo finite-difference solver on a 251-point grid with \(5\times10^4\) simulations per evaluation point is used as baseline. The reported ensemble RMSE on 100-point test sets is around \(4\times 10^{-3}\) in the stochastic case and \(9\times10^{-3}\) in the deterministic case [2603.19463].

| Problem | Setup | Reported outcome |
|---|---|---|
| Heat HJB/Kolmogorov | \(d=25\), \(p=25\), one hidden layer, 600 neurons, \(N=250\) | QHPDE generally superior; actor-critic gains especially strong |
| Heat trace-class HJB | Same base setup | Critic RMSE \(9.042\times10^{-3}\), actor RMSE \(6.146\times10^{-3}\) |
| Burgers control | \(d=100\), \(N=500\), LSTM-like DGM architecture | Ensemble RMSE around \(4\times10^{-3}\) stochastic, \(9\times10^{-3}\) deterministic |

## 5. Relation to adjacent Hilbert-space neural operator theories

Two nearby developments are highly relevant to HGNOs even though neither is presented as an HGNO theory in name. The first studies a Residually-Guided Neural Operator (RNO) for the rules-to-equilibrium map of infinite-dimensional LQ mean field games on Hilbert spaces [2510.20017]. The input “rule space” is a Hilbert space
\[
\mathcal H
\cong
\mathcal{HS}(H) \oplus \mathcal{HS}(U,H) \oplus \mathcal{HS}\big(H,\mathcal{HS}(V,H)\big),
\]
and the output space is
\[
M \equiv \mathcal M^2(\mathcal T;U).
\]
With orthonormal bases \((e_i)\) of \(\mathcal H\) and \((\eta_j)\) of \(M\), the projection and embedding maps are
\[
P_i^{\mathcal H}(x)=\big(\langle x,e_k\rangle\big)_{k=1}^i,
\qquad
E_j(\beta)=\sum_{k=1}^j \beta_k\eta_k.
\]
The RNO realization is
\[
f(x)=E_{N_2}(A_\Delta x_\Delta+b_\Delta)+y^\dagger,
\qquad
x_{l+1}=\operatorname{ReLU}(A_lx_l+b_l),
\qquad
x_0=P_{N_1}^{\mathcal H}(x+x^\dagger).
\]
The paper proves a regular universal approximation theorem in which the approximating operator preserves the same \((\alpha,L)\)-Hölder class as the target, proves local Lipschitz continuity of the rules-to-equilibrium map
\[
\|\mathfrak R(A,B,F_2)-\mathfrak R(\tilde A,\tilde B,\tilde F_2)\|_M
\le
L\|(A,B,F_2)-(\tilde A,\tilde B,\tilde F_2)\|_{\mathcal H},
\]
and derives PAC-style generalization bounds under a Karhunen--Loève decomposition with exponentially decaying coefficients. The paper explicitly notes that it is “more ‘Hilbert-projection neural operator’ than ‘Galerkin neural operator’” in the narrow sense, since it does not use an explicit weak formulation or PDE discretization-consistency theorem.

The second adjacent framework is FrameNet, developed in a statistical learning theory for nonlinear operator regression between separable Hilbert spaces [2412.17582]. The architecture factors as
\[
G=D_Y\circ g\circ S_r\circ E_X,
\]
where \(E_X\) encodes the input into frame coefficients, \(S_r\) rescales to a bounded cube, \(g\) is a finite-dimensional latent neural network, and \(D_Y\) decodes coefficients back into the output Hilbert space. The paper proves general empirical-risk-minimization results for compact operator classes \(\mathbf G\subseteq L^\infty(X,Y)\), including the mean-squared-error bound
\[
E_{G_0}\,\Big[\lVert \hat{G}_n-G_0\rVert_{L^2(\gamma)}^2\Big]
\leq
C\Big( \delta_n^2+\tilde{\delta}_n^2+\frac{\sigma^2+F_\infty^2}{n}\Big)
+
8\inf_{G^*\in G}\lVert G^*-G_0\rVert_{L^2(\gamma)}^2.
\]
Under holomorphicity assumptions on the target operator and entropy bounds of the form \(N\log(\delta^{-1})\), the sparse ReLU FrameNet class achieves
\[
E_{G_0}[\|\hat G_n-G_0\|_{L^2(\gamma)}^2]
\le
C n^{-\frac{\kappa}{\kappa+1}+\tau}.
\]
A parametric elliptic PDE example on the torus yields algebraic statistical rates for a nonlinear coefficient-to-solution map between Sobolev Hilbert spaces. Structurally, FrameNet is very close to an HGNO: fixed Hilbert encoder, finite-dimensional latent map, and decoder back to the ambient space.

Taken together, these theories show that basis projection, finite-rank truncation, regularity control, and statistical learnability are not specific to one HGNO construction. They also clarify a distinction internal to the literature: some Hilbert-space neural operators are Galerkin-like because they rely on basis truncation, while others add a stronger claim of directly targeting an infinite-dimensional PDE residual or discretization-consistent finite-dimensional realizations.

## 6. Restrictions, common misconceptions, and open problems

A persistent misconception is that continuum invertibility automatically yields principled Galerkin discretization. The no-go theorem for arbitrary Hilbert-space diffeomorphisms shows that this is false: there is no discretization rule into finite-dimensional diffeomorphisms that is both convergent and continuous in the natural representation sense [2412.03393]. A second misconception is that using a finite-dimensional ansatz implies that the target problem has been reduced to a projected PDE. In the Deep Hilbert--Galerkin framework, this is explicitly rejected: the ansatz \(v^{d,\theta}\) is finite-dimensional, but the residual minimized is that of the full PDE on \(H\), evaluated at states represented to a larger truncation \(N\gg d\) [2603.19463].

The limitations are equally clear. The PDE approximation theory is proved mainly for classical solutions \(v\in C^2(H)\) with \(LDv\in C^0(H;H)\). Approximation of unbounded-operator terms requires basis-dependent consistency assumptions. The sequential continuity assumption on the nonlinear operator \(F\) in the Hessian argument is nonstandard, and the relevant compact-open and strong-operator topologies are globally non-sequential and non-metrizable. On the discretization side, strong monotonicity is restrictive, bilipschitz factorization is local on bounded balls, the generalized neural-operator layer assumes compact linear maps \(T_1,T_2\), and inverse constructions are local and iterative rather than global closed forms. No general optimization-convergence theorem is proved, and practical performance remains dependent on basis choice, the sampling measure \(\mu\), and truncation levels.

Several open directions are identified in the source material. One is to determine how broad the class of physically relevant operators is that become strongly monotone after suitable reparameterization. Another is the choice of Galerkin spaces \(V_n\): the theory is largely basis-agnostic, but it does not prescribe optimal trial or test spaces. Sharper quantitative rates for bilipschitz decomposition and discretization remain open. The discretization of nonlinear activations \(\sigma\) in norm topology is described as subtle unless one arranges invariant finite-dimensional spaces or changes layerwise spaces. A further question is whether one can build a full operator-learning theory for HGNOs in which training is consistent across multiple Galerkin levels under monotonicity constraints. These problems delimit the current state of the subject: HGNOs now have a precise Hilbert-space ansatz, strong approximation results for PDE residuals on \(H\), and a rigorous discretization-safe subclass, but not yet a fully general theory of infinite-dimensional operator learning without structural restrictions.

Source: https://www.emergentmind.com/topics/hilbert-galerkin-neural-operators-hgnos