GeoFAE: Geometric Autoencoder for Inverse Problems

• GeoFAE is a geometric function autoencoder that encodes complex fields on irregular domains into fixed-length latent representations.
• It employs a Perceiver-style encoder and a coordinate-based CViT-style decoder to process point-cloud data and enable continuous field reconstruction.
• The design circumvents mesh-dependent issues, achieving state-of-the-art accuracy on PDE benchmarks while facilitating inverse problem solving.

Geometric Function Autoencoder (GeoFAE) denotes a geometric function autoencoder introduced as the representation layer of GeoFunFlow for inverse problems on complex geometries. For a geometry $\Omega\subset\mathbb{R}^d$ and a physical field $u:\Omega\to\mathbb{R}^p$, it learns an encoder–decoder pair

$$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$

where $\mathbf{c}$ encodes the geometry as a point cloud plus sparse measurements, $\mathbf{z}$ is a geometry-aware latent code, and $q\in\Omega$ is any query point. The defining property of GeoFAE is that it encodes an entire field on an irregular domain into a fixed-length latent vector and decodes that vector into a continuous neural field evaluable at arbitrary coordinates; within GeoFunFlow, rectified flow then acts only in latent space rather than directly on discretized meshes (Wang et al., 28 Sep 2025).

1. Conceptual scope and problem setting

GeoFAE is formulated for geometric functions $u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$, $s\ge 1$, defined over sample-dependent geometries. The conditioning object is $\mathbf{c}=(\Omega,X,\mathbf{y})$, where $\Omega$ is the geometry, $u:\Omega\to\mathbb{R}^p$0 is the sensor set, and $u:\Omega\to\mathbb{R}^p$1 are noisy readings. The global target in GeoFunFlow is a conditional probabilistic operator

$u:\Omega\to\mathbb{R}^p$2

mapping conditioning instances to posteriors over fields on $u:\Omega\to\mathbb{R}^p$3. GeoFAE supplies the fixed-dimensional representation required to make this operator tractable on irregular domains (Wang et al., 28 Sep 2025).

The motivation is operational rather than purely notional. Direct diffusion or flow modeling on discretized fields over unstructured meshes is problematic because mesh sizes vary by orders of magnitude, mesh connectivity and geometry vary across samples, and a fixed tensor layout does not exist. GeoFAE addresses this by compressing each field and its geometry into a latent vector $u:\Omega\to\mathbb{R}^p$4 whose size is independent of mesh size and layout, while preserving continuous decoding back to the original domain (Wang et al., 28 Sep 2025).

In a broader geometric reading, GeoFAE belongs to the class of autoencoders understood as jointly learning a manifold and a coordinate chart. That viewpoint emphasizes that autoencoders do not merely compress data: they choose an embedded manifold through the decoder image and a coordinate system through the encoder–decoder pair, which is precisely the interpretive frame within which geometric regularization, latent metrics, and latent manifolds become meaningful design objects (Lee, 2023).

2. Representation of geometry, observations, and fields

Each geometry $u:\Omega\to\mathbb{R}^p$5 is discretized into a point cloud

$u:\Omega\to\mathbb{R}^p$6

with $u:\Omega\to\mathbb{R}^p$7 varying across samples and ranging from $u:\Omega\to\mathbb{R}^p$8 to $u:\Omega\to\mathbb{R}^p$9 in the reported experiments. Sensors form a subset $$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$0, and the observation pattern is encoded by the node-wise mask

$$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$1

At each node $$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$2, the feature is

$$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$3

so values appear only where $$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$4, and the conditioning instance presented to the encoder is

$$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$5

(Wang et al., 28 Sep 2025).

GeoFAE is explicitly mesh-agnostic in the sense used by the paper. It does not assume any grid structure; geometric information is conveyed through coordinates $$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$6, embedded with random Fourier features, and there is no dependence on graph connectivity or regular lattice. This is a specific design choice: the model treats the domain as a point cloud with coordinate embeddings, and its geometric expressivity comes from set processing and coordinate-conditional decoding rather than from explicit mesh operators (Wang et al., 28 Sep 2025).

A common misconception is to equate GeoFAE with a discrete mesh autoencoder that reconstructs nodal arrays at a fixed resolution. The decoder instead defines a continuous reconstruction. For any query set $$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$7, the latent representation is evaluated at those coordinates, so the same latent code can be queried on the training mesh, on a refined mesh, or at sensor locations chosen after inference. This continuous-query formulation is central to its role in inverse problems, where sensor configurations and target resolutions vary across instances (Wang et al., 28 Sep 2025).

3. Encoder–decoder architecture

GeoFAE uses a Perceiver-style encoder followed by a CViT-style decoder. At the encoder input, each node $$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$8 receives a coordinate embedding

$$\mathcal{E}_\theta:\mathbf{c}\mapsto \mathbf{z}\in Z=\mathbb{R}^\ell,\qquad \mathcal{D}_\phi:(\mathbf{z},q)\mapsto \hat u(q),$$9

where $\mathbf{c}$0 has entries sampled from $\mathbf{c}$1, together with an MLP embedding of $\mathbf{c}$2. These are concatenated into node tokens $\mathbf{c}$3, stacked as $\mathbf{c}$4 (Wang et al., 28 Sep 2025).

Compression from a variable-size point set to a fixed representation is handled by $\mathbf{c}$5 learnable latent queries $\mathbf{c}$6, which attend to the node tokens through Perceiver cross-attention: $\mathbf{c}$7 followed by

$\mathbf{c}$8

The aggregated tokens are then refined by $\mathbf{c}$9 pre-norm Transformer blocks with multi-head self-attention over the $\mathbf{z}$0 latent tokens, producing $\mathbf{z}$1. This token set serves two roles: it is the encoder feature set supplied to the decoder by cross-attention, and it is aggregated into the latent vector $\mathbf{z}$2 used by the latent flow model (Wang et al., 28 Sep 2025).

The decoder is a coordinate-based neural field. Query coordinates $\mathbf{z}$3 are embedded by the same type of Fourier mapping into query tokens $\mathbf{z}$4. These are updated through $\mathbf{z}$5 cross-attention blocks with keys and values from $\mathbf{z}$6: $\mathbf{z}$7

$\mathbf{z}$8

followed by a final MLP $\mathbf{z}$9 giving

$q\in\Omega$0

Equivalently, the decoder defines an operator

$q\in\Omega$1

so continuous evaluation is part of the model definition rather than a post-processing step (Wang et al., 28 Sep 2025).

Training of the autoencoder stage uses random query points $q\in\Omega$2 and the reconstruction loss

$q\in\Omega$3

Instances and query points are resampled at each iteration. The paper states that no explicit PDE residual or boundary-term is used; GeoFAE is therefore a purely data-driven geometric function autoencoder, even though it is trained on PDE solution families (Wang et al., 28 Sep 2025).

4. Latent space, rectified flow, and posterior reconstruction

GeoFAE is the representation engine of GeoFunFlow, not the diffusion model itself. After GeoFAE is trained, it is frozen, and latent rectified flow is trained on its codes. The encoder produces $q\in\Omega$4, the latent encoding of the full field on the full point cloud, and $q\in\Omega$5, the latent encoding of partial noisy observations. With $q\in\Omega$6 and $q\in\Omega$7, the interpolation

$q\in\Omega$8

is used to train a conditional velocity field $q\in\Omega$9 to satisfy

$u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$0

under the latent CRF objective

$u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$1

(Wang et al., 28 Sep 2025).

Inference proceeds in three stages. First, the conditioning instance is encoded to $u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$2. Second, latent sampling is performed by solving the ODE

$u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$3

whose terminal state $u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$4 is a sample from the model posterior over latents. Third, the sampled latent is decoded at any desired query set: $u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$5 Repeated latent sampling yields posterior samples and uncertainty quantification in function space (Wang et al., 28 Sep 2025).

The paper also provides a function-space error bound that makes the decoder geometry explicit. If the decoder is Lipschitz in latent space,

$u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$6

if the latent flow approximation error satisfies $u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$7, and if the autoencoder reconstruction error is $u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$8, then

$u\in\mathcal{U}(\Omega)=H^s(\Omega;\mathbb{R}^p)$9

With additional Sobolev regularity $s\ge 1$0 and quasi-uniform sensors, the paper further states

$s\ge 1$1

where $s\ge 1$2 is the fill distance of sensors. This makes decoder smoothness and autoencoder fidelity structurally central to posterior accuracy (Wang et al., 28 Sep 2025).

A second misconception is that GeoFAE is already physics-informed because it operates on PDE solution families. The paper is explicit that GeoFAE is purely data-driven and that future work could integrate PDE constraints or physics-informed losses into GeoFAE training (Wang et al., 28 Sep 2025).

5. Relation to the broader literature on geometric autoencoders

GeoFAE sits within a broader body of work that treats autoencoders geometrically, but it occupies a specific niche: continuous function encoding and decoding over irregular geometries. The general geometric perspective states that an autoencoder learns both a manifold and a coordinate chart, and it also highlights the non-uniqueness of both manifold and chart under reconstruction loss alone. That perspective motivates explicit geometric regularization and helps explain why GeoFAE’s architecture treats geometry as a first-class modeling object rather than as an incidental property of the latent code (Lee, 2023).

Several nearby lines of work illustrate alternative mechanisms for geometry preservation. Geometry-Regularized Autoencoders regularize the bottleneck of an autoencoder toward a precomputed PHATE embedding so that the learned latent representation follows intrinsic data geometry while retaining out-of-sample extension and reconstruction (Duque et al., 2020). Multi-Scale Geometric Autoencoder imposes global distance constraints on the encoder and local Jacobian constraints on the decoder, explicitly separating global and local geometry in an asymmetric design (Zhan et al., 29 Sep 2025). Convergent low-bending, low-distortion autoencoder embeddings regularize an encoder using local geodesic distances and Riemannian midpoints so that the latent embedding approaches an isometric, extrinsically flat manifold embedding (Braunsmann et al., 2022). These works suggest a shared theme—control of geometry through distances, Jacobians, or manifold structure—even though GeoFAE itself realizes that theme through point-cloud set encoding and coordinate-conditional neural fields rather than through a direct latent isometry loss.

A second family of related works concerns latent manifold structure rather than encoder–decoder geometry alone. Geometry-Aware Hamiltonian VAE treats the latent space as a Riemannian manifold endowed with a learned metric tensor and uses Riemannian Hamiltonian dynamics for inference (Chadebec et al., 2020). Geodesic Calculus on Latent Spaces represents latent manifolds as implicit submanifolds, learns an approximate projection onto the manifold, and then computes discrete geodesics and exponential maps (Hartwig et al., 10 Oct 2025). Lie Group Auto-Encoder represents Gaussian latent distributions as points on a Lie group and uses Lie algebra coordinates, exponential maps, and intrinsic losses (Gong et al., 2019). Relative to these approaches, GeoFAE does not endow latent space with an explicit learned Riemannian metric in the same manner; its principal geometric act is to map between irregular domains and continuous fields through a fixed-dimensional latent representation.

A third, distinct notion of geometric autoencoding appears in sparse coding and optimization-based encoding. “Sparse, Geometric Autoencoder Models of V1” introduces a weighted-$s\ge 1$3 regularizer that favors basis functions close to the input in ambient space and yields latent representations “implicitly, locally organized for spectral clustering” (Huml et al., 2023). Gradient Flow Encoding replaces the encoder by latent-space optimization driven only by the decoder and a reconstruction loss, using gradient-flow ODEs and adaptive step size (Flouris et al., 2021). These works use “geometric” in senses tied respectively to structured sparsity and optimization geometry, which clarifies that GeoFAE in GeoFunFlow is not a generic label for all geometry-aware autoencoders but a specific architecture for geometry-conditioned function representation.

6. Empirical behavior, limitations, and directions

On the five PDE benchmarks reported in the GeoFunFlow paper—Darcy flow on domains with notches, Cylinder flow, Plasticity, Airfoil flow, and Ahmed-body aerodynamics—GeoFAE alone, used as a deterministic reconstructor, achieves state-of-the-art or best deterministic accuracy: it has the lowest average relative $s\ge 1$4 error on Darcy, Cylinder, Plasticity, and Airfoil, and is second-best on Ahmed body. The paper further states that it outperforms Geo-FNO and Transolver significantly on several tasks, despite having fewer parameters, with GeoFAE at approximately $s\ge 1$5M parameters versus approximately $s\ge 1$6M for those baselines (Wang et al., 28 Sep 2025).

The integration with latent rectified flow changes the objective from deterministic regression to posterior modeling. GeoFunFlow therefore gives slightly higher reconstruction error than GeoFAE alone, which the paper explicitly identifies as expected because the model is optimized for posterior modelling rather than pure regression. The gain is that posterior sampling and calibrated uncertainty quantification become available while maintaining continuous decoding over complex geometries (Wang et al., 28 Sep 2025).

Ablations sharpen the interpretation of the encoder and latent space. On the Cylinder dataset, increasing the number of latent queries $s\ge 1$7 yields only small gains, and a single Perceiver cross-attention block is often sufficient, indicating that the Perceiver acts robustly as a set-to-fixed-latent compressor. Latent sampling is also efficient: even a single integration step for the rectified flow produces reconstructions with very low error, and additional ODE steps yield diminishing returns. The paper interprets this as evidence that GeoFAE’s latent space is well-structured so that the learned flow is nearly straight and short (Wang et al., 28 Sep 2025).

The stated limitations are equally specific. GeoFAE treats the domain as a point cloud with coordinate embeddings and does not exploit mesh connectivity or explicit geometric priors such as boundary normals or topology features beyond coordinates. Posterior variance is sometimes relatively high, which the authors take as evidence that richer geometric encoders, including graph-based or equivariant architectures, could improve representation quality. The framework handles up to $s\ge 1$8 nodes in the reported experiments, but scaling to millions of elements would require additional architectural and computational optimization. Extensions to more complex multi-physics operators, higher-dimensional domains, and direct incorporation of physics-informed constraints are identified as natural next steps (Wang et al., 28 Sep 2025).

Taken together, these properties define GeoFAE as a geometry-aware, mesh-size-agnostic, continuous neural-field autoencoder specialized to inverse operator learning on complex geometries. A plausible implication is that its main contribution is not merely compression, but the establishment of a common latent interface between irregular geometric domains and latent generative modeling. Within the current formulation, however, that interface remains coordinate-based and data-driven rather than connectivity-aware or physics-constrained.