PDEformer-2: A 2D Foundation Model for PDEs

• PDEformer-2 is a versatile foundation model that encodes 2D PDEs as computational graphs, integrating symbolic structure, geometry, coefficients, and boundary conditions.
• It employs a mesh-free implicit neural representation to directly query solution values at arbitrary spatio-temporal coordinates, enabling strong zero-shot and low-shot performance.
• Pretrained on a massive 40 TB dataset covering multiple PDE families, the model supports rapid adaptation and inverse problem-solving for diverse physical systems.

Searching arXiv for the requested topic and closely related papers to ground the article in current research. tool call: arxiv_search({"query":"PDEformer-2 A Versatile Foundation Model for Two-Dimensional Partial Differential Equations", "max_results": 5, "sort_by": "relevance"}) tool result: { "results": [ { "arxiv_id": "(Ye et al., 21 Jul 2025)", "title": "PDEformer-2: A Versatile Foundation Model for Two-Dimensional Partial Differential Equations", "authors": [ "Zhanhong Ye", "Zining Liu", "Bingyang Wu", "Hongjie Jiang", "Leheng Chen", "Minyan Zhang", "Xiang Huang", "Qinghe Meng", "Jingyuan Zou", "Hongsheng Liu", "Bin Dong" ], "abstract": "Partial differential equations (PDEs) play a central role in describing many physical phenomena. Various scientific and engineering applications demand a versatile and differentiable PDE solver that can quickly generate solutions with adequate accuracy, and limitations of the traditional solvers and specialized neural operators motivate the development of foundation models for solving PDEs. This paper introduces PDEformer-2, a versatile foundation model for two-dimensional PDEs. Based on our previous one-dimensional PDEformer-1 model, PDEformer-2 receives the PDE form as network input via computational graph representation, which has the flexibility to encode most common PDEs. The mesh-free predicted solutions can be directly queried at arbitrary spatio-temporal coordinates. A large (40TB) diverse dataset is employed to pretrain the current model, making it capable of simultaneously addressing PDEs with different symbolic forms, domain shapes, boundary conditions, number of variables, and time-dependency. Accurate zero-shot prediction is allowed for PDEs that resemble the pretraining ones. When adapted to new unseen PDEs, PDEformer-2 demonstrates faster learning than many specialized models, and has smaller errors given limited (less than 100) samples. Additionally, PDEformer-2 can be employed in the inverse problems thanks to its fast and differentiable nature and produces reasonable results in our experiments to recover coefficient scalars and fields of a PDE.", "published": "2025-07-21", "categories": [ "cs.LG", "math.NA" ], "pdf_url": "http://arxiv.org/pdf/([2507.15409](/papers/2507.15409))v2", "abs_url": "https://arxiv.org/abs/([2507.15409](/papers/2507.15409))v2" } ] } Searching for PDEformer-1 and adjacent PDE foundation-model papers used for historical and architectural context. tool call: arxiv_search({"query":"PDEformer Towards a Foundation Model for One-Dimensional Partial Differential Equations", "max_results": 5, "sort_by": "relevance"}) tool result: { "results": [ { "arxiv_id": "(Ye et al., 2024)", "title": "PDEformer: Towards a Foundation Model for One-Dimensional Partial Differential Equations", "authors": [ "Zhanhong Ye", "Bingyang Wu", "Zining Liu", "Xiang Huang", "Siyi Du", "Qinghe Meng", "Leheng Chen", "Peng Chen", "Liang Wang", "Minyan Zhang", "Hongsheng Liu", "Bin Dong" ], "abstract": "This paper introduces PDEformer, a neural solver for partial differential equations (PDEs) capable of simultaneously addressing various types of PDEs. We propose to represent the PDE in the form of a computational graph, facilitating the seamless integration of both symbolic and numerical information inherent in a PDE. A graph Transformer and an implicit neural representation (INR) are employed to generate mesh-free predicted solutions. Following pretraining on data exhibiting a certain level of diversity, our model achieves zero-shot accuracies on benchmark datasets that is comparable to those of specifically trained expert models. Additionally, PDEformer demonstrates promising results in the inverse problem of PDE coefficient recovery.", "published": "2024-02-20", "categories": [ "cs.LG", "cs.NA", "math.NA" ], "pdf_url": "http://arxiv.org/pdf/([2402.12652](/papers/2402.12652))v3", "abs_url": "https://arxiv.org/abs/([2402.12652](/papers/2402.12652))v3" } ] } Searching for related transformer-based PDE foundation-model backbones for contextual comparison. tool call: arxiv_search({"query":"PDE-Transformer Efficient and Versatile Transformers for Physics Simulations", "max_results": 5, "sort_by": "relevance"}) tool result: { "results": [ { "arxiv_id": "(Holzschuh et al., 30 May 2025)", "title": "PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations", "authors": [ "Moritz Dacke", "Tim Leibfried", "Alexandros Kamitzis", "Theodoros Parnassos", "Johannes Brandstetter" ], "abstract": "We introduce PDE-Transformer, an improved transformer-based architecture for surrogate modeling of physics simulations on regular grids. We combine recent architectural improvements of diffusion transformers with adjustments specific for large-scale simulations to yield a more scalable and versatile general-purpose transformer architecture, which can be used as the backbone for building large-scale foundation models in physical sciences. We demonstrate that our proposed architecture outperforms state-of-the-art transformer architectures for computer vision on a large dataset of 16 different types of PDEs. We propose to embed different physical channels individually as spatio-temporal tokens, which interact via channel-wise self-attention. This helps to maintain a consistent information density of tokens when learning multiple types of PDEs simultaneously. We demonstrate that our pre-trained models achieve improved performance on several challenging downstream tasks compared to training from scratch and also beat other foundation model architectures for physics simulations.", "published": "2025-05-30", "categories": [ "cs.LG" ], "pdf_url": "http://arxiv.org/pdf/([2505.24717](/papers/2505.24717))v1", "abs_url": "https://arxiv.org/abs/([2505.24717](/papers/2505.24717))v1" } ] } Searching for a modular, non-monolithic alternative cited against PDEFormer-style models. tool call: arxiv_search({"query":"CompNO A Novel Foundation Model approach for solving Partial Differential Equations", "max_results": 5, "sort_by": "relevance"}) tool result: { "results": [ { "arxiv_id": "(Hmida et al., 12 Jan 2026)", "title": "CompNO: A Novel Foundation Model approach for solving Partial Differential Equations", "authors": [ "Arjun Desai", "Michele Morrone", "Gennaro Curbelo", "Andrea Tosin" ], "abstract": "Partial differential equations (PDEs) govern a wide range of physical phenomena, but their numerical solution remains computationally demanding, especially when repeated simulations are required across many parameter settings. Recent Scientific Foundation Models (SFMs) aim to alleviate this cost by learning universal surrogates from large collections of simulated systems, yet they typically rely on monolithic architectures with limited interpretability and high pretraining expense. In this work we introduce Compositional Neural Operators (CompNO), a compositional neural operator framework for parametric PDEs. Instead of pretraining a single large model on heterogeneous data, CompNO first learns a library of Foundation Blocks, where each block is a parametric Fourier neural operator specialized to a fundamental differential operator (e.g. convection, diffusion, nonlinear convection). These blocks are then assembled, via lightweight Adaptation Blocks, into task-specific solvers that approximate the temporal evolution operator for target PDEs. A dedicated boundary-condition operator further enforces Dirichlet constraints exactly at inference time. We validate CompNO on one-dimensional convection, diffusion, convection--diffusion and Burgers' equations from the PDEBench suite. The proposed framework achieves lower relative L2 error than strong baselines (PFNO, PDEFormer and in-context learning based models) on linear parametric systems, while remaining competitive on nonlinear Burgers' flows. The model maintains exact boundary satisfaction with zero loss at domain boundaries, and exhibits robust generalization across a broad range of Peclet and Reynolds numbers. These results demonstrate that compositional neural operators provide a scalable and physically interpretable pathway towards foundation models for PDEs.", "published": "2026-01-12", "categories": [ "cs.LG" ], "pdf_url": "http://arxiv.org/pdf/([2601.07384](/papers/2601.07384))v1", "abs_url": "https://arxiv.org/abs/([2601.07384](/papers/2601.07384))v1" } ] } Searching for work that connects PDE foundation-model backbones to inverse estimation, which is relevant to PDEformer-2’s inverse-use claims. tool call: arxiv_search({"query":"PDE foundation model-accelerated inverse estimation of system parameters in inertial confinement fusion", "max_results": 5, "sort_by": "relevance"}) tool result: { "results": [ { "arxiv_id": "(Rautela et al., 4 Mar 2026)", "title": "PDE foundation model-accelerated inverse estimation of system parameters in inertial confinement fusion", "authors": [ "Mark Allen Adams", "Andy Li", "Babatunde Olugboyega", "Jonghyun Kim", "Michael M. Henry de Frahan", "Rama Govindaraju", "Rose Yu", "Mingchao Liu", "Liping Liu", "Lin Yang" ], "abstract": "PDE foundation models are typically pretrained on large, diverse corpora of PDE datasets and can be adapted to new settings with limited task-specific data. However, most downstream evaluations focus on forward problems, such as autoregressive rollout prediction. In this work, we study an inverse problem in inertial confinement fusion (ICF): estimating system parameters (inputs) from multi-modal, snapshot-style observations (outputs). Using the open JAG benchmark, which provides hyperspectral X-ray images and scalar observables per simulation, we finetune the PDE foundation model and train a lightweight task-specific head to jointly reconstruct hyperspectral images and regress system parameters. The fine-tuned model achieves accurate hyperspectral reconstruction (test MSE 1.2e-3) and strong parameter-estimation performance (up to R^2=0.995). Data-scaling experiments (5%-100% of the training set) show consistent improvements in both reconstruction and regression losses as the amount of training data increases, with the largest marginal gains in the low-data regime. Finally, finetuning from pretrained MORPH weights outperforms training the same architecture from scratch, demonstrating that foundation-model initialization improves sample efficiency for data-limited inverse problems in ICF.", "published": "2026-03-04", "categories": [ "cs.LG", "physics.plasm-ph" ], "pdf_url": "http://arxiv.org/pdf/([2603.04606](/papers/2603.04606))v1", "abs_url": "https://arxiv.org/abs/([2603.04606](/papers/2603.04606))v1" } ] } Searching for an earlier structured PDE-learning precursor for historical context. tool call: arxiv_search({"query":"PDE-Net 2.0 Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network", "max_results": 5, "sort_by": "relevance"}) tool result: { "results": [ { "arxiv_id": "(Long et al., 2018)", "title": "PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network", "authors": [ "Zichao Long", "Yiping Lu", "Xianzhong Ma", "Bin Dong" ], "abstract": "PDE-Net 2.0 is a hybrid neural architecture for data-driven discovery of time-dependent PDEs from spatiotemporal observations. Its key idea is to hard-wire a numerical PDE discretization structure into a deep network while leaving the unknown physics learnable in two interpretable places: the differential operators are represented by constrained convolution kernels, and the nonlinear response function is represented by a compact symbolic neural network, SymNet. This makes the model both a predictor and a system-identification tool: after training, it can roll forward unseen initial conditions, and it can also be read as an explicit candidate PDE.", "published": "2018-11-30", "categories": [ "cs.LG", "math.NA", "math.AP" ], "pdf_url": "http://arxiv.org/pdf/([1812.04426](/papers/1812.04426))v2", "abs_url": "https://arxiv.org/abs/([1812.04426](/papers/1812.04426))v2" } ] } PDEformer-2 is a foundation-model-style surrogate solver for two-dimensional partial differential equations that takes the PDE itself as input through a computational graph, encodes symbolic structure together with coefficients, geometry, and boundary conditions, and predicts solution values directly at arbitrary spatio-temporal coordinates through a mesh-free implicit neural representation. Introduced by Zhanhong Ye, Zining Liu, Bingyang Wu, Hongjie Jiang, Leheng Chen, Minyan Zhang, Xiang Huang, Qinghe Meng, Jingyuan Zou, Hongsheng Liu, and Bin Dong, it extends the earlier one-dimensional PDEformer line from 1D to 2D and is pretrained on a 40 TB multi-family corpus spanning varied symbolic forms, domain shapes, boundary conditions, numbers of variables, and both time-dependent and steady-state tasks (Ye et al., 21 Jul 2025).

1. Concept and problem setting

PDEformer-2 is designed around a broad solver interface for PDEs posed on

$$(t,\boldsymbol r)\in [0,1]\times \Omega,\qquad \boldsymbol r=(x,y)\in \Omega\subseteq [0,1]^2,$$

with interior operator

$\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$

Here, $u_j$ are unknown fields, $c_i$ are scalar coefficients, and $s_i(\boldsymbol r)$ are scalar-valued spatial functions that may serve as initial conditions, source terms, coefficient fields, or boundary data (Ye et al., 21 Jul 2025).

The model therefore targets the operator that maps domain geometry, symbolic PDE structure, coefficients, function-valued inputs, and boundary operators to the unknown solution fields. It explicitly supports time-dependent PDEs, steady-state PDEs, single-variable and multi-variable systems, multiple equations with coupling, different domain shapes, and periodic as well as non-periodic boundaries (Ye et al., 21 Jul 2025). Pretraining includes up to 4 variables, and the reported zero-shot tests include extrapolation to 5-variable systems (Ye et al., 21 Jul 2025).

A central distinguishing property is that PDEformer-2 is not framed as an autoregressive history-to-next-step surrogate. Instead, it is a direct space-time predictor: once a PDE instance is encoded, the decoder can be queried at arbitrary $(t,x,y)$. This places it closer to equation-conditioned operator learning than to regular-grid rollout architectures such as PDE-Transformer, which operate on raw simulation tensors and learn next-step maps on 2D regular grids (Holzschuh et al., 30 May 2025). It also preserves the core recipe introduced in PDEformer for 1D PDEs—computational-graph PDE representation, graph Transformer encoding, and conditional INR decoding—while extending that recipe to 2D geometry and 2D boundary conditions (Ye et al., 2024).

2. Computational-graph representation of PDEs

The defining interface of PDEformer-2 is a directed acyclic computational graph whose nodes encode mathematical objects, differential operators, algebraic operators, special functions, and auxiliary interface nodes (Ye et al., 21 Jul 2025). The listed mathematical-object node types are UF, SC, CF, VC, IC, and [SDF](https://www.emergentmind.com/topics/masked-signed-distance-field-sdf); the differential-operator node types are dt, dx, and dy; the algebraic operators are $+$, $\times$, unary $-$, $(\cdot)^2$, and =0; the special functions are sin, cos, exp10, log10, and AT; and the auxiliary nodes are branch nodes $\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$0, modulation nodes $\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$1, and pad nodes for batching (Ye et al., 21 Jul 2025).

This representation is intended to encode both symbolic and numerical aspects of a PDE instance. Terms such as $\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$2, $\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$3, or $\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$4 are translated into operator subgraphs whose leaves hold unknowns, coefficients, coefficient fields, or initial data, and whose root is typically an =0 residual node. Boundary conditions are also represented symbolically. Periodic boundary conditions are assumed by default and add no extra graph nodes, whereas non-periodic conditions are encoded explicitly, including Dirichlet, Neumann, Robin type I, Robin type II, and generalized Mur absorbing conditions (Ye et al., 21 Jul 2025).

The most consequential 2D extension is the treatment of geometry and boundary subsets through signed distance functions. For a set $\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$5,

$\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$6

Both the domain $\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$7 and boundary subsets $\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$8 are encoded in this way (Ye et al., 21 Jul 2025). The paper argues that SDFs are preferable to characteristic functions because they contain richer geometric information, are more sensitive to topology changes, and can represent codimension-1 boundaries naturally (Ye et al., 21 Jul 2025).

Function-valued inputs defined only on subsets are extended to the full ambient square by nearest-neighbor extension,

$\begin{split} \mathcal{F}(u_1,u_2,\dots,c_1,c_2,\dots,s_1(\boldsymbol r),s_2(\boldsymbol r),\dots)&=0\quad \text{in }\Omega,\ \mathcal{B}_i(u_1,u_2,\dots,c_{i1},c_{i2},\dots,s_{i1}(\boldsymbol r),s_{i2}(\boldsymbol r),\dots)&=0\quad \text{on }\Gamma_i. \end{split}$9

so that the function encoder can process them on a fixed $u_j$0 grid (Ye et al., 21 Jul 2025). Normal derivatives in non-periodic boundary conditions are expressed through the SDF of the domain:

$u_j$1

This converts geometric boundary information into graph-composable differential expressions (Ye et al., 21 Jul 2025).

The AT node is a further extension beyond fully specified symbolic PDEs. It represents an arbitrary transformation $u_j$2 and is used in experiments where the PDE input is incomplete or partially unknown (Ye et al., 21 Jul 2025). This suggests that the graph interface is intended not only for exact symbolic conditioning but also for adaptation under imperfect scientific knowledge.

3. Encoder, decoder, and modulation mechanism

PDEformer-2 consists of three main components: a PDE encoder, a Graphormer-style graph Transformer, and a mesh-free INR decoder conditioned by graph latents (Ye et al., 21 Jul 2025). The encoder itself contains a scalar encoder and a function encoder. Scalar coefficients are processed by a 3-layer MLP with input dimension 1, two hidden layers of width 256, and output dimension $u_j$3. Function-valued inputs are processed by a lightweight CNN on a $u_j$4 single-channel grid using three Conv2D layers with channel counts 32, 128, and 768, each with stride 4 and kernel $u_j$5, interleaved with ReLU, then flattened and split into $u_j$6 embeddings for branch nodes (Ye et al., 21 Jul 2025).

For graph node $u_j$7, the initial embedding is

$u_j$8

where $u_j$9 is the numeric feature embedding, $c_i$0 is a learnable type embedding, and $c_i$1 and $c_i$2 are learnable in-degree and out-degree embeddings (Ye et al., 21 Jul 2025). Graph attention incorporates a shortest-path structural bias and masks disconnected node pairs. Both the base and fast variants use 12 graph Transformer layers, embedding dimension $c_i$3, FFN hidden dimension 1536, and 32 attention heads (Ye et al., 21 Jul 2025).

For each unknown field $c_i$4, the graph Transformer reads out the embeddings of the attached modulation nodes,

$c_i$5

and these $c_i$6 vectors condition the $c_i$7 layers of the decoder (Ye et al., 21 Jul 2025). The decoder is an adapted Poly-INR that takes $c_i$8 as input and predicts $c_i$9 directly. The paper states that, unlike the original Poly-INR, the hypernetworks do not generate the full input weights $s_i(\boldsymbol r)$0, for efficiency, and that the activation is a sine activation scaled by $s_i(\boldsymbol r)$1 (Ye et al., 21 Jul 2025).

The model is therefore mesh-free, non-autoregressive, and coordinate-queryable. This differentiates it sharply from state-space transformer surrogates such as PDE-Transformer, whose design centers on shifted-window attention over regular-grid spatio-temporal tokens and autoregressive next-step prediction (Holzschuh et al., 30 May 2025). It also differentiates PDEformer-2 from modular operator-library approaches such as CompNO, which assemble pretrained operator-specialized blocks into 1D time-advancement solvers rather than ingesting symbolic PDE structure through a graph (Hmida et al., 12 Jan 2026).

The only stated architectural difference between PDEformer-2-base and PDEformer-2-fast is the decoder width. Both use an INR with $s_i(\boldsymbol r)$2 hidden layers, but the base model uses width 768 while the fast model uses width 256 (Ye et al., 21 Jul 2025). This narrower decoder trades some accuracy for speed.

4. Pretraining corpus and optimization

A defining feature of PDEformer-2 is its 40 TB pretraining dataset comprising 3.29 million training samples over 8 generic PDE types, with 1k test samples per dataset (Ye et al., 21 Jul 2025). The paper describes this dataset as varying in symbolic PDE form, scalar coefficients, coefficient fields, source terms, initial conditions, domain shapes, boundary conditions, variable counts, and time dependence (Ye et al., 21 Jul 2025).

PDE type	Training samples	Data size
DCR	745k	4.85 TB
Wave	605k	4.34 TB
MV-DCR	660k	12.24 TB
DC-DCR	400k	7.49 TB
MV-Wave	320k	6.20 TB
DC-Wave	200k	2.62 TB
G-SWE	100k	1.93 TB
Elasticity	255k	0.38 TB

The generic families include diffusion-convection-reaction equations, wave equations, multi-variable reaction/transport systems, divergence-constrained systems with pressure, multi-variable wave systems, generalized shallow-water equations, and steady-state elasticity (Ye et al., 21 Jul 2025). The divergence-constrained DCR family includes incompressible Navier–Stokes in conservation form as a special case, and the paper also notes Maxwell equations as special cases of MV-DCR (Ye et al., 21 Jul 2025). Domains include square periodic and non-periodic cases, partially periodic strip-like squares, and random disks; boundary conditions include periodic, Dirichlet, Neumann, Robin, and generalized Mur conditions; and variable counts span 1 to 4 in pretraining (Ye et al., 21 Jul 2025).

The dataset is generated with Dedalus v3 and FEniCSx. Dedalus is used for spectral-method generation, while FEniCSx is used especially for elasticity and for validating some Dedalus cases (Ye et al., 21 Jul 2025). Randomly sampled PDE instances are discarded if the classical solver fails, if the solution blows up or exceeds $s_i(\boldsymbol r)$3, or if numerical issues arise; reported acceptance rates vary from 10% to 90% depending on dataset (Ye et al., 21 Jul 2025).

Training is fully supervised. Because full $s_i(\boldsymbol r)$4 trajectories are expensive for the INR, the paper uses Monte Carlo point sampling with $s_i(\boldsymbol r)$5 sampled indices per sample and an nRMSE objective normalized by target energy; $s_i(\boldsymbol r)$6 is used during training and $s_i(\boldsymbol r)$7 during evaluation (Ye et al., 21 Jul 2025). For PDEs with $s_i(\boldsymbol r)$8 variables, the loss is computed one variable at a time, so a data sample is used $s_i(\boldsymbol r)$9 times per epoch (Ye et al., 21 Jul 2025).

Pretraining uses Adam with no weight decay, gradient clipping with max norm 1, a maximum learning rate of $(t,x,y)$0 in early stages and then $(t,x,y)$1, 10-epoch linear warmup, and cosine decay (Ye et al., 21 Jul 2025). The implementation uses MindSpore on a machine with 8 NPUs, 192 CPU cores, 768 GB RAM, local disk of about 3 TB plus remote storage, and batch size 80 total, or 10 per device (Ye et al., 21 Jul 2025). Because the full dataset does not fit local storage, training uses a dynamic local data buffer of about 50 files, each with 1k samples, swapped in from remote storage during training (Ye et al., 21 Jul 2025). The paper further states that pretraining was effectively continual or incremental over more than 10 runs, with each stage trained for about 1–2 days, and estimates that full-data-at-once training would require roughly 5–10 days (Ye et al., 21 Jul 2025).

This scale of pretraining is one of the sharpest distinctions from PDEformer-1, whose 1D pretraining used 500k samples and about 79 hours on 8 NPUs (Ye et al., 2024). A plausible implication is that PDEformer-2’s broader 2D scope depends not only on architectural redesign but on a substantial expansion of pretraining diversity and volume.

5. Empirical performance, adaptation, and inverse problems

On its own pretraining test sets, PDEformer-2-base achieves overall nRMSE 8.7% and PDEformer-2-fast achieves 10.3% (Ye et al., 21 Jul 2025). Per-family results reported for the base model range from 3.8% on elasticity to 17.0% on DC-DCR, with DCR at 6.9%, Wave at 6.5%, MV-DCR at 12.6%, MV-Wave at 5.3%, DC-Wave at 9.5%, and G-SWE at 13.0% (Ye et al., 21 Jul 2025). The paper also reports that, despite pretraining only up to 4 variables, zero-shot prediction on 5-variable MV-DCR and MV-Wave systems is reasonable (Ye et al., 21 Jul 2025).

The forward adaptation benchmarks expose a more differentiated picture. On the in-distribution Sine-Gordon problem,

$(t,x,y)$2

PDEformer-2 already performs very well zero-shot, and the full-prediction errors reported after training are 0.018 for PDEformer-2-base and 0.027 for PDEformer-2-fast, compared with 0.047 for FNO3D, 0.075 for FNO2D, 0.278 for U-Net2D, 0.291 for DeepONet(CNN), 0.431 for DeepONet(MLP), 0.427 for MPP-B, and 0.436 for Poseidon-B (Ye et al., 21 Jul 2025). Fine-tuning yields little additional gain in this case (Ye et al., 21 Jul 2025).

On INS-Tracer, which couples incompressible Navier–Stokes with tracer transport and is described as near-distribution because it combines known mechanisms in a new way, zero-shot is worse than on Sine-Gordon but still reasonable; PDEformer-2 benefits from fine-tuning as data grows, but Poseidon surpasses it by around 80 samples and FNO eventually surpasses it with enough data (Ye et al., 21 Jul 2025). On INS-Pipe, whose no-slip wall boundary conditions are not satisfactory in zero-shot, the model adapts efficiently and remains superior across all tested data sizes; the full-prediction errors reported are 0.103 for PDEformer-2-base and 0.115 for PDEformer-2-fast, compared with 0.128 for FNO3D, 0.140 for FNO2D, 0.158 for Poseidon-B, 0.131 for MPP-B, 0.247 for U-Net2D, 0.379 for DeepONet(CNN), and 0.333 for DeepONet(MLP) (Ye et al., 21 Jul 2025).

Two cautions are explicit. First, transfer degrades on farther OOD wave problems such as Wave-Gauss and especially Wave-C-Sines on a disk, where accuracy is unsatisfactory and Geo-FNO performs better (Ye et al., 21 Jul 2025). The authors attribute this to high-frequency initial conditions not matched by pretraining, insufficient non-square-domain pretraining, and possible architectural limitations for high-frequency input (Ye et al., 21 Jul 2025). Second, when task-specific data are abundant, specialized neural operators can outperform PDEformer-2 on some targets (Ye et al., 21 Jul 2025). A common misconception is therefore that the model is uniformly superior to specialized surrogates; the reported evidence instead supports a narrower claim of strong zero-shot behavior for sufficiently similar PDEs and strong low-shot adaptation for many unseen ones.

The paper also reports several timing figures. For full-trajectory inference on Sine-Gordon, PDEformer-2-base takes 0.631 s and PDEformer-2-fast 0.202 s, versus Dedalus at 2.398 s; for INS-Pipe the corresponding times are 1.262 s, 0.406 s, and 24.300 s (Ye et al., 21 Jul 2025). However, FNO3D is still faster for dense full-grid outputs at 0.062 s on both cited tasks (Ye et al., 21 Jul 2025). Encoder-only “INR as output” cost is about 0.020–0.022 s, which the paper identifies as useful for sparse-query settings (Ye et al., 21 Jul 2025). This makes the speed story nuanced: PDEformer-2 is slower than grid neural operators for dense outputs but faster than the cited classical solver, while retaining mesh-free and differentiable querying.

Inverse use is a major secondary theme. The model is used as a differentiable surrogate forward operator for recovering scalar coefficients, source fields, and wave-velocity fields from sparse noisy observations (Ye et al., 21 Jul 2025). For coefficient recovery on 40 DCR PDEs with 25 initial conditions per PDE, 128 random spatial locations, average 20 observed times per spatial point, and 1% observation and initial-condition noise, optimization is done with PSO using population size 100 and max 200 generations; the paper states that most coefficients are successfully recovered (Ye et al., 21 Jul 2025). For system identification with a full candidate library, most zero coefficients are approximately identified, and only 6 out of 29 recovered zero coefficients exceed absolute value 0.1, despite no explicit sparsity penalty (Ye et al., 21 Jul 2025). For source-field recovery on 40 source fields, using Adam with learning rate 0.1, 150 iterations, $(t,x,y)$3, and regularization $(t,x,y)$4, the average recovered-field nRMSE is 0.159 (Ye et al., 21 Jul 2025). For wave-velocity recovery, the paper notes local minima and uses multiple restarts over 20000 Adam iterations (Ye et al., 21 Jul 2025).

These inverse demonstrations align PDEformer-2 with a broader trend in PDE foundation models toward using pretrained forward surrogates as inverse-ready latent priors. Related work based on the MORPH backbone likewise reports that foundation-model initialization improves sample efficiency for a data-limited inverse problem in inertial confinement fusion, though that study uses a different architecture and a multimodal inverse head rather than computational-graph PDE conditioning (Rautela et al., 4 Mar 2026).

6. Historical position, comparisons, and limitations

PDEformer-2 is best understood as the two-dimensional continuation of the PDEformer line. PDEformer established the core paradigm for 1D time-dependent PDEs: represent the PDE as a computational graph, encode that graph with a graph Transformer, and decode the solution with a conditional INR (Ye et al., 2024). PDEformer-2 preserves that paradigm while redesigning the input side for 2D geometry and boundary conditions through SDFs, replacing the 1D initial-condition patch interface with a 2D CNN function encoder, and scaling pretraining to a much larger and more varied corpus (Ye et al., 21 Jul 2025).

Within the broader landscape of PDE foundation models, PDEformer-2 occupies a specific point. It differs from PDE-Transformer, which is presented as a general-purpose backbone for raw simulation fields on regular 2D grids and emphasizes shifted-window attention, multiscale U-shaped token hierarchies, and separate-channel tokenization for autoregressive surrogate modeling (Holzschuh et al., 30 May 2025). It also differs from CompNO, which rejects monolithic equation-conditioned architectures in favor of compositional libraries of operator-specialized PFNO blocks assembled by lightweight adapters for a narrow class of 1D parametric PDEs (Hmida et al., 12 Jan 2026). By contrast, PDEformer-2’s claim to versatility rests on explicit symbolic graph conditioning and mesh-free coordinate querying (Ye et al., 21 Jul 2025).

Historically, the emphasis on exposing PDE structure also distinguishes the PDEformer line from earlier discovery-oriented architectures such as PDE-Net 2.0, which coupled constrained convolutional derivative filters with a symbolic neural network to identify local time-dependent PDEs from data (Long et al., 2018). PDE-Net 2.0 is closer to numeric-symbolic discovery within a fixed local PDE ansatz, whereas PDEformer-2 is closer to broad pretrained representation learning over a space of PDE expressions and associated boundary-value problems.

The principal limitations are explicit. Dense inference is relatively expensive because the INR queries points independently. Accuracy is not always state of the art when abundant task-specific data are available. Generalization remains limited for farther OOD cases, especially high-frequency wave regimes and underrepresented geometries. Dependence on pretraining solver coverage remains important; when the teacher solver fails on a class, PDEformer-2 may also struggle. The paper further identifies more complicated domain shapes, time-dependent coefficients and domains, integro-differential equations, 3D PDEs, real-world validation, and downstream design and control as future directions (Ye et al., 21 Jul 2025).

The strongest misconception to avoid is that PDEformer-2 constitutes a universal PDE solver. The paper instead supports a more qualified interpretation: it is a versatile 2D foundation model whose zero-shot behavior is strong when targets resemble the pretraining distribution, whose low-shot adaptation is often favorable relative to specialized models, and whose differentiable structure makes inverse use practical, but whose far-OOD robustness, dense-output efficiency, and ultimate breadth remain open research problems (Ye et al., 21 Jul 2025).