EquiformerV2: SE(3) Graph Transformer

• EquiformerV2 is an SE(3)-equivariant graph Transformer that scales atomistic modeling by replacing expensive SO(3) convolutions with efficient eSCN operations.
• It introduces key architectural innovations including attention re-normalization, separable S² activation, and separable layer normalization to leverage higher-degree feature representations.
• Large-scale pretraining and foundation-model scaling on DFT datasets demonstrate improved energy, force, and stress predictions across materials modeling and geometric learning tasks.

EquiformerV2 is an $SE(3)$-equivariant graph Transformer for 3D atomistic systems that was introduced as an improved version of Equiformer for scaling to higher-degree representations. Its defining move is to preserve the equivariant Transformer formulation while replacing expensive $SO(3)$ convolutions with eSCN convolutions and adding three architectural changes—attention re-normalization, separable $S^2$ activation, and separable layer normalization—that make higher-order irreducible representations practically useful rather than merely affordable. In subsequent work, EquiformerV2 also became a foundation-model family through large-scale pretraining on OMat24 and related corpora, and later papers reused it as a pretrained surrogate, a backbone, or an architectural template across materials modeling, catalysis, and geometric learning (Liao et al., 2023, Barroso-Luque et al., 2024).

1. Origin, design objective, and problem setting

EquiformerV2 was proposed for the standard atomistic learning regime in which atoms are nodes of a 3D graph, positions supply the geometry, and targets include invariant energies and equivariant forces. The motivating limitation of the original Equiformer was not that equivariant graph attention failed in principle, but that it did not scale well to higher degrees of $SO(3)$ representations because of the computational cost of its tensor-product machinery. The original paper makes the design question explicit: can equivariant Transformers benefit from larger representation degree, and what changes are required to make that regime trainable and efficient (Liao et al., 2023)?

The answer given by EquiformerV2 is architectural rather than purely computational. The model keeps the broad Transformer skeleton—equivariant graph attention, feedforward blocks, and equivariant outputs—but changes the internal equivariant operations so that higher angular bandwidth can be used effectively. In the original report, this yields improvements of up to $9\%$ on forces and $4\%$ on energies on OC20, better speed–accuracy trade-offs, and a $2\times$ reduction in DFT calculations needed for AdsorbML adsorption-energy evaluation at comparable quality (Liao et al., 2023).

A later successor paper characterizes EquiformerV2 retrospectively as the stage in the Equiformer lineage that made higher-degree representations operational by combining eSCN convolutions with attention re-normalization, separable $S^2$ activation, and separable layer normalization. That retrospective framing is useful because it distinguishes EquiformerV2 from both the original Equiformer and later EquiformerV3: V2 is the version centered on scaling the equivariant Transformer to larger $L_{\max}$, whereas V3 is presented as addressing efficiency, expressivity, and physical consistency beyond V2 (Liao et al., 10 Apr 2026).

2. Representation-theoretic structure and core architectural mechanics

EquiformerV2 represents each node by a stack of $SO(3)$ irreducible features. In the notation used by the original paper, the feature tensor is

$SO(3)$0

with entries $SO(3)$1, where $SO(3)$2 is the degree, $SO(3)$3 is the order, and $SO(3)$4 indexes channels. Degree $SO(3)$5 corresponds to scalars; $SO(3)$6 corresponds to vectors; higher $SO(3)$7 carry finer angular structure (Liao et al., 2023).

Equivariant interactions are built from irreducible tensor products. The paper writes

$SO(3)$8

where the Clebsch–Gordan coefficients couple degrees subject to the usual angular-momentum selection rule. This is the algebraic core of equivariant feature mixing, but EquiformerV2 avoids using the original Equiformer’s expensive $SO(3)$9 convolutions directly. Instead, it adopts eSCN convolutions, rotating features into an edge-aligned frame, applying efficient $S^2$0-structured linear operations, and rotating back. The original paper presents this as reducing the relevant complexity from roughly

$S^2$1

which is the main reason higher $S^2$2 becomes feasible (Liao et al., 2023).

Attention remains scalar in its weights and equivariant in its values. The attention logits are computed from scalar edge features by

$S^2$3

where the extra layer normalization inside the attention-logit path is the “attention re-normalization” introduced by EquiformerV2. The message values are equivariant features, so the attention acts as scalar reweighting of irreducible tensors rather than a purely scalar aggregation rule (Liao et al., 2023).

The nonlinearity is also specialized. Rather than using gate activation alone, EquiformerV2 uses separable $S^2$4 activation based on the spherical projection

$S^2$5

The paper’s point is that a nonlinearity acting through samples on $S^2$6 can mix information across degrees, which is especially important once higher-degree channels are present in quantity. Likewise, EquiformerV2 replaces degree-wise equivariant layer normalization with separable layer normalization so that scalar features and non-scalar features are normalized differently, preserving relative magnitude information across $S^2$7 channels better than the earlier design (Liao et al., 2023).

At the output level, the original paper states that scalar quantities such as energies are produced from scalar readouts summed over nodes, while atom-wise forces are produced by an additional equivariant graph-attention block whose degree-1 output is treated as the force prediction. Later OMat24 work makes the corresponding engineering choice explicit for large pretrained checkpoints: energy, force, and stress are predicted through separate heads, and the models are optimized by directly predicting forces and stress rather than relying on energy derivatives; in that implementation, the symmetric stress tensor is decomposed into an $S^2$8 isotropic component and an $S^2$9 anisotropic component before reconstruction at test time (Barroso-Luque et al., 2024).

3. Foundation-model scaling, pretraining, and empirical scaling laws

EquiformerV2 ceased to be only an architecture paper once the OMat24 release turned it into a full foundation-model family. That work focused solely on EquiformerV2 because it was already “the top performing model on the OC20, OC22 and ODAC23 leaderboards,” and paired the architecture with a dataset containing over 110 million DFT calculations. The released OMat24 family comprises eqV2-S, eqV2-M, and eqV2-L with 31,207,434, 86,589,068, and about 153.8M parameters, respectively, and reported throughputs of 9.4, 7.4, and 4.9 samples/GPU-sec on an NVIDIA A100 (Barroso-Luque et al., 2024).

Model	Parameters	Throughput
eqV2-S	31,207,434	9.4 samples/GPU-sec
eqV2-M	86,589,068	7.4 samples/GPU-sec
eqV2-L	$SO(3)$0M	4.9 samples/GPU-sec

Those models define much of the later EquiformerV2 ecosystem. On OMat24 validation, the reported progression is systematic: eqV2-S, eqV2-M, and eqV2-L reach energy MAE 11, 10, and 9.6 meV/atom; force MAE 49.2, 44.8, and 43.1 meV/$SO(3)$1; and stress MAE 2.4, 2.3, and 2.3 meV/$SO(3)$2 (Barroso-Luque et al., 2024). For Matbench Discovery, the same paper distinguishes “compliant” MPtrj-only training from larger-data fine-tuning. In the compliant regime, eqV2-L-DeNS reaches $SO(3)$3 and MAE $SO(3)$4 meV/atom on the unique-prototype split. In the non-compliant regime, the best OMat24-pretrained model—eqV2-M fine-tuned on MPtrj+sAlex—reaches $SO(3)$5 and MAE $SO(3)$6 meV/atom, which the paper describes as crossing the symbolic $SO(3)$7 barrier (Barroso-Luque et al., 2024).

A separate scaling-law study then used EquiformerV2 not as a new architecture but as one of two model families for supervised prediction of DFT-derived energy, forces, and stresses on OMat24 subsets. In that study, a representative 2.1M-parameter EquiformerV2 obeys

$SO(3)$8

for dataset size $SO(3)$9, non-embedding parameter count $9\%$0, and training compute $9\%$1 in FLOPs, respectively. The reported exponents are substantially steeper than those of the encoder-only transformer baseline in the same paper, which the authors interpret as better data-, parameter-, and compute-efficiency for the symmetry-aware model family (Trikha et al., 26 Sep 2025).

These results also sharpen an important practical point: “EquiformerV2” no longer denotes only the 2023 OC20 architecture. It also denotes a family of pretrained checkpoints whose behavior depends materially on training corpus, fine-tuning path, auxiliary objectives such as DeNS, and output heads. Later benchmarking consistently shows that checkpoint choice can be as consequential as architecture choice (Barroso-Luque et al., 2024).

4. Scientific use as a pretrained surrogate and backbone

In later materials work, EquiformerV2 is repeatedly used as an existing surrogate rather than rederived. In defect modeling for metals and random alloys, ten pretrained EquiformerV2 models trained on MPtrj, OMat24, or OMat24+MPtrj+sAlex are benchmarked on 54,084 DFT configurations spanning grain boundaries, dislocations, alloy defects, hydrogen interactions, and solute–defect structures. The paper’s headline claim is that the latest EquiformerV2 models reach below about 5 meV/atom for energies and below about 100 meV/$9\%$2 for forces across many bulk defect problems, often outperforming specialized ACE and MTP baselines trained for the target systems (Shuang et al., 5 Feb 2025).

For phonons, EquiformerV2 is one of the most extensively tested universal MLIP families in a 2,429-material OQMD-derived benchmark. There the picture is differentiated but favorable: the OMat24-pretrained EquiformerV2 gives the best force RMSE, 99.54 meV/$9\%$3, and the best third-order IFC RMSE, 11.53 eV/$9\%$4, while the fine-tuned EquiformerV2 gives the best second-order IFC RMSE, 1.16 eV/$9\%$5, the highest dynamic-stability recovery rate, 81.0%, and the best lattice thermal conductivity log-MAE, 0.174 (Anam et al., 3 Sep 2025). This does not mean that one checkpoint dominates every stage of the phonon pipeline; rather, it shows that EquiformerV2 checkpoints can trade off raw force accuracy against downstream harmonic and anharmonic observables.

In catalyst-related workflows, EquiformerV2 also appears as a component inside larger algorithms. AdsorbFlow uses an EquiformerV2 backbone to parameterize an energy-conditioned deterministic flow field over adsorbate rigid-body configurations, reporting 34.1% SR@1 and 61.4% SR@10 on OC20-Dense with only 5 ODE steps for the EqV2 version. The same paper is explicit, however, that AdsorbFlow—not EquiformerV2—is the main algorithmic contribution; EquiformerV2 is the adopted equivariant backbone and the stronger of the two backbones tested there (Qiu et al., 22 Feb 2026). In hybrid SSNEB, a 153M-parameter eqV2-L-OMat model provides per-image energies, forces, and stresses as a surrogate calculator; for CsPbI$9\%$6 $9\%$7 with 10 images, restarting DFT from the eqV2-converged path reduces the SSNEB iteration count from 370 to 81, and the overall framework achieves up to a 7-fold speedup while converging to the same pathway predicted by first-principles calculations (Zhang et al., 11 Jun 2026).

The model family is also used in teacher–student form. In LES-based distillation, AQCat25-ev2—an EquiformerV2 teacher with 153.0M parameters trained on AQCat25 and OC20—supplies energies and forces for a lightweight LES-augmented student, allowing latent electrostatics, Born effective charges, and IR spectra to be extracted in bulk water. The paper’s conclusion is deliberately modest: EquiformerV2 is not uniquely privileged among teacher models, but it is a viable teacher from which electrostatic response can be distilled (Wang et al., 12 Jun 2026).

5. Limitations, benchmark caveats, and common misconceptions

A recurring misconception is that strong single-point force or energy accuracy guarantees a smooth or reliable potential-energy surface in downstream search or simulation. Later benchmarks explicitly reject that inference for EquiformerV2. In unconstrained evolutionary global optimization, EquiformerV2 has the second-lowest ranking RMSE, 6.94 meV/atom, with energy proximity 1.89 meV/atom and structure proximity 0.66, yet achieves only 9/12 search success and generates unusually large pools of low-symmetry candidate structures, exceeding 2,000 structures in the merged pool. The same paper reports $9\%$8 meV/atom jumps along the anomalous hcp-Zn $9\%$9 energy profile and interprets them as a smoothness artifact consistent with known discretization issues in the EquiformerV2/eSCN family (Marcial et al., 26 Feb 2026).

A second misconception is that EquiformerV2 is uniformly dominant across chemically diverse downstream tasks. Several benchmark papers show a more mixed picture. In the 0123D reduced-dimensionality benchmark, EquiformerV2 is one of the strongest models for geometry optimization when it converges, but its unconverged relaxation counts are 620, 80, 35, and 3 for 0D, 1D, 2D, and 3D systems, respectively, and the paper explicitly links these failures to the non-conservative direct-force formulation of the tested checkpoint (Benedini et al., 21 Aug 2025). In zeolite benchmarking, EqV2(OC22) gives RMSE 0.26 kJ mol$4\%$0 for Cu/CHA and 0.09 kJ mol$4\%$1 for K-OSDA/ERI, which the paper judges acceptable but not best-in-benchmark; eSEN-30M-OAM is the most consistent model there (Ito et al., 9 Sep 2025). On the Ti$4\%$2CT$4\%$3 MXene benchmark, EquiformerV2 small reaches 3.5 meV/atom and 15.0 meV/$4\%$4 on the original test set, but degrades to 5.0 meV/atom and 44.4 meV/$4\%$5 on the larger stitched-system OOD benchmark, trailing the strongest MACE and PET baselines in both energy and large-system generalization (Melnyk et al., 30 May 2026).

A third misconception is historical: many later papers that mention EquiformerV2 are not “EquiformerV2 papers” in the architectural sense. The scaling-law study, SSNEB paper, defect benchmarks, phonon benchmark, and reduced-dimensionality benchmark all use existing checkpoints or implementations and often provide very little architecture-specific mathematics. That matters because claims made in those papers generally concern EquiformerV2-as-deployed, not the introduction of new EquiformerV2 variants (Trikha et al., 26 Sep 2025).

These caveats are precisely what the EquiformerV3 paper turns into a design brief. It identifies several V2 limitations: redundant software operations, normalization that still does not fully preserve relative degree importance, attention without smooth radius cutoff, and an $4\%$6-activation regime that can introduce sampling-induced equivariance error. On Matbench Discovery, the cited EquiformerV2+ model attains strong $4\%$7 but poor thermal-conductivity-related KSRME, whereas V3 attributes major improvements to smooth-cutoff attention and SwiGLU-$4\%$8 activations (Liao et al., 10 Apr 2026). This does not invalidate V2; it situates V2 as an important but not final stage in the family.

6. Adaptations beyond atomistic pretraining and direct successors

EquiformerV2 has also been repurposed outside its original atomistic-surrogate role. OrbitGrasp uses a “novel variation on EquiFormerV2” as the core equivariant point-cloud backbone for $4\%$9-equivariant grasp learning. The modifications are explicit: the original $2\times$0 embedding is replaced by a single equivariant linear layer from point normals, the flat graph is turned into a UNet-style encoder–decoder with FPS downsampling, KNN graph rebuilding, edge-reversal upsampling, and skip connections, and the output at each point is a set of spherical harmonic coefficients defining a continuous grasp-quality function

$2\times$1

The point is not that OrbitGrasp is “just EquiformerV2,” but that the EquiformerV2/eSCN machinery can be transplanted into dense point-cloud perception while preserving equivariance (Hu et al., 2024).

A different adaptation appears in Solvaformer, which is described explicitly as a modification of EquiformerV2 for solution-phase prediction. The crucial change there is symmetry: instead of one molecule with a single global $2\times$2 action, the input consists of multiple molecules with independent rigid symmetries, so Solvaformer keeps intramolecular equivariant attention but adds scalar intermolecular attention rather than imposing spurious relative geometry. Under matched hyperparameters, the EquiformerV2 ablation gives MAE 0.691 and RMSE 0.926 on BigSolDBv2 solubility prediction, while Solvaformer improves to MAE 0.643 and RMSE 0.837 (Broadbent et al., 12 Nov 2025). The architectural lesson is that EquiformerV2’s equivariant core is transferable, but not always sufficient without task-specific redesign of the interaction pattern.

The direct successor is EquiformerV3, introduced explicitly as “building on EquiformerV2.” V3 retains the V2 backbone of high-$2\times$3 equivariant graph attention with eSCN-style operations, but adds an optimized implementation for a reported $2\times$4 speedup, equivariant merged layer normalization, larger FFNs, smooth-cutoff attention, and SwiGLU-$2\times$5 activations. In the V3 paper, these changes are motivated as fixes to V2’s remaining limitations in efficiency, expressivity, and physical consistency (Liao et al., 10 Apr 2026).

Taken together, these later adaptations suggest that EquiformerV2 is best understood not only as a specific 2023 atomistic model, but also as a reusable design pattern: an $2\times$6-equivariant graph-attention backbone whose combination of irreducible features, efficient higher-degree processing, and Transformer-style aggregation proved robust enough to seed a large family of pretrained materials models and several nontrivial architectural descendants.