---
title: 'Geometric Vector Perceptrons: 3D Equivariant Modules'
url: https://www.emergentmind.com/topics/geometric-vector-perceptrons-gvps
type: topic
---

# Geometric Vector Perceptrons: 3D Equivariant Modules

Searching arXiv for relevant papers on Geometric Vector Perceptrons and follow-up work.
Geometric Vector Perceptrons (GVPs) are neural modules for data whose latent state contains both invariant scalar features and equivariant Euclidean vector features. Introduced for learning from protein structure, they extend dense layers to operate on paired representations $(\mathbf{s}, \mathbf{V})$, where $\mathbf{s}\in\mathbb{R}^n$ and $\mathbf{V}\in\mathbb{R}^{\nu\times 3}$, and produce outputs $(\mathbf{s}', \mathbf{V}')$ with the property that scalar outputs remain invariant while vector outputs transform equivariantly under rotations and reflections of the input geometry [2009.01411]. In practice, GVPs are used as the fundamental building block of GVP-based graph neural networks, enabling simultaneous relational reasoning on molecular graphs and direct reasoning over 3D orientation-dependent structure in proteins, RNA, ligands, and molecular complexes [2106.03843].

## 1. Origins and problem setting

GVPs were introduced to address a specific gap in learning from 3D macromolecular structure: existing methods for proteins had tended to emphasize either geometric reasoning or relational reasoning, but not both in a unified, scalable architecture [2009.01411]. Voxelized 3D CNNs can detect local shape, motifs, and pockets from occupancy grids, whereas graph neural networks are well suited to residue- or atom-level interaction graphs. The motivating observation was that proteins are simultaneously non-Euclidean relational objects and embedded 3D structures whose function depends critically on positions, directions, and orientations.

The original formulation targets two structure-based protein tasks: model quality assessment (MQA), which predicts how good a candidate protein structure is, and computational protein design (CPD), which predicts which amino acid sequence is compatible with a given backbone structure [2009.01411]. In both settings, scalarizing geometry into distances, dot products, or local-coordinate descriptors was presented as inadequate because it obscures directional structure and makes propagation of geometric objects through message passing less natural.

A central design claim is that prior graph-based protein models often encoded geometry only indirectly through rotation-invariant scalar features. GVPs instead allow graph networks to process vector-valued geometric features directly, so learned hidden states can carry directional and orientation information through the network while still supporting standard graph-based aggregation [2009.01411]. This suggests a deliberate geometric inductive bias: represent genuine geometric quantities as vectors and constrain computations so that they respect Euclidean symmetry.

## 2. Layer definition and computational mechanism

A GVP maps
$$
(\mathbf{s}, \mathbf{V}) \in \mathbb{R}^n \times \mathbb{R}^{\nu\times 3}
$$
to
$$
(\mathbf{s}', \mathbf{V}') \in \mathbb{R}^m \times \mathbb{R}^{\mu\times 3}.
$$
The hidden state is thus partitioned into scalar channels and vector channels, with the latter interpreted literally as collections of 3D vectors [2009.01411].

In the original formulation, vectors are processed only through operations compatible with Euclidean symmetry: linear combinations of vector channels, row-wise $L_2$ norms, and scalar multiplication of vectors. Let $h=\max(\nu,\mu)$. The layer computes intermediate vector transforms
$$
\mathbf{V}_h=\mathbf{W}_h\mathbf{V}\in\mathbb{R}^{h\times 3}, \qquad
\mathbf{V}_\mu=\mathbf{W}_\mu\mathbf{V}_h\in\mathbb{R}^{\mu\times 3},
$$
where the learned matrices act across vector channels rather than across Cartesian coordinates [2009.01411]. This distinction is essential: channel mixing preserves the transformation law of each 3D vector.

The scalar branch receives invariant summaries of the vector branch through row-wise norms:
$$
\mathbf{s}_h=\|\mathbf{V}_h\|\in\mathbb{R}^h, \qquad
\mathbf{v}_\mu=\|\mathbf{V}_\mu\|\in\mathbb{R}^\mu.
$$
These norms are concatenated with the input scalars,
$$
\mathbf{s}_{h+n}=\operatorname{concat}(\mathbf{s}_h,\mathbf{s}),
$$
followed by an affine map and scalar nonlinearity:
$$
\mathbf{s}_m=\mathbf{W}_m\mathbf{s}_{h+n}+\mathbf{b}, \qquad
\mathbf{s}'=\sigma(\mathbf{s}_m).
$$
Vector outputs are then obtained by norm-based scaling,
$$
\mathbf{V}'=\sigma^+(\mathbf{v}_\mu)\odot \mathbf{V}_\mu,
$$
with row-wise multiplication [2009.01411]. The paper notes that this vector nonlinearity functions as an effective norm-based gating mechanism.

The 2021 extension modifies this final step through explicit vector gating. There, scalar activations are used to generate invariant gates
$$
g=\sigma_g\!\left(W_g[\sigma^+(s_m)] + b_g\right)\in\mathbb{R}^{\mu},
$$
and the output vectors are
$$
V' = g\odot V_\mu.
$$
This modification allows scalar information to control vector outputs while preserving equivariance [2106.03843]. The stated motivation is atomic-level modeling: unlike residue-level protein nodes, atoms may not have intrinsic orientation-bearing vector features at input, so scalar-to-vector information flow becomes important.

## 3. Symmetry properties and approximation results

The defining formal property of the original GVP is that if
$$
\operatorname{GVP}(\mathbf{s},\mathbf{V})=(\mathbf{s}',\mathbf{V}'),
$$
then for any composition $R$ of rotations and reflections in 3D,
$$
\operatorname{GVP}(\mathbf{s},R(\mathbf{V}))=(\mathbf{s}',R(\mathbf{V}')).
$$
Scalar outputs are therefore invariant, and vector outputs equivariant, under the orthogonal group action considered in the paper [2009.01411]. The reason is structural: the only operations applied to vector-valued inputs are scalar multiplication, channelwise linear combination, and row-wise Euclidean norm, all of which commute appropriately with rotations and reflections.

The 2021 paper emphasizes rotation equivariance in practical message passing, while noting that the theorem is stated for rotations and/or reflections [2106.03843]. Translation dependence is avoided by constructing features from relative directions and lengths rather than absolute coordinates.

Beyond symmetry preservation, the original paper proves an approximation theorem for invariant scalar-valued functions. For $\nu\ge 3$, with $\Omega^\nu\subset \mathbb{R}^{\nu\times 3}$ defined so that the first three vectors are linearly independent and all vectors have bounded nonzero norm, any continuous $F:\Omega^\nu\rightarrow\mathbb{R}$ satisfying
$$
F(R(\mathbf{V}))=F(\mathbf{V})
$$
can be approximated arbitrarily well by a form $f(\mathbf{V})=\mathbf{w}^T G_s(\mathbf{V})$ [2009.01411]. The intended significance is that GVPs are not merely symmetry-respecting; they are expressive for continuous invariant targets under the stated assumptions.

The modified GVP with vector gating is given a stronger approximation result: its vector outputs can approximate certain continuous equivariant vector-valued functions $F:\Omega^\nu\to\mathbb{R}^3$ satisfying
$$
(F\circ R)(\mathbf{V})=(R\circ F)(\mathbf{V})
$$
[2106.03843]. A plausible implication is that vector gating is not only a practical remedy for atomic graphs, but also a principled way to expand the function class represented by the module.

## 4. Integration into graph neural networks

GVPs are used by replacing dense or MLP submodules inside a message-passing GNN with GVP blocks, producing a GVP-GNN [2009.01411]. In the protein-structure formulation, the graph is
$$
\mathcal{G}=(\mathcal{V},\mathcal{E}),
$$
with each node corresponding to an amino acid residue. Node and edge embeddings each contain both scalar and vector channels.

For residue-level protein graphs, node scalar features include $\sin$ and $\cos$ of backbone dihedral angles $\phi,\psi,\omega$, along with amino acid identity one-hot features when available. Node vector features include forward and reverse unit vectors along the backbone and an imputed unit vector in the direction of $\mathrm{C}\beta_i-\mathrm{C}\alpha_i$ constructed from tetrahedral geometry. Directed edges are defined by $k$-nearest neighbors with $k=30$ using $\mathrm{C}_\alpha$ distance, and each edge carries a unit direction vector together with Gaussian radial basis function encoding of distance and sinusoidal encoding of sequence offset. The paper uses 16 Gaussian radial basis functions with centers evenly spaced between $0$ and $20$ Å [2009.01411].

The message function is
$$
\mathbf{h}^{(j\rightarrow i)}_m := g\left(\operatorname{concat}\left(\mathbf{h}^{(j)}_\mathfrak{v}, \mathbf{h}^{(j\rightarrow i)}_e\right)\right),
$$
where $g$ is a sequence of three GVPs. Node updates use residual connections, LayerNorm, Dropout, and averaged incoming messages, followed by a pointwise feed-forward update implemented as a sequence of two GVPs [2009.01411]. The architecture is therefore closely analogous to alternating message-passing and feed-forward residual blocks, except that the MLP-like components are GVPs.

The atomic-level extension applies the same overall message-passing formulation but changes graph construction. Nodes are atoms, each with a one-hot encoding of element type as scalar features; vector features are not explicitly initialized from atom orientation. Edges connect atom pairs at distance less than $4.5$ Å, with scalar features given by Gaussian radial basis encoding of edge length and vector features given by the unit edge direction [2106.03843]. The model uses hidden embeddings with 16 vector channels and 100 scalar channels, 5 GVP-GNN layers, mean pooling unless otherwise noted, and 2 dense layers; for all GVPs, $\sigma=\mathrm{ReLU}$ and $\sigma^+=\mathrm{id}$ [2106.03843].

## 5. Protein-structure applications and empirical performance

The original paper evaluates GVP-GNNs on CPD and MQA and reports improvements over existing classes of graph-based and voxel-based methods [2009.01411]. In CPD, the task is autoregressive sequence modeling conditioned on backbone structure. On the CATH 4.2 split from Ingraham et al., consisting of 18,204 training structures, 608 validation structures, and 1,120 test structures, the reported per-residue perplexity and recovery are 5.29 and 40.2% for GVP-GNN, compared with 6.55 and 37.3% for a Structured GNN and 6.85 and 36.4% for a Structured Transformer [2009.01411]. On TS50, GVP-GNN reaches 44.9% recovery, compared with 30.0% for Rosetta, 30.8% for ProteinSolver, 40.7% for ProDCoNN, and 50.7% for DenseCPD; the paper explicitly cautions that TS50 comparisons should be interpreted carefully because there is no canonical train/validation split.

For MQA, the task is regression to GDT-TS. The model is trained on 79,200 candidate structures for 528 CASP 5–10 targets and evaluated on CASP 11 and 12 benchmarks with 20,880 structures across 84 + 40 targets, as well as CASP 13 with 1,472 stage-2 structures across 20 targets [2009.01411]. The architecture uses 3 graph propagation steps, followed by a node-wise GVP reducing node embeddings to scalars, averaging over nodes, and a final dense network. Training uses a loss combining absolute Huber terms with a pairwise ranking-style Huber term:
$$
\mathscr{L} = H(y^{(i)}-\hat{y}^{(i)}) + H(y^{(j)}-\hat{y}^{(j)}) + H\left((y^{(i)}-y^{(j)})-(\hat{y}^{(i)}-\hat{y}^{(j)})\right).
$$

On CASP 11 stage 2, the reported global and per-target Pearson correlations are 0.87 and 0.45 for GVP-GNN, versus 0.82 and 0.38 for GraphQA, 0.64 and 0.40 for 3DCNN, 0.63 and 0.39 for Ornate, and 0.77 and 0.44 for ProQ3D. On CASP 12 stage 2, the corresponding values are 0.82 and 0.62 for GVP-GNN, versus 0.81 and 0.61 for GraphQA, 0.61 and 0.51 for 3DCNN, 0.67 and 0.49 for Ornate, and 0.81 and 0.60 for ProQ3D. On CASP 13, GVP-GNN reports 0.888 global and 0.671 per-target correlation, described as best overall in global correlation and best among structure-only methods in per-target correlation [2009.01411].

The 2021 extension broadens the application domain to the eight ATOM3D tasks: SMP, PPI, RES, MSP, LBA, LEP, PSR, and RSR [2106.03843]. It reports that GVP-GNN is best on PPI, RES, and MSP, tied for first on SMP and PSR, and competitive with higher-order equivariant networks. Specific numbers highlighted in the paper include PPI AUROC $0.866\pm 0.004$, RES accuracy $0.527\pm 0.003$, MSP AUROC $0.680\pm 0.015$, and PSR global $R_S=0.845\pm 0.008$ [2106.03843]. The same paper also studies transfer learning, reporting improvements such as PPI AUROC $0.866\to 0.874$ with SMP pretraining and MSP AUROC $0.680\to 0.711$ with SMP pretraining [2106.03843].

## 6. Architectural significance, efficiency, and limitations

The original GVP work positions the architecture as a lightweight alternative to equivariant methods based on irreducible representations of $SO(3)$ and higher-order tensors [2009.01411]. Instead of spherical harmonics or tensor fields of degree greater than one, GVPs use only scalars and vectors, with norms providing the scalar-vector interface. This choice was explicitly motivated by large biomolecules, where memory and scalability are significant constraints.

Reported implementation details in the protein paper include hidden node embeddings with 16 vector and 100 scalar channels, edge embeddings with 1 vector and 32 scalar channels, training on a single Titan X GPU, approximately two days to train each model, and GPU memory rather than compute as the bottleneck [2009.01411]. The architecture also introduces vector-channel dropout, which drops whole vector channels rather than individual coordinates, and vector layer normalization without trainable parameters, described as scaling row vectors so that their root-mean-square norm is one.

Ablation studies in the original paper support the dual scalar/vector design. Replacing GVP with an MLP hurts performance; using only scalar channels also hurts; using only vector channels hurts the most; and removing $\mathbf{W}_\mu$ causes a smaller but real drop [2009.01411]. The stated conclusion is that both scalar and vector channels are essential, as scalar features such as torsions and identities remain important while vector channels provide direct geometric access.

The 2021 paper places GVP-GNN in relation to higher-order equivariant neural networks such as Cormorant, describing GVP-GNN as a mathematically simpler alternative that uses only 1st-order tensors while remaining competitive with models using equivariant representations up to $L=3$ [2106.03843]. This suggests a tradeoff between representation order and implementation complexity. The paper also notes that equivariance is not uniformly beneficial across all tasks and hypothesizes that poorer performance on ligand-protein tasks such as LBA and LEP may result from deliberately standardized atomic input representations that omit bond information, bond type, and molecule identity in complexes.

A recurring limitation is that the architecture handles only scalar and vector features rather than general higher-order tensor features [2009.01411]. The theory in the original paper concerns invariant scalar-valued functions under specific assumptions, while the later vector-gated extension enlarges the corresponding expressive class for equivariant vector outputs [2106.03843]. In the original protein tasks, the representation is backbone-only, and the model is not presented as a full all-coordinate generative equivariant model.

## 7. Conceptual interpretation and subsequent role

GVPs encode a specific view of geometric learning in molecular systems: directional information should be represented as directional latent variables rather than reduced prematurely to handcrafted invariants. In proteins, this is especially natural because backbone orientation, local residue frame, edge direction, and long-range spatial arrangement all matter simultaneously [2009.01411]. The residue-level representation was described as using a minimal number of scalar and vector features sufficient to specify the 3D structure of the molecule at the backbone level, and as more efficient than storing all orientations only relative to all neighbors.

The synthetic experiments in the original work sharpen this interpretation. Three tasks—Off-center, Perimeter, and Combined—were designed to isolate geometric reasoning, relational reasoning, and both jointly. The reported standardized MSEs were 0.319 / 0.532 / 0.522 for a CNN, 0.871 / 0.128 / 0.421 for a GNN, and 0.206 / 0.106 / 0.155 for GVP-GNN [2009.01411]. The intended inference is that GVP-GNN combines the strengths of geometric and relational architectures more effectively than either class alone.

The atomic-level extension broadens the scope from residue graphs to a general macromolecular setting that includes proteins, RNAs, ligands, and complexes [2106.03843]. Its main technical addition, vector gating, addresses a practical asymmetry between residue-level and atom-level inputs: residues can be endowed with initial orientation, whereas atoms often cannot. This modification preserves the scalar/vector decomposition while giving scalar chemistry a direct route to instantiate directional hidden states. A plausible implication is that GVPs occupy a middle position between scalar-only message passing and higher-order equivariant field networks: they preserve essential 3D symmetry, remain compatible with standard GNN design patterns, and often achieve strong empirical performance without requiring spherical harmonic machinery.

In this sense, GVPs are best understood not as a general replacement for all equivariant architectures, but as a compact and task-effective formalism for graph-structured 3D learning when scalar identities and directional geometry must be modeled jointly [2009.01411; 2106.03843].

Source: https://www.emergentmind.com/topics/geometric-vector-perceptrons-gvps