Geometry-Embedding Module (GEM)

• Geometry-Embedding Module (GEM) is a design pattern that integrates explicit geometric priors, like camera poses or molecular bond angles, into learned representations.
• GEM addresses limitations of Euclidean feature pipelines by enhancing 3D consistency and physical plausibility in tasks such as video depth estimation and point cloud analysis.
• By incorporating domain-specific geometric variables, GEM improves model alignment, discrimination, and overall performance across diverse applications.

Searching arXiv for the cited GEM-related papers to ground the article in current records. arxiv_search(query="(Cheng et al., 11 May 2026) GemDepth Geometry-Embedded Features for 3D-Consistent Video Depth", max_results=5) Geometry-Embedding Module (GEM) denotes a family of architecture components that inject geometric structure into learned representations. The acronym is used in distinct research contexts, including video depth estimation, molecular representation learning, point-cloud processing, and dynamics modeling. In each case, GEM introduces an explicit geometric prior into an otherwise generic feature pipeline: in GemDepth, it embeds camera motion and scale into video features for 3D-consistent depth (Cheng et al., 11 May 2026); in ChemRL-GEM, it encodes molecular geometry through a double-graph GNN and geometry-level self-supervision (Fang et al., 2021); in PyramNet, it constructs covariance-based point-neighborhood embeddings for point-cloud classification and segmentation (Zhiheng et al., 2019); and in the Group Enhanced Model, it maps states to Lie-group structure and learns dynamics in the corresponding Lie algebra (Hansen-Estruch et al., 2021). The shared theme is not a single canonical module design, but the use of geometry-aware intermediate representations to improve consistency, discrimination, or physical plausibility.

1. Terminological scope and cross-domain meaning

The term GEM is not attached to one unique architecture. In "GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth" (Cheng et al., 11 May 2026), GEM refers to a Geometry-Embedding Module that predicts inter-frame camera poses and a global scale factor, converts them into implicit camera embeddings, and fuses those embeddings into visual features. In "ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction" (Fang et al., 2021), GEM refers to Geometry Enhanced Molecular representation learning, whose core is a geometry-based GNN architecture together with geometry-level self-supervised tasks. In "PyramNet: Point Cloud Pyramid Attention Network and Graph Embedding Module for Classification and Segmentation" (Zhiheng et al., 2019), GEM is a Graph Embedding Module for point clouds. In "GEM: Group Enhanced Model for Learning Dynamical Control Systems" (Hansen-Estruch et al., 2021), GEM denotes a Group Enhanced Model that exploits Lie groups and Lie algebras.

These uses are methodologically related but technically distinct. Each version of GEM addresses a limitation of representations built purely in Euclidean feature space or purely on topological connectivity. This suggests that “geometry embedding” is best understood as a design pattern rather than a single operator: the module injects domain-appropriate geometric variables—camera pose, bond angles, covariance-defined neighborhood structure, or group actions—into learned features so that downstream processing is constrained by physically or structurally meaningful relations.

A common misconception is that all GEM modules construct explicit geometric objects such as cost volumes, meshes, or point clouds. That is not generally the case. GemDepth states that GEM “does not build an explicit 3D cost volume or point cloud inside the network” and instead maps explicit pose and scale into feature space (Cheng et al., 11 May 2026). ChemRL-GEM uses invariant geometric scalars such as bond lengths, bond angles, and interatomic distances rather than raw coordinates (Fang et al., 2021). PyramNet’s GEM relies on covariance-based similarity among point features rather than explicit surface reconstruction (Zhiheng et al., 2019). The dynamics GEM learns in Lie algebra coordinates and enforces valid manifold transitions through the exponential map (Hansen-Estruch et al., 2021).

2. Geometry embedding in 3D-consistent video depth

In GemDepth, the Geometry-Embedding Module is introduced to remedy the fact that previous discriminative video depth models “model temporal consistency purely in 2D image space with temporal attention or smoothing, and do not explicitly know the camera motion or the 3D structure” (Cheng et al., 11 May 2026). The paper identifies several resulting failure modes: lack of explicit 3D awareness, unreliable alignment under camera rotations or large egomotion, global smoothing that blurs thin structures, and frame-to-frame scale ambiguity.

GEM addresses this by predicting explicit 6-DoF inter-frame camera poses plus a global scale factor in a unified canonical frame and encoding them into a compact camera feature embedding. In the GemDepth pipeline, a frozen DINOv2 ViT encoder extracts per-frame features $F_j$, GEM operates on the top-level features $F_4$ together with a learnable camera token, and outputs rotation $Q$, normalized translation $\hat{\mathbf{T}}$, global scale factor $\mathcal{Z}$, and an implicit camera embedding $F_{\text{cam}}$ (Cheng et al., 11 May 2026). These camera features are fused back into $F_4$, yielding geometry-embedded visual features that are passed to the Alternating Spatio-Temporal Transformer.

The architecture is “built upon a lightweight EfficientPoseNet,” instantiated as a 4-layer alternating-attention transformer that processes the high-level visual feature map $F_4$ and a learnable camera token $t \in \mathbb{R}^{(B \times N) \times 1 \times D}$ (Cheng et al., 11 May 2026). An MLP-based pose head predicts a 7-dimensional vector per frame,

$$g_i = f_{\text{pose}}(\tilde{t}_i) \in \mathbb{R}^7, \quad g_i = [Q_i, \mathbf{T}_i],$$

where $F_4$0 is a quaternion and $F_4$1 is translation. Poses are then canonicalized, and translation is normalized by a global scale factor,

$F_4$2

Rotations, normalized translations, and scale are mapped through separate geometric MLPs,

$F_4$3

and fused into visual features by

$F_4$4

The resulting representation is described as an implicit geometric embedding: explicit $F_4$5 pose and global scale are projected into the network’s feature space, where the subsequent transformer learns to use them for temporal alignment and correspondence (Cheng et al., 11 May 2026). ASTT operates directly on $F_4$6, adding RoPE for spatial positions and an inter-frame index embedding, then alternating temporal and spatial attention. GemDepth states that GEM acts like a geometry-aware conditioning or positional encoding, shifting the feature representation so that dot-product similarity becomes geometry-aware (Cheng et al., 11 May 2026).

Training couples explicit pose supervision and depth supervision. GEM is directly trained with a camera pose loss

$F_4$7

where both terms use the Huber loss, and depth is supervised with $F_4$8, $F_4$9, and $Q$0, combined as

$Q$1

with $Q$2, $Q$3, and $Q$4 (Cheng et al., 11 May 2026). The paper reports that adding GEM to Baseline+ASTT improves KITTI AbsRel from 0.080 to 0.074, Sintel AbsRel from 0.328 to 0.295, and Scannet TAE from 0.566 to 0.538, and that the full model attains Bonn F1 = 90.43 versus 78.44 for DA3 (Cheng et al., 11 May 2026). A plausible implication is that this form of GEM is not merely a regularizer for temporal smoothness, but a mechanism for encoding a shared metric frame across a sequence.

3. Geometry embedding in molecular representation learning

ChemRL-GEM is motivated by the claim that many molecular properties depend not only on graph topology but also on how atoms are arranged in 3D (Fang et al., 2021). The paper emphasizes that molecules with identical topology but different geometries, such as cis- and trans-isomers, can have very different properties, and that standard molecular GNNs and pre-training strategies often ignore bond angles and broader spatial structure.

Its core architectural component is a Geometry-based Graph Neural Network (GeoGNN) built on two coupled graphs. The first is an atom-bond graph $Q$5, whose nodes are atoms and edges are bonds. The second is a bond-angle graph $Q$6, whose nodes are bonds and whose edges are bond angles $Q$7 (Fang et al., 2021). Bond embeddings are updated using angle information in $Q$8, then reused as edge features in $Q$9 to update atom embeddings. Molecular embeddings are obtained by average pooling atom representations.

The message passing structure is explicit. For a bond $\hat{\mathbf{T}}$0, the bond-angle update is

$\hat{\mathbf{T}}$1

and in the concrete implementation uses GIN-style sum aggregation and an MLP (Fang et al., 2021). Atom updates then incorporate the geometry-aware bond embeddings through the atom-bond graph. The implementation uses 8 GeoGNN blocks for the atom-bond graph and 8 for the bond-angle graph, with hidden size 32 and 2-layer MLP combines of hidden size 32 (Fang et al., 2021).

Geometry enters both as input features and as pre-training targets. The paper uses RDKit’s MMFF94 to generate a single 3D conformation for each molecule and computes bond lengths $\hat{\mathbf{T}}$2, bond angles $\hat{\mathbf{T}}$3, and pairwise distances $\hat{\mathbf{T}}$4 (Fang et al., 2021). Continuous geometry features are encoded with an RBF expansion,

$\hat{\mathbf{T}}$5

with $\hat{\mathbf{T}}$6 and $\hat{\mathbf{T}}$7 on a uniform grid with stride 0.1 (Fang et al., 2021). Because the model uses distances and angles rather than raw coordinates, its representation is invariant to rotation and translation of the entire molecule.

A central contribution is geometry-level self-supervision. For bond length and bond angle prediction, 15% of atoms are selected, their 1-hop neighborhoods are masked, and the model predicts the masked local geometry: $\hat{\mathbf{T}}$8

$\hat{\mathbf{T}}$9

For global structure, pairwise distances are discretized into 30 bins and predicted with a cross-entropy loss,

$\mathcal{Z}$0

The pre-training loss is

$\mathcal{Z}$1

(Fang et al., 2021).

Empirically, ChemRL-GEM reports state-of-the-art performance on 11/12 MoleculeNet datasets, an overall relative improvement of 8.8% on regression tasks and 3.7% average ROC-AUC improvement on classification tasks over the previous best (Fang et al., 2021). GeoGNN without pre-training already yields an overall relative improvement of 7.9% versus the best previous architectures, and using accurate QM9 coordinates instead of RDKit conformers improves QM9 MAE from 0.00746 to 0.00652 (Fang et al., 2021). This suggests that, in this domain, GEM is best viewed as the combination of a geometry-aware architecture and geometry-aware self-supervision rather than merely an embedding layer in the narrow sense.

4. Graph embedding for point clouds

In PyramNet, GEM is a graph-based local geometry module designed for point-cloud classification and semantic segmentation (Zhiheng et al., 2019). The module takes point features produced by an MLP, projects them onto a graph, computes covariance-based similarity between points, selects the most correlated neighbors, pools their features, and concatenates the pooled representation with the original point features.

Given $\mathcal{Z}$2 points with pointwise features $\mathcal{Z}$3, the module constructs a covariance matrix $\mathcal{Z}$4,

$\mathcal{Z}$5

and, for each point $\mathcal{Z}$6, keeps the top-$\mathcal{Z}$7 correlated neighbors to form an adjacency similarity matrix $\mathcal{Z}$8 (Zhiheng et al., 2019). The feature map therefore expands from $\mathcal{Z}$9 to $F_{\text{cam}}$0. Global average pooling over the neighbor dimension yields $F_{\text{cam}}$1, which is concatenated with the original input so that the output becomes $F_{\text{cam}}$2. The paper summarizes this as: the “feature map of $F_{\text{cam}}$3 becomes $F_{\text{cam}}$4,” and after pooling and concatenation “the output of GEM becomes $F_{\text{cam}}$5” (Zhiheng et al., 2019).

PyramNet uses GEM twice. For classification, an initial MLP generates $F_{\text{cam}}$6 features, which are processed by the first GEM to yield $F_{\text{cam}}$7. After parallel MLP and Pyramid Attention Network branches are concatenated into $F_{\text{cam}}$8, a second GEM maps this to $F_{\text{cam}}$9 (Zhiheng et al., 2019). In segmentation, the second GEM output is further combined with earlier MLP features through shortcut connections.

The paper’s design rationale is that MLP-only point processing can miss local geometric relationships, while covariance-based similarity can connect points that are semantically similar in feature space rather than merely close in Euclidean space (Zhiheng et al., 2019). Relative to EdgeConv, GEM uses covariance similarity instead of Euclidean distance and uses average pooling plus concatenation rather than a learned edge function. Relative to PointNet++, it does not perform hierarchical subsampling but enriches features over the full point set.

Empirical results indicate that GEM supplies the larger individual contribution among PyramNet’s two new modules. On ModelNet40, Baseline+PAN improves overall accuracy from 89.6 to 89.9, while Baseline+GEM improves it from 89.6 to 90.6; the full PyramNet reaches 91.5 (Zhiheng et al., 2019). The ablation on the neighborhood size shows that $F_4$0 yields 91.5% overall accuracy, compared with 88.6 for $F_4$1 and 90.1 for $F_4$2 (Zhiheng et al., 2019). On S3DIS scene segmentation, PyramNet reaches 55.6 mIoU and 85.6 overall accuracy (Zhiheng et al., 2019). A plausible implication is that this GEM variant functions as a lightweight graph-construction and context-aggregation operator whose benefit depends strongly on the neighborhood-size heuristic.

5. Group-structured embedding for dynamical control systems

In the dynamics paper, GEM stands for Group Enhanced Model and is motivated by the observation that the state spaces of many physical systems have natural Lie-group structure (Hansen-Estruch et al., 2021). Joint angles live on $F_4$3 or $F_4$4, 3D orientations live on $F_4$5, and rigid poses live on $F_4$6. The paper hypothesizes that learning dynamics in the linear Lie algebra $F_4$7 is more effective than learning a direct state transition model in an unstructured Euclidean space.

The state is decomposed into static geometric components $F_4$8 and dynamic components $F_4$9, with $F_4$0 encoded as a group element $F_4$1 and $F_4$2 retained as a Euclidean vector (Hansen-Estruch et al., 2021). For a planar joint angle,

$F_4$3

while rigid-body pose is represented as a homogeneous transform in $F_4$4. The dynamics on the group are modeled as

$F_4$5

where $F_4$6 is a Lie algebra element expressed in a basis $F_4$7 (Hansen-Estruch et al., 2021).

The learned architecture comprises two modules. The Coefficient Model $F_4$8 predicts Lie algebra coefficients,

$F_4$9

and the Velocity Model $t \in \mathbb{R}^{(B \times N) \times 1 \times D}$0 predicts delta velocities,

$t \in \mathbb{R}^{(B \times N) \times 1 \times D}$1

The static state is then advanced analytically by

$t \in \mathbb{R}^{(B \times N) \times 1 \times D}$2

and velocities are updated by $t \in \mathbb{R}^{(B \times N) \times 1 \times D}$3 (Hansen-Estruch et al., 2021).

Training uses a group-space loss for the static component,

$t \in \mathbb{R}^{(B \times N) \times 1 \times D}$4

with the Frobenius norm for matrix-group representations, and a standard squared Euclidean loss for delta velocities (Hansen-Estruch et al., 2021). The total loss is $t \in \mathbb{R}^{(B \times N) \times 1 \times D}$5. The paper reports that GEMs “significantly outperform conventional transition models” on long-term prediction, planning, and model-based reinforcement learning, and that plugging GEM into PETS improves performance on complex environments such as Ant and Humanoid (Hansen-Estruch et al., 2021). Unlike the other GEM usages, geometry here is not about spatial locality or feature alignment but about respecting the manifold structure of state transitions.

6. Comparative design principles, limitations, and recurring themes

Despite the domain differences, the four GEM formulations exhibit several recurring design principles.

First, each GEM introduces explicit geometric variables into an otherwise generic learned representation. GemDepth injects pose and scale into video tokens (Cheng et al., 11 May 2026). ChemRL-GEM introduces bond lengths, bond angles, and interatomic distances into GNN message passing and pre-training (Fang et al., 2021). PyramNet’s GEM derives a graph from covariance relations among point features (Zhiheng et al., 2019). The dynamics GEM encodes system state in Lie groups and predicts algebra coefficients (Hansen-Estruch et al., 2021).

Second, each version attempts to improve behavior under failure modes associated with geometry-free processing. In video depth, purely 2D temporal smoothing fails under rotations and large egomotion and blurs thin structures (Cheng et al., 11 May 2026). In molecular learning, purely topological GNNs cannot distinguish many geometric isomers (Fang et al., 2021). In point-cloud learning, pointwise MLPs lack explicit local geometric structure (Zhiheng et al., 2019). In dynamical systems, Euclidean transition models do not preserve manifold constraints and accumulate long-horizon errors (Hansen-Estruch et al., 2021).

Third, the embedding may be either implicit or structural. GemDepth explicitly describes its representation as an implicit geometric embedding, where $t \in \mathbb{R}^{(B \times N) \times 1 \times D}$6 motion is projected into feature space (Cheng et al., 11 May 2026). ChemRL-GEM uses structural coupling between two graphs and supervised reconstruction of geometry (Fang et al., 2021). PyramNet’s embedding is structural and aggregation-based rather than learned through a parameterized attention mechanism (Zhiheng et al., 2019). The Group Enhanced Model uses analytic group operations, making the geometry hard-constrained rather than only feature-conditioned (Hansen-Estruch et al., 2021).

Limitations are likewise domain-specific. GemDepth is sensitive to pose quality, and performance degrades significantly under extreme pose noise; it models only camera ego-motion and not explicit object-wise motion (Cheng et al., 11 May 2026). ChemRL-GEM depends on 3D conformers, does not model torsional angles, and its distance-matrix task scales as $t \in \mathbb{R}^{(B \times N) \times 1 \times D}$7 (Fang et al., 2021). PyramNet’s GEM requires an $t \in \mathbb{R}^{(B \times N) \times 1 \times D}$8 covariance matrix and uses only average pooling across neighbors, making scalability and aggregation expressivity natural concerns (Zhiheng et al., 2019). The Group Enhanced Model assumes known geometric structure of the system and still treats velocities in Euclidean space (Hansen-Estruch et al., 2021).

Taken together, these works indicate that the term GEM identifies a broader methodological class: modules that alter representation spaces so that downstream computation is constrained by geometry, either through embeddings, graph constructions, or analytic transformations. A plausible implication is that future uses of the acronym will continue to vary by field while preserving the same underlying objective—embedding physically or structurally meaningful relations directly into the learned intermediate representation.