---
title: Geometry-Embedding Module (GEM)
url: https://www.emergentmind.com/topics/geometry-embedding-module-gem
type: topic
---

# Geometry-Embedding Module (GEM)

Searching arXiv for the cited GEM-related papers to ground the article in current records.
arxiv_search(query="2605.10525 GemDepth Geometry-Embedded Features for 3D-Consistent Video Depth", max_results=5)
Geometry-Embedding Module (GEM) denotes a family of architecture components that inject geometric structure into learned representations. The acronym is used in distinct research contexts, including video depth estimation, molecular representation learning, point-cloud processing, and dynamics modeling. In each case, GEM introduces an explicit geometric prior into an otherwise generic feature pipeline: in GemDepth, it embeds camera motion and scale into video features for 3D-consistent depth [2605.10525]; in ChemRL-GEM, it encodes molecular geometry through a double-graph GNN and geometry-level self-supervision [2106.06130]; in PyramNet, it constructs covariance-based point-neighborhood embeddings for point-cloud classification and segmentation [1906.03299]; and in the Group Enhanced Model, it maps states to Lie-group structure and learns dynamics in the corresponding Lie algebra [2104.02844]. The shared theme is not a single canonical module design, but the use of geometry-aware intermediate representations to improve consistency, discrimination, or physical plausibility.

## 1. Terminological scope and cross-domain meaning

The term GEM is not attached to one unique architecture. In "GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth" [2605.10525], GEM refers to a **Geometry-Embedding Module** that predicts inter-frame camera poses and a global scale factor, converts them into implicit camera embeddings, and fuses those embeddings into visual features. In "ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction" [2106.06130], GEM refers to **Geometry Enhanced Molecular representation learning**, whose core is a geometry-based GNN architecture together with geometry-level self-supervised tasks. In "PyramNet: Point Cloud Pyramid Attention Network and Graph Embedding Module for Classification and Segmentation" [1906.03299], GEM is a **Graph Embedding Module** for point clouds. In "GEM: Group Enhanced Model for Learning Dynamical Control Systems" [2104.02844], GEM denotes a **Group Enhanced Model** that exploits Lie groups and Lie algebras.

These uses are methodologically related but technically distinct. Each version of GEM addresses a limitation of representations built purely in Euclidean feature space or purely on topological connectivity. This suggests that “geometry embedding” is best understood as a design pattern rather than a single operator: the module injects domain-appropriate geometric variables—camera pose, bond angles, covariance-defined neighborhood structure, or group actions—into learned features so that downstream processing is constrained by physically or structurally meaningful relations.

A common misconception is that all GEM modules construct explicit geometric objects such as cost volumes, meshes, or point clouds. That is not generally the case. GemDepth states that GEM “does not build an explicit 3D cost volume or point cloud inside the network” and instead maps explicit pose and scale into feature space [2605.10525]. ChemRL-GEM uses invariant geometric scalars such as bond lengths, bond angles, and interatomic distances rather than raw coordinates [2106.06130]. PyramNet’s GEM relies on covariance-based similarity among point features rather than explicit surface reconstruction [1906.03299]. The dynamics GEM learns in Lie algebra coordinates and enforces valid manifold transitions through the exponential map [2104.02844].

## 2. Geometry embedding in 3D-consistent video depth

In GemDepth, the Geometry-Embedding Module is introduced to remedy the fact that previous discriminative video depth models “model temporal consistency purely in 2D image space with temporal attention or smoothing, and do not explicitly know the camera motion or the 3D structure” [2605.10525]. The paper identifies several resulting failure modes: lack of explicit 3D awareness, unreliable alignment under camera rotations or large egomotion, global smoothing that blurs thin structures, and frame-to-frame scale ambiguity.

GEM addresses this by predicting explicit **6-DoF inter-frame camera poses plus a global scale factor** in a unified canonical frame and encoding them into a compact camera feature embedding. In the GemDepth pipeline, a frozen DINOv2 ViT encoder extracts per-frame features \(F_j\), GEM operates on the top-level features \(F_4\) together with a learnable camera token, and outputs rotation \(Q\), normalized translation \(\hat{\mathbf{T}}\), global scale factor \(\mathcal{Z}\), and an implicit camera embedding \(F_{\text{cam}}\) [2605.10525]. These camera features are fused back into \(F_4\), yielding geometry-embedded visual features that are passed to the Alternating Spatio-Temporal Transformer.

The architecture is “built upon a lightweight EfficientPoseNet,” instantiated as a **4-layer alternating-attention transformer** that processes the high-level visual feature map \(F_4\) and a learnable camera token \(t \in \mathbb{R}^{(B \times N) \times 1 \times D}\) [2605.10525]. An MLP-based pose head predicts a 7-dimensional vector per frame,
\[
g_i = f_{\text{pose}}(\tilde{t}_i) \in \mathbb{R}^7, \quad g_i = [Q_i, \mathbf{T}_i],
\]
where \(Q_i \in \mathbb{R}^4\) is a quaternion and \(\mathbf{T}_i \in \mathbb{R}^3\) is translation. Poses are then canonicalized, and translation is normalized by a global scale factor,
\[
\mathcal{Z} = \sum_i \lVert \mathbf{T}_i \rVert_1, \quad \hat{\mathbf{T}}_i = \frac{\mathbf{T}_i}{\mathcal{Z}}.
\]
Rotations, normalized translations, and scale are mapped through separate geometric MLPs,
\[
F_{\text{cam},i} = \phi_Q(Q_i) + \phi_T(\hat{\mathbf{T}}_i) + \phi_Z(\mathcal{Z}_i) \in \mathbb{R}^D,
\]
and fused into visual features by
\[
\tilde{F}_4(i, x) = F_4(i, x) + W_{\text{fuse}} F_{\text{cam},i}.
\]

The resulting representation is described as an **implicit geometric embedding**: explicit \(SE(3)\) pose and global scale are projected into the network’s feature space, where the subsequent transformer learns to use them for temporal alignment and correspondence [2605.10525]. ASTT operates directly on \(\tilde{F}_4\), adding RoPE for spatial positions and an inter-frame index embedding, then alternating temporal and spatial attention. GemDepth states that GEM acts like a geometry-aware conditioning or positional encoding, shifting the feature representation so that dot-product similarity becomes geometry-aware [2605.10525].

Training couples explicit pose supervision and depth supervision. GEM is directly trained with a camera pose loss
\[
L_{\text{cam}} = \frac{1}{N} \sum_{i=1}^{N} \left( L_{\text{rot}} + \lambda L_{\text{trans}} \right),
\]
where both terms use the Huber loss, and depth is supervised with \(L_{\text{ssi}}\), \(L_{\text{gm}}\), and \(L_{\text{tgm}}\), combined as
\[
L_{\text{total}} = L_{\text{ssi}} + \alpha L_{\text{gm}} + \beta L_{\text{tgm}} + \gamma L_{\text{cam}}
\]
with \(\alpha = 0.5\), \(\beta = 10\), and \(\gamma = 0.2\) [2605.10525]. The paper reports that adding GEM to Baseline+ASTT improves KITTI AbsRel from 0.080 to 0.074, Sintel AbsRel from 0.328 to 0.295, and Scannet TAE from 0.566 to 0.538, and that the full model attains Bonn F1 = 90.43 versus 78.44 for DA3 [2605.10525]. A plausible implication is that this form of GEM is not merely a regularizer for temporal smoothness, but a mechanism for encoding a shared metric frame across a sequence.

## 3. Geometry embedding in molecular representation learning

ChemRL-GEM is motivated by the claim that many molecular properties depend not only on graph topology but also on **how atoms are arranged in 3D** [2106.06130]. The paper emphasizes that molecules with identical topology but different geometries, such as cis- and trans-isomers, can have very different properties, and that standard molecular GNNs and pre-training strategies often ignore bond angles and broader spatial structure.

Its core architectural component is a **Geometry-based Graph Neural Network (GeoGNN)** built on two coupled graphs. The first is an atom-bond graph \(G = (\mathcal V, \mathcal E)\), whose nodes are atoms and edges are bonds. The second is a bond-angle graph \(H = (\mathcal E, \mathcal A)\), whose nodes are bonds and whose edges are bond angles \((u,v,w)\) [2106.06130]. Bond embeddings are updated using angle information in \(H\), then reused as edge features in \(G\) to update atom embeddings. Molecular embeddings are obtained by average pooling atom representations.

The message passing structure is explicit. For a bond \((u,v)\), the bond-angle update is
\[
\begin{split}
a_{uv}^{(k)} = &\ AGGREGATE_{\text{bond-angle}}^{(k)}\Big(\{(h_{uv}^{(k-1)}, h_{uw}^{(k-1)}, x_{wuv}) : w \in \mathcal N(u)\} \\
&\quad\cup \{(h_{uv}^{(k-1)}, h_{vw}^{(k-1)}, x_{uvw}) : w \in \mathcal N(v)\}\Big), \\
h_{uv}^{(k)} = &\ COMBINE_{\text{bond-angle}}^{(k)}(h_{uv}^{(k-1)}, a_{uv}^{(k)}),
\end{split}
\]
and in the concrete implementation uses GIN-style sum aggregation and an MLP [2106.06130]. Atom updates then incorporate the geometry-aware bond embeddings through the atom-bond graph. The implementation uses **8 GeoGNN blocks for the atom-bond graph and 8 for the bond-angle graph**, with hidden size 32 and 2-layer MLP combines of hidden size 32 [2106.06130].

Geometry enters both as input features and as pre-training targets. The paper uses RDKit’s MMFF94 to generate a **single 3D conformation** for each molecule and computes bond lengths \(l_{uv}\), bond angles \(\phi_{uvw}\), and pairwise distances \(d_{uv}\) [2106.06130]. Continuous geometry features are encoded with an RBF expansion,
\[
e_m(x) = \exp(-\gamma \|x - \mu_m\|^2), \quad m=1,\dots,M,
\]
with \(\gamma = 10\) and \(\{\mu_m\}\) on a uniform grid with stride 0.1 [2106.06130]. Because the model uses distances and angles rather than raw coordinates, its representation is invariant to rotation and translation of the entire molecule.

A central contribution is geometry-level self-supervision. For bond length and bond angle prediction, 15% of atoms are selected, their 1-hop neighborhoods are masked, and the model predicts the masked local geometry:
\[
L_{\text{length}}(\mathcal E) = \frac{1}{|\mathcal E|} \sum_{(u,v)\in \mathcal E} \big(f_{\text{length}}(h_u^{(K)}, h_v^{(K)}) - l_{uv}\big)^2,
\]
\[
L_{\text{angle}}(\mathcal A) = \frac{1}{|\mathcal A|} \sum_{(u,v,w)\in \mathcal A} \big(f_{\text{angle}}(h_u^{(K)}, h_v^{(K)}, h_w^{(K)}) - \phi_{uvw}\big)^2.
\]
For global structure, pairwise distances are discretized into 30 bins and predicted with a cross-entropy loss,
\[
L_{\text{distance}}(\mathcal V) = \frac{1}{|\mathcal V|^2} \sum_{u,v \in \mathcal V} -\, bin^T(d_{uv}) \cdot \log\big(f_{\text{distance}}(h_u^{(K)}, h_v^{(K)})\big).
\]
The pre-training loss is
\[
L_{\text{pre}}(G,H) = L_{\text{length}}(\mathcal E) + L_{\text{angle}}(\mathcal A) + L_{\text{distance}}(\mathcal V)
\]
[2106.06130].

Empirically, ChemRL-GEM reports state-of-the-art performance on **11/12** MoleculeNet datasets, an **overall relative improvement of 8.8%** on regression tasks and **3.7%** average ROC-AUC improvement on classification tasks over the previous best [2106.06130]. GeoGNN without pre-training already yields an **overall relative improvement of 7.9%** versus the best previous architectures, and using accurate QM9 coordinates instead of RDKit conformers improves QM9 MAE from 0.00746 to 0.00652 [2106.06130]. This suggests that, in this domain, GEM is best viewed as the combination of a geometry-aware architecture and geometry-aware self-supervision rather than merely an embedding layer in the narrow sense.

## 4. Graph embedding for point clouds

In PyramNet, GEM is a **graph-based local geometry module** designed for point-cloud classification and semantic segmentation [1906.03299]. The module takes point features produced by an MLP, projects them onto a graph, computes covariance-based similarity between points, selects the most correlated neighbors, pools their features, and concatenates the pooled representation with the original point features.

Given \(N\) points with pointwise features \(P = \{P_i\}_{i=1}^N \subset \mathbb{R}^F\), the module constructs a covariance matrix \(S \in \mathbb{R}^{N \times N}\),
\[
S_{ij} = \text{conv}(P_i, P_j) = \mathbb{E}\left[ (P_i - \mu_A^i)(P_j - \mu_A^j) \right],
\]
and, for each point \(i\), keeps the top-\(k\) correlated neighbors to form an adjacency similarity matrix \(M_{AS} \subseteq \mathbb{R}^{N \times k}\) [1906.03299]. The feature map therefore expands from \(N \times 1 \times F\) to \(N \times k \times F\). Global average pooling over the neighbor dimension yields \(\tilde{X} \in \mathbb{R}^{N \times 1 \times F}\), which is concatenated with the original input so that the output becomes \(N \times 1 \times 2F\). The paper summarizes this as: the “feature map of \(N \times 1 \times F\) becomes \(N \times k \times F\),” and after pooling and concatenation “the output of GEM becomes \(N \times 2F\)” [1906.03299].

PyramNet uses GEM twice. For classification, an initial MLP generates \(N \times 1 \times 32\) features, which are processed by the first GEM to yield \(N \times 64\). After parallel MLP and Pyramid Attention Network branches are concatenated into \(N \times 1 \times 544\), a second GEM maps this to \(N \times 1 \times 1088\) [1906.03299]. In segmentation, the second GEM output is further combined with earlier MLP features through shortcut connections.

The paper’s design rationale is that MLP-only point processing can miss local geometric relationships, while covariance-based similarity can connect points that are semantically similar in feature space rather than merely close in Euclidean space [1906.03299]. Relative to EdgeConv, GEM uses covariance similarity instead of Euclidean distance and uses average pooling plus concatenation rather than a learned edge function. Relative to PointNet++, it does not perform hierarchical subsampling but enriches features over the full point set.

Empirical results indicate that GEM supplies the larger individual contribution among PyramNet’s two new modules. On ModelNet40, Baseline+PAN improves overall accuracy from 89.6 to 89.9, while **Baseline+GEM** improves it from 89.6 to **90.6**; the full PyramNet reaches **91.5** [1906.03299]. The ablation on the neighborhood size shows that \(k = \left\lceil \frac{F}{4} \right\rceil\) yields **91.5%** overall accuracy, compared with 88.6 for \(k=20\) and 90.1 for \(k=30\) [1906.03299]. On S3DIS scene segmentation, PyramNet reaches **55.6** mIoU and **85.6** overall accuracy [1906.03299]. A plausible implication is that this GEM variant functions as a lightweight graph-construction and context-aggregation operator whose benefit depends strongly on the neighborhood-size heuristic.

## 5. Group-structured embedding for dynamical control systems

In the dynamics paper, GEM stands for **Group Enhanced Model** and is motivated by the observation that the state spaces of many physical systems have natural Lie-group structure [2104.02844]. Joint angles live on \(S^1\) or \(SO(2)\), 3D orientations live on \(SO(3)\), and rigid poses live on \(SE(3)\). The paper hypothesizes that learning dynamics in the linear Lie algebra \(\mathfrak g\) is more effective than learning a direct state transition model in an unstructured Euclidean space.

The state is decomposed into static geometric components \(\theta_t\) and dynamic components \(\omega_t\), with \(\theta_t\) encoded as a group element \(G_t \in G\) and \(\omega_t\) retained as a Euclidean vector [2104.02844]. For a planar joint angle,
\[
R(\theta) =
\begin{bmatrix}
\cos\theta & -\sin\theta \\
\sin\theta & \cos\theta
\end{bmatrix}
\in SO(2),
\]
while rigid-body pose is represented as a homogeneous transform in \(SE(3)\). The dynamics on the group are modeled as
\[
G_{t+1} = \exp(g(\alpha_t)) \circ G_t,
\]
where \(g(\alpha_t) = \sum_{i=1}^K \alpha_{t,i} E_i\) is a Lie algebra element expressed in a basis \(E = \{E_1,\dots,E_K\}\) [2104.02844].

The learned architecture comprises two modules. The **Coefficient Model** \(\mathcal{L}_\phi^\alpha\) predicts Lie algebra coefficients,
\[
\hat{\alpha}_t = \mathcal{L}_\phi^\alpha(G_t, \omega_t, a_t),
\]
and the **Velocity Model** \(\mathcal{L}_\psi^\omega\) predicts delta velocities,
\[
\Delta \hat{\omega}_t = \mathcal{L}_\psi^\omega(G_t, \omega_t, \hat{\alpha}_t, a_t).
\]
The static state is then advanced analytically by
\[
\hat{G}_{t+1} = \exp\bigl(g(\hat{\alpha}_t)\bigr) \circ G_t,
\]
and velocities are updated by \(\hat{\omega}_{t+1} = \omega_t + \Delta \hat{\omega}_t\) [2104.02844].

Training uses a group-space loss for the static component,
\[
L^\alpha(\phi) = \frac{1}{T}\sum_{t=1}^T
\tilde{l}\Bigl(\exp(g(\hat{\alpha}_t)) \circ G_t,\; G_{t+1}\Bigr),
\]
with the Frobenius norm for matrix-group representations, and a standard squared Euclidean loss for delta velocities [2104.02844]. The total loss is \(L(\phi,\psi) = L^\alpha(\phi) + L^\omega(\psi)\). The paper reports that GEMs “significantly outperform conventional transition models” on long-term prediction, planning, and model-based reinforcement learning, and that plugging GEM into PETS improves performance on complex environments such as Ant and Humanoid [2104.02844]. Unlike the other GEM usages, geometry here is not about spatial locality or feature alignment but about respecting the manifold structure of state transitions.

## 6. Comparative design principles, limitations, and recurring themes

Despite the domain differences, the four GEM formulations exhibit several recurring design principles.

First, each GEM introduces **explicit geometric variables** into an otherwise generic learned representation. GemDepth injects pose and scale into video tokens [2605.10525]. ChemRL-GEM introduces bond lengths, bond angles, and interatomic distances into GNN message passing and pre-training [2106.06130]. PyramNet’s GEM derives a graph from covariance relations among point features [1906.03299]. The dynamics GEM encodes system state in Lie groups and predicts algebra coefficients [2104.02844].

Second, each version attempts to improve behavior under failure modes associated with geometry-free processing. In video depth, purely 2D temporal smoothing fails under rotations and large egomotion and blurs thin structures [2605.10525]. In molecular learning, purely topological GNNs cannot distinguish many geometric isomers [2106.06130]. In point-cloud learning, pointwise MLPs lack explicit local geometric structure [1906.03299]. In dynamical systems, Euclidean transition models do not preserve manifold constraints and accumulate long-horizon errors [2104.02844].

Third, the embedding may be either **implicit** or **structural**. GemDepth explicitly describes its representation as an implicit geometric embedding, where \(SE(3)\) motion is projected into feature space [2605.10525]. ChemRL-GEM uses structural coupling between two graphs and supervised reconstruction of geometry [2106.06130]. PyramNet’s embedding is structural and aggregation-based rather than learned through a parameterized attention mechanism [1906.03299]. The Group Enhanced Model uses analytic group operations, making the geometry hard-constrained rather than only feature-conditioned [2104.02844].

Limitations are likewise domain-specific. GemDepth is sensitive to pose quality, and performance degrades significantly under extreme pose noise; it models only camera ego-motion and not explicit object-wise motion [2605.10525]. ChemRL-GEM depends on 3D conformers, does not model torsional angles, and its distance-matrix task scales as \(O(|\mathcal V|^2)\) [2106.06130]. PyramNet’s GEM requires an \(O(N^2)\) covariance matrix and uses only average pooling across neighbors, making scalability and aggregation expressivity natural concerns [1906.03299]. The Group Enhanced Model assumes known geometric structure of the system and still treats velocities in Euclidean space [2104.02844].

Taken together, these works indicate that the term GEM identifies a broader methodological class: modules that alter representation spaces so that downstream computation is constrained by geometry, either through embeddings, graph constructions, or analytic transformations. A plausible implication is that future uses of the acronym will continue to vary by field while preserving the same underlying objective—embedding physically or structurally meaningful relations directly into the learned intermediate representation.

Source: https://www.emergentmind.com/topics/geometry-embedding-module-gem