Geometric Deep Learning

• Geometric deep learning is a field that applies deep learning to non-Euclidean data by leveraging symmetry, topology, and geometric structures.
• It employs spatial and spectral architectures to integrate local and global inductive biases for enhanced data efficiency and generalization.
• Applications include molecular modeling, 3D computer vision, neuroscience, and weather forecasting, driving advances in data-driven science.

Geometric deep learning (GDL) is the field concerned with the extension of deep neural networks to data domains that exhibit non-Euclidean structure, such as graphs, manifolds, mesh surfaces, point clouds, and broader geometric objects. Unlike the classical regime of Euclidean data (images, grids, sequences), these domains require architectures that leverage their intrinsic symmetry groups, local and global topologies, and geometric structure. The unifying principle of GDL is to encode symmetries and inductive biases dictated by the geometry of the underlying data domain into the network design, yielding improved data efficiency, generalization, and theoretical tractability (Bronstein et al., 2016, Bronstein et al., 2021, Gerken et al., 2021).

1. Mathematical Foundations: Symmetries, Equivariance, and Invariance

The core mathematical concept in GDL is the group-theoretic abstraction of symmetry. Given a domain Ω and a group $G$ acting on Ω (e.g., translations on images, permutations on graphs, rotations on spheres), a function or layer $F$ is said to be G-equivariant if

$$F(\rho(g) x) = \rho'(g) F(x), \quad \forall g \in G$$

where $\rho, \rho'$ denote group representations on the input and output spaces, respectively. If $\rho'$ is trivial, $F$ is G-invariant. This principle, which subsumes both invariance and equivariance under the relevant transformations, underlies CNNs (shift equivariance), GNNs (permutation equivariance), and manifold networks (isometry/gauge equivariance) (Gerken et al., 2021, Bronstein et al., 2021, Atz et al., 2021).

Homogeneous spaces, fiber bundles, and gauge structures provide the geometric and algebraic substrate for GDL. For example, the sphere $S^2$ is realized as $SO(3)/SO(2)$; convolution on $S^2$ leverages spherical harmonics and Wigner D-matrix representations to enforce $SO(3)$ equivariance (Gerken et al., 2021).

2. Architectures: From Grids to Graphs, Manifolds, and Gauge Bundles

The five canonical domains of GDL, as unified in the "Erlangen Program" perspective, are: (1) Grids and Euclidean CNNs, (2) Group convolutional networks (G-CNNs), (3) Graph neural networks (GNNs), (4) Manifold and geodesic networks, (5) Gauge-equivariant architectures (Bronstein et al., 2021, Gerken et al., 2021).

Spatial GDL methods define local chart-like neighborhoods (patch operators, e.g., geodesic CNNs [Masci et al.]) or message-passing mechanisms (GAT, GraphSAGE) in which features are aggregated from the relevant localities according to the domain topology or adjacency (Bronstein et al., 2016, Heidari et al., 2024).

Spectral GDL designs filters in the (graph or Laplace–Beltrami) eigenbasis—using the (generalized) Fourier transform on the domain. Spectral filters correspond to functions of the Laplacian eigenvalues and are efficiently implemented via polynomial approximations (e.g., ChebNet, GCN) on graphs (Bronstein et al., 2016, Heidari et al., 2024).

Equivariant and Gauge-Equivariant Models generalize these ideas to arbitrary Lie groups and local gauge transformations, allowing for SE(3)-equivariant GNNs (on 3D point clouds, molecules), spherical CNNs, and mesh-based networks with local tangent frames (Gerken et al., 2021, Sommer et al., 2019). Convolution is constructed either in position space via parallel transport or in frequency space via spherical/irreducible representations.

Fibration Symmetries: Recent advances formalize both global (automorphism group) and local (fibration) symmetries as graph fibrations and corresponding node partitions, revealing a hierarchy of inductive biases—permutation invariance is global; fibration is local, tightly matching the bias of GNN architectures (Velarde et al., 2024).

3. Applications and Empirical Methodologies

GDL methods have demonstrated impact across domains:

• Molecular and chemical modeling: SE(3)- and E(3)-equivariant GNNs dominate property prediction and reaction modeling, with models such as 3DReact employing explicit irreducible O(3) representations, tensor products, and equivariant attention across atomic structures (Gerwen et al., 2023, Atz et al., 2021, Isert et al., 2022). GDL has also enabled advances in protein engineering, drug design, and annotation of macromolecular interactions (García-Vinuesa et al., 19 Jun 2025, Isert et al., 2022).
• 3D Computer Vision and CAD: Tasks include mesh segmentation, CAD assembly analysis, shape generation, and reverse engineering, leveraging both B-Rep graph representations and point-cloud approaches on datasets such as ABC, ModelNet, and Fusion360 (Heidari et al., 2024, Koch et al., 2018).
• Neuroscience and Medical Imaging: Graph convolutional and manifold learning approaches for cortical surface registration (e.g., GeoMorph uses MoNet spatial graph convolutions and deep conditional random fields) achieve smoother, faster, and more accurate alignment than classical methods (Suliman et al., 2023).
• Physical Systems and Weather Forecasting: Spatio-temporal GNNs, learning both temporal filters and non-Euclidean spatial adjacencies, provide improved accuracy for tasks such as precipitation nowcasting compared to traditional ConvLSTM architectures (Zhao et al., 2023).
• Fundamental Theory: Mathematical analyses of DNN expressivity use notions of rectified linear complexity and affine spline tessellation to relate network geometry to manifold complexity and function approximation (Balestriero et al., 2024, Lei et al., 2018).

4. Theoretical Expressivity and Inductive Biases

The expressivity of GDL models is determined by the combination of architectural capacity and the symmetry constraints. Message-passing GNNs are bounded by the discriminative power of graph fibrations, with fibration-based reductions giving both an upper bound on distinguishability and an algorithmic node-compression technique (Velarde et al., 2024).

For E(3)-invariant functions on point clouds, the paper "On the Completeness of Invariant Geometric Deep Learning Models" provides necessary and sufficient conditions for completeness: standard distance-based MPNNs are nearly E(3)-complete except for pathologically symmetric cases; completeness is achieved by augmenting with local anchor information, as in GeoNGNN, and more elaborate models (DimeNet, GemNet, SphereNet) are E(3)-complete under appropriate assumptions (Li et al., 2024).

Mathematically, the spline-based and rectified linear analysis quantifies the network's partitioning of input space and the minimal architectural complexity required to realize a given manifold structure, thus setting limits for any fixed architecture (Balestriero et al., 2024, Lei et al., 2018).

5. Empirical Benchmarks, Datasets, and Open Problems

Large high-quality datasets specifically tailored to GDL—such as ABC (for CAD), ABC-Dataset (analytical ground-truth for surface normal/curvature estimation), ShapeNet (3D shapes), and cheminformatics benchmarks (e.g., MoleculeNet, PDBbind)—enable systematic evaluation (Koch et al., 2018, Heidari et al., 2024). For molecular systems, standard regression and classification metrics (RMSE, MAE, AUC-ROC) are adopted, with new benchmarks for docking, pose accuracy, and de novo generation (Gerwen et al., 2023, Isert et al., 2022, García-Vinuesa et al., 19 Jun 2025).

A comparative analysis reveals that data-driven models—while competitive on raw point clouds—can underperform simple analytic methods when adjacency or mesh topology is present (e.g., area-weighted mesh normals far exceed learned estimators on well-conditioned CAD surfaces) (Koch et al., 2018). This exposes gaps in current GDL models’ capacity to learn certain linear geometric operators.

Key challenges remain in scalability (memory and computation costs for full equivariant layers), transferability (domain differences in Laplacian spectra or coordinate systems), stability (model robustness to graph or mesh perturbations), data scarcity (especially for quantum and biological data), applicability to highly heterogeneous or dynamic structures, and the explicit incorporation of uncertainty and interpretability (Atz et al., 2021, Bronstein et al., 2016, García-Vinuesa et al., 19 Jun 2025).

6. Advanced Directions and Local vs Global Symmetries

The mathematical structure of GDL is now being generalized beyond global group-based invariance. Local symmetries—formally modeled as graph fibrations—capture the regularities of input neighborhoods and explain both the bounds and limitations of classical GNNs (Velarde et al., 2024). Gauge-equivariant networks leverage principal-bundle formalism to support local chart changes (gauges) and fiber symmetries on manifolds, yielding architectures naturally suited to surface and mesh data (Gerken et al., 2021, Sommer et al., 2019).

Manifold stochasticity and horizontal flows (e.g., horizontal transport of filters across the frame bundle) offer curvature-aware, gauge-equivariant constructions of convolution and mean-pooling operators. These generalizations allow for smooth averaging across holonomy classes and local geometry, surpassing restrictions of parallel transport along shortest geodesics (Sommer et al., 2019).

A further frontier is the integration of domain-specific priors, uncertainty quantification, and explainability into the GDL pipeline—especially for protein engineering, where high-consequence design decisions must be backed by both geometric reasoning and interpretable model outputs (García-Vinuesa et al., 19 Jun 2025).

7. Synthesis: "Erlangen Program" for Deep Learning

Geometric deep learning embodies the Kleinian view: "geometry as the study of properties invariant under a group of transformations." The systematic use of equivariance, invariance, and inductive bias from the data domain’s symmetry group is now the dominant paradigm for the design of modern neural architectures on non-Euclidean data (Bronstein et al., 2021). This synthesis drives advances across natural sciences, engineering, and data-rich applied fields by uniting mathematical rigor, computational efficiency, and domain-specific geometric priors.

---

References

• (Bronstein et al., 2016) Geometric deep learning: going beyond Euclidean data
• (Bronstein et al., 2021) Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• (Gerken et al., 2021) Geometric Deep Learning and Equivariant Neural Networks
• (Atz et al., 2021) Geometric Deep Learning on Molecular Representations
• (Heidari et al., 2024) Geometric Deep Learning for Computer-Aided Design: A Survey
• (Koch et al., 2018) ABC: A Big CAD Model Dataset For Geometric Deep Learning
• (Velarde et al., 2024) The Role of Fibration Symmetries in Geometric Deep Learning
• (Zhao et al., 2023) Exploring Geometric Deep Learning For Precipitation Nowcasting
• (Suliman et al., 2023) Unsupervised Multimodal Surface Registration with Geometric Deep Learning
• (Gerwen et al., 2023) 3DReact: Geometric deep learning for chemical reactions
• (Li et al., 2024) On the Completeness of Invariant Geometric Deep Learning Models
• (Isert et al., 2022) Structure-based drug design with geometric deep learning
• (Sommer et al., 2019) Horizontal Flows and Manifold Stochastics in Geometric Deep Learning
• (Balestriero et al., 2024) On the Geometry of Deep Learning
• (Lei et al., 2018) Geometric Understanding of Deep Learning
• (García-Vinuesa et al., 19 Jun 2025) Geometric deep learning assists protein engineering. Opportunities and Challenges
• (Kang, 2 Sep 2025) Doctoral Thesis: Geometric Deep Learning For Camera Pose Prediction, Registration, Depth Estimation, and 3D Reconstruction