Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves

Abstract: Modern deep learning architectures increasingly contend with sophisticated signals that are natively infinite-dimensional, such as time series, probability distributions, or operators, and are defined over irregular domains. Yet, a unified learning theory for these settings has been lacking. To start addressing this gap, we introduce a novel convolutional learning framework for possibly infinite-dimensional signals supported on a manifold. Namely, we use the connection Laplacian associated with a Hilbert bundle as a convolutional operator, and we derive filters and neural networks, dubbed as \textit{HilbNets}. We make HilbNets and, more generally, the convolution operation, implementable via a two-stage sampling procedure. First, we show that sampling the manifold induces a Hilbert Cellular Sheaf, a generalized graph structure with Hilbert feature spaces and edge-wise coupling rules, and we prove that its sheaf Laplacian converges in probability to the underlying connection Laplacian as the sampling density increases. Notably, this result is a generalization to the infinite-dimensional bundle setting of the Belkin & Niyogi \cite{BELKIN20081289} convergence result for the graph Laplacian to the manifold Laplacian, a theoretical cornerstone of geometric learning methods. Second, we discretize the signals and prove that the discretized (implementable) HilbNets converge to the underlying continuous architectures and are transferable across different samplings of the same bundle, providing consistency for learning. Finally, we validate our framework on synthetic and real-world tasks. Overall, our results broaden the scope of geometric learning as a whole by lifting classical Laplacian-based frameworks to settings where the signal at each point lives in its own Hilbert space.

• The paper presents HilbNets, a novel deep learning architecture that generalizes Laplacian-based convolutional methods to infinite-dimensional signals using Hilbert bundles.
• It introduces a two-level discretization via cellular sheaf theory, ensuring convergence and robustness from the continuous connection Laplacian to its discrete sheaf counterpart.
• Numerical validations on synthetic datasets and traffic forecasting benchmarks confirm enhanced accuracy, transferability, and parameter efficiency compared to traditional geometric models.

Consistent Geometric Deep Learning with Hilbert Bundles and Cellular Sheaves

Introduction and Problem Setting

The paper "Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves" (2605.06395) establishes a rigorous framework for geometric deep learning on infinite-dimensional signals—such as spatiotemporal fields and distributions—supported on manifolds. Previous geometric learning architectures (e.g., GCNs, CNNs on manifolds, sheaf NNs) largely consider finite-dimensional data over structured domains (graphs, simplicial complexes, or manifolds). However, practical datasets frequently require processing infinite-dimensional signals indexed on non-Euclidean domains (for instance, timeseries at each point in a spatial domain, or measures assigned to nodes).

This work closes the theoretical and practical gap by generalizing Laplacian-based convolutional architectures to signals valued in Hilbert bundles, architecturally realized as "HilbNets." The authors provide a principled discretization—via cellular sheaf theory—for practical deployment, accompanied by strong probabilistic consistency and transferability results. They further validate their theory on synthetic and real problems, including traffic forecasting.

Hilbert Bundles and Generalized Convolution

Signals of interest are modeled as smooth sections of a Hilbert bundle $\mathcal{E}$ over a closed Riemannian manifold $\mathcal{M}$, i.e., the value at $x\in\mathcal{M}$ lies in the Hilbert space $\mathcal{E}_x$. This abstraction captures infinite-dimensional signal modalities (e.g., $L^2$ functions or distributions at each $x$). The core geometric operator is the connection Laplacian $\Delta_\nabla$, defined using a Fréchet connection $\nabla$ compatible with the bundle metric, whose spectral properties generalize the classical Laplacian and enable geometry-aware filtering.

Convolutional filtering is defined via the Borel functional calculus: for a bounded Borel function $g$, the filtered signal is $g(\Delta_\nabla)S$. Neural architectures, HilbNets, are then compositions of such filters and (Lipschitz) nonlinearities along the fibers. This formalism unifies CNNs, GCNs, manifold CNNs, vector-field networks, and other geometric deep learning models as special or limiting cases.

Figure 1: Overview of the HilbNets framework. A HilbNet is a convolutional neural network processing infinite-dimensional signals supported on $\mathcal{M}$0; the discrete architecture uses a sampled sheaf Laplacian consistent with the continuous connection Laplacian.

Discretization via Hilbert Cellular Sheaves

Direct implementation is infeasible due to the infinite-dimensional and continuous nature of $\mathcal{M}$1 and the fibers. The authors introduce a two-level discretization using cellular sheaf theory:

• Spatial sampling: Sample $\mathcal{M}$2 points $\mathcal{M}$3, creating a graph where each node corresponds to a sampled location and each edge encodes proximity.
• Fiber discretization: Signals on each node are discretized in a fixed finite-dimensional subspace of the Hilbert fiber, yielding a signal of dimension $\mathcal{M}$4.

A Hilbert cellular sheaf $\mathcal{M}$5 is constructed, assigning to each vertex/edge a Hilbert space and to each incidence a restriction map (typically a weighted/transported copy according to the bundle's parallel transport). This induces a sheaf Laplacian $\mathcal{M}$6, which block-generalizes the classical graph Laplacian. Discretized HilbNets—called $\mathcal{M}$7-HilbNets—replace $\mathcal{M}$8 with $\mathcal{M}$9.

Theoretical Consistency and Transferability

A principal theoretical contribution is the extension of the Belkin-Niyogi convergence result [BELKIN20081289] for graph Laplacians: as spatial ($x\in\mathcal{M}$0) and fiber ($x\in\mathcal{M}$1) discretizations are refined, the discrete sheaf Laplacian converges in mean-squared error to the underlying continuous connection Laplacian in probability. Corollaries establish convergence of HilbNet outputs and transferability across different samplings of $x\in\mathcal{M}$2. Thus, discretized architectures enjoy strong resolution-consistency and robustness guarantees, and Laplacian-based methods such as spectral clustering and embedding are generalizable to settings with infinite-dimensional features.

Figure 2: Spectral stability of $x\in\mathcal{M}$3; low-frequency eigenvalues stabilize rapidly as sampling density increases, confirming operator-level convergence.

Practical Architectures and Numerical Results

The theoretical framework supports a broad space of architectures through the choice of connection (parallel transport), filter class, and discretization. Parallel transport can be instantiated:

• via analytical domain knowledge (physics, statistics, geometry),
• learned from data,
• or regularized through structural constraints (e.g., circulant, orthogonal, or identity).

Empirical results validate the theory in both synthetic and real scenarios:

• Synthetic bundle transport recovery: On a statistical bundle (space of symmetric positive-definite matrices with the Otto-Wasserstein metric), free orthogonal transport parameterizations recover ground-truth parallel transports to nearly numerical precision ($x\in\mathcal{M}$4), while more restricted parametrizations (circulant, identity) align quantitatively with theoretical projection bounds.
• Traffic forecasting: On METR-LA and PEMS-BAY, HilbNet variants with learned non-trivial (free or circulant) transport matrices outperform both fiber-only and standard GCN (frozen-identity transport) baselines on all forecasting metrics, even when controlling for parameter count. The circulant class, encoding temporal alignment priors, remains highly competitive but uses an order-of-magnitude fewer parameters.

Broader Implications and Theoretical Extensions

The introduction of HilbNets and the underlying consistency theory has several theoretical and practical consequences:

• Generalized Laplacian-based methods: Clustering, embedding, and other spectral methods that rely on Laplacian convergence are theoretically grounded for infinite-dimensional signal settings via the established limit theorems.
• Structured self-supervised learning: The sheaf structure suggests new invariance- and equivariance-based SSL objectives that enforce local consistency and geometric coherence across fibers.
• Transformers and attention mechanisms: The framework provides a precise geometric and spectral lens to interpret positional encodings and attention via the lens of Hilbert bundles and data-driven connection Laplacians.
• Universality and unification: Most geometric DL models, including GCNs, equivariant CNNs, spatiotemporal GNNs, and recent message-passing sheaf NNs, are seen as HilbNets for special choices of bundles and connections.

Performance and scalability remain lines of future inquiry, particularly for very large graphs and high-dimensional signals, motivating further study into sparsified or low-rank approximations.

Conclusion

This work advances the mathematical and algorithmic foundations of geometric deep learning by lifting convolutional architectures to the setting of infinite-dimensional signal bundles, providing both theoretical consistency and practical implementability through cellular sheaf theory. The framework unifies diverse geometric architectures and enables principled generalization of Laplacian-based methods. The probabilistic convergence, transferability, and robustness results provide a strong theoretical foundation for further research on geometric learning in complex and high-dimensional domains, as well as inspire new, domain-specific model classes exploiting bundle-theoretic structure.

---

References:

[Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves, (2605.06395)] [BELKIN20081289: Towards a theoretical foundation for Laplacian-based manifold methods]