Geometry & Topology of Neural Representations

• Neural representations are modeled as high-dimensional manifolds where geometric and topological features capture the transformation of input data.
• Persistent homology and related invariants track the systematic reduction in topological complexity across neural network layers.
• Riemannian metrics and topological measures provide robust tools for comparing representations, aiding model selection and enhancing generalization.

The geometry and topology of neural representations constitute a central theme in contemporary deep learning and systems neuroscience, blending tools from algebraic topology, Riemannian geometry, and dynamical systems theory. This framework interprets layerwise transformations in neural networks—and population activity in the brain—as mappings between high-dimensional manifolds. These mappings systematically “simplify” the input space, collapsing originally complex geometric and topological features into class-separable representations. The emergent representational manifolds encode task variables, support generalization, and reflect neural circuit mechanisms. The use of persistent homology, relative homology, geometric invariants (curvature, dimension), and divergence measures provides a rigorous foundation for quantifying these phenomena, yielding insight into learning dynamics, architectural effects, and the fundamental capabilities of artificial and biological neural circuits.

1. Theoretical Foundations: Manifold Hypothesis, Geometry, and Topology

Neural representations, whether in artificial neural networks (ANNs) or biological circuits, are modeled as point clouds $X \subset \mathbb{R}^N$ whose structure often approximates a low-dimensional manifold $\mathcal{M} \subset \mathbb{R}^N$ (“manifold hypothesis”) (Acosta et al., 2022, Chung et al., 2021). The geometry of such a manifold is determined by the pullback of the ambient Euclidean metric, yielding a Riemannian structure with intrinsic distances, geodesics, and curvature—quantities relevant for perceptual discriminability and decoding (Acosta et al., 2022, Pellegrino et al., 3 Dec 2025).

Topological invariants characterize qualitative, coordinate-free aspects of $\mathcal{M}$: connected components ($\beta_0$), loops ($\beta_1$), voids ($\beta_2$), and higher Betti numbers. Persistent homology computes these invariants across scales by constructing Vietoris–Rips complexes as the neighborhood radius parameter $\epsilon$ increases, yielding barcode or persistence diagrams (Naitzat et al., 2020, Magai, 2023). The rank-decomposition framework further refines this picture by associating topological change in representations to low-rank regions in the piecewise-affine map induced by ReLU networks (Beshkov et al., 2024, Beshkov, 3 Feb 2025). These invariants, together with geometric descriptors (intrinsic dimension, curvature), provide a basis for comparing representations across layers, architectures, and brain regions (Lin et al., 2023, Świder, 2024).

2. Persistent Homology and Topological Simplification in Neural Networks

Across well-trained deep networks, the flow of data through layers yields a “monotonic” reduction in topological complexity: the Betti numbers of the class manifolds decrease, with $\beta_k \rightarrow 0$ for $k \ge 1$ and $\beta_0 \rightarrow 1$ per class at the last hidden layer (Naitzat et al., 2020). This collapse is not merely geometric (e.g., vector “flattening”) but topological, eliminating all cycles (loops, tunnels, higher holes) in the representations. Persistent homology makes these transitions quantitatively and visually explicit, tracking the birth and death of topological features through barcodes (Magai et al., 2022, Magai, 2023).

The mechanism for topology change is strongly activation-dependent. Piecewise-linear, non-invertible nonlinearities such as ReLU effect discontinuous, non-homeomorphic folds, destroying topological features rapidly between layers. In contrast, smooth bijections (e.g., Tanh) preserve topology unless reached only in the presence of floating-point saturation (Naitzat et al., 2020, Shahidullah, 2022). Shallow networks tend to preserve topology until the final layer, enacting one large collapse, whereas deep networks spread topology changes more evenly—facilitating optimization and learning (Naitzat et al., 2020, Lange et al., 2022).

3. Computational and Statistical Tools for Geometry and Topology

Persistent homology algorithms construct Vietoris–Rips complexes (or alternatives) from point clouds of layer activations or weight vectors. The computation of Betti numbers and barcodes employs matrix-reduction, union-find, and linear programming for relative homology (Magai et al., 2022, Świder, 2024, Beshkov et al., 2024). The persistent-homological fractal dimension (PHdim) quantifies the effective dimension of the feature manifold via scaling laws of lifespan sums (Magai, 2023, Magai et al., 2022).

Curvature and Riemannian metrics are extracted via explicit parameterizations (e.g., topological variational autoencoders), with extrinsic mean curvature computed from second fundamental forms and pullback metrics (Acosta et al., 2022). Angular-CKA and shape metrics endow the space of representations with Riemannian structure, enabling the computation of geodesics, angles, and projections between layerwise “states” (Lange et al., 2022).

Relative homology and overlap decompositions afford a rigorous, metric-independent approach to attributing topology change within ReLU networks. Here, the equivalence relation imposed by the network is dissected into low-rank projections and polyhedral overlaps, with Betti numbers computed directly by relative homology of the quotient pair $(M, O_\Phi)$ (Beshkov, 3 Feb 2025, Beshkov et al., 2024).

Comparison and divergence of representations employ TDA-based metrics such as Representation Topology Divergence (RTD), quantifying the difference in multiscale topology between layers, checkpoints, or models; as well as topological representational similarity analysis (tRSA), which interpolates between full geometry (RDMs) and pure topology (Barannikov et al., 2021, Lin et al., 2023).

4. Dynamics Across Network Depth, Architecture, and Task

The flow of topology and geometry is architecture-dependent. Deep convolutional networks exhibit a gradual decline of Betti numbers and PHdim through the hierarchy, with a mid-network peak in feature complexity and terminal collapse (“neural collapse”) (Magai, 2023, Magai et al., 2022). Vision Transformers (ViT) maintain higher topological complexity until late layers, with rapid topological transition at classification heads, while CNN–ViT hybrids interpolate between these behaviors (Świder, 2024, Magai, 2023). Residual networks (ResNets) induce smoother topological transitions layer-wise than VGG-type CNNs (Świder, 2024). Pretraining establishes an architectural “prior” on early-layer topology, with finetuning modifying topology mainly in deeper layers.

Task structure modulates representational topology: discrete classification drives the emergence of low-rank, topology-destroying regions that collapse manifold cycles, whereas regression preserves manifold structure (Beshkov et al., 2024). Training dynamics trace a trajectory through representation space, which can be explicitly described as a geodesic (with orthogonality and “progress” quantified at each step via representational metrics) (Lange et al., 2022).

In recurrent neural networks and dynamical systems, the time-varying representation manifold is an immersed submanifold whose topology mirrors that of the input manifold, and whose curvature and metric structure evolve to amplify or suppress task-relevant directions (Pellegrino et al., 3 Dec 2025). This produces task-specific “warping” of the population geometry during behavioral computation.

5. Neural Manifold Geometry and Curvature

The explicit parameterization of neural manifolds enables precise estimation of extrinsic and intrinsic geometric properties. Given a decoder $f: \mathcal{Z} \rightarrow \mathbb{R}^N$ parameterizing the manifold, Riemannian quantities—pullback metric $g_{\alpha\beta}$, Christoffel symbols, second fundamental form, mean curvature—are computed by automatic differentiation (Acosta et al., 2022). These quantities are invariant to reparameterization and neuron permutation.

Empirically, neural and artificial representations estimated from synthetic and biological data (e.g., hippocampal place-cell manifolds) recover expected topological types (circle, sphere, torus), with curvature profiles corresponding to regions of sharp transition in activity spaces (Acosta et al., 2022). Quantification of curvature informs interpretations of cognitive map warping, perceptual distortions, and the physical realizability of readout axes for downstream computation (Chung et al., 2021).

6. Comparative and Applied Perspectives: Model Selection, Generalization, and Robustness

Topological and geometric summaries provide robust, invariant criteria for comparing neural representations—layer-to-layer, model-to-model, and model-to-brain (Barannikov et al., 2021, Lin, 2022, Lin et al., 2023). Statistics such as the lifespan sum of 0-homology, PHdim at the final layer, RTD, and generalization-sensitive tRSA transforms show strong correlation with out-of-sample test error, adversarial robustness, and transferability (Magai et al., 2022, Magai, 2023, Barannikov et al., 2021, Lin et al., 2023). TDA pipelines are stable under sampling variation and robust to outlier removal, especially when sample sizes are matched (Świder, 2024).

The geometry and topology of filters themselves (not just activations) reveal that convolutional layers learn simple, low-dimensional templates—e.g., edge detector circles—whose persistence (H₁-barcode lifespan) predicts generalization (Gabrielsson et al., 2018, Carlsson et al., 2018). Architecture-aware design—initializing, regularizing, or augmenting layers according to topological templates—can accelerate training or improve cross-domain transfer (Carlsson et al., 2018).

In the context of brain-inspired architectures, the introduction of cohomological flows, spectral Laplacians, and sheaf cohomology yields models with higher manifold consistency and noise stability than standard GNNs or manifold autoencoders, establishing theoretical and empirical advantages for representations that encode persistent topological invariants (Girish et al., 9 Dec 2025).

7. Outlook and Synthesis

Geometry and topology provide an integrated, quantitative framework for the characterization, comparison, and interpretation of neural representations in artificial and biological systems. The universality of phenomena such as topological simplification, curvature-driven readout, and dynamical warping suggests a deep link between architectural design, learning dynamics, and the ultimate computational capabilities of neural networks (Naitzat et al., 2020, Chung et al., 2021, Magai et al., 2022, Pellegrino et al., 3 Dec 2025). Emerging tools that combine TDA, Riemannian geometry, and relative homology stand to deepen the mechanistic understanding and controllability of high-dimensional representations, offering principled metrics for generalization, robustness, and architectural innovation.