Geometric State-Space Neural Networks

• Geometric State-Space Neural Networks are architectures where hidden representations and dynamics evolve on non-Euclidean manifolds, ensuring invariance and interpretability.
• They integrate Riemannian, symplectic, and combinatorial geometric principles to craft learning rules that respect physical invariants and optimize state transitions.
• These models enhance sample efficiency, preserve energy and volume, and unify methods from information geometry and topological analysis for robust real-world performance.

A geometric state-space neural network (GSSNN) is any neural architecture in which both the hidden or latent representations (states) and the dynamics that evolve them are intrinsically constrained by non-Euclidean geometric, symplectic, or topological structure. Rather than living in an unconstrained vector space, the state evolves on a curved manifold, respects group actions, or obeys invariants (such as symplecticity, volume, positivity, or convexity) stemming from the geometry of the data or the scientific problem. This unifies developments from information geometry, Riemannian dynamics, geometric control, and topological machine learning. Recent work provides concrete instantiations of this paradigm across statistical manifolds, SPD manifolds, symplectic phase space, and activation spaces controlled by convex or combinatorial geometry.

1. Foundational Principles of Geometric State-Space Neural Networks

The geometric state-space neural network paradigm rests on encoding geometric structure both in the state-space (hidden representation) and the evolution/inference rules. The state-space may be:

• A smooth statistical manifold, e.g. the lognormal family, with Fisher information metric and Amari gradient structure (Assandje et al., 30 Sep 2025).
• A Riemannian manifold, notably the symmetric positive definite (SPD) cone, with affine-invariant or Stein distances, supporting group actions and Fréchet means (Dan et al., 20 Jan 2026, Conti et al., 19 Feb 2025).
• A symplectic manifold, i.e., phase space with canonical symplectic form $\omega=dq\wedge dp$, respecting Hamiltonian dynamics (Aboussalah et al., 21 Jul 2025, Celledoni et al., 2022).
• A piecewise linear combinatorial manifold, e.g. the Hamming cube of binary activation vectors under ReLU folding (Lewandowski et al., 14 Feb 2025).
• A geometric constraint enforced by convexity, extreme point structure, or separation in activation spaces (Jia et al., 2019).

The dynamics are typically crafted by lifting natural gradient flows, Hamiltonian vector fields, or structure-preserving discretizations to the appropriate manifold. The core principle is that the network's forward map and parameter inference are fully dictated by geometric objects, promoting invariance, stability, and interpretability beyond standard black-box deep learning.

2. Formulation: Hamiltonian, Riemannian, and Symplectic Geometries

Hamiltonian and symplectic GSSNNs: These architectures encode learning as evolution under Hamiltonian dynamics on a phase space endowed with a (possibly learned) symplectic or Riemannian structure. In the case of the lognormal statistical manifold (Assandje et al., 30 Sep 2025), the Amari natural gradient flow

$$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$

is shown to be equivalent to a Hamiltonian system. The inputs are mapped to points on the Poincaré disk, and layer transformations correspond to actions by the Lie group $SU(1,1)$ (rotations), with nonlinearities arising from the symplectic form (e.g., the activation $$f(z(\beta)) = \exp(\tfrac12\cos\beta)\exp(i\tfrac12\sin\beta)$$ is a direct function of geometric features).

Symplectic structure in learning and autoencoding: For example, GeoHNNs (Aboussalah et al., 21 Jul 2025) enforce the symplectic property in latent dynamics by using a bi-orthogonal manifold parameterization of encoder/decoder weights, guaranteeing volume preservation in latent phase space. The inertia matrix $M(q)$ is always symmetric positive definite and learned via the Riemannian exponential map, ensuring the correct Riemannian geometry on state-dependent inertias.

State-space evolution on the SPD manifold: GeoDynamics (Dan et al., 20 Jan 2026) treats trajectories of SPD matrices directly as points on Riemannian manifolds, with state updates and observations realized as Fréchet means (weighted barycenters) composed with group actions. Operators such as the Riemannian exponential, logarithm, and matrix exponential ensure all dynamics remain within the geometric manifold.

3. State-Space Model Architectures with Geometric Constraints

Discrete geometric SSMs for sparse geometric data: STREAM (Schöne et al., 2024) is designed for unordered or sparse geometric sequences (e.g., point clouds, event streams). The model injects coordinate difference $\Delta_k=(t_k-t_{k-1})/\delta$ directly into the state transition kernel, so that the effective linear dynamics are sensitive to geometric structure rather than pure index ordering. This design gives strong inductive bias for geometric data, improves sample complexity, and enables $O(N)$ inference.

ODE-based and structure-preserving discretization: Neural ODE and ResNet architectures can be interpreted as finite-difference dynamical systems, where skip connections implement Euler or higher-order discretizations (Hauser et al., 2018). Imposing further geometric discretizations (e.g., symplectic Euler) leads to exact conservation of invariants such as energy or volume (Celledoni et al., 2022). Layer-wise or block-wise parameter constraints (enforced via proj, exponential maps, or orthogonalization) translate directly into manifold structure on the hidden state evolution.

Combinatorial and convex geometric constraints in activation spaces: In fully connected ReLU nets, activation states can be associated with vertices of a Hamming hypercube. The network implements space-folding, mapping Euclidean segments into intricate walks on the cube, quantifiable by the folding measure $\chi$ (Lewandowski et al., 14 Feb 2025). Geometric structure can also be encoded via convex-hull constraints: inter-class hull separation, all-point extremality, and convex projection layers (Jia et al., 2019). These constraints provide explicit geometric meaning to the state-space at each layer.

4. Riemannian, Information-Geometric, and Topological Learning Mechanisms

Geometric learning mechanisms in GSSNNs involve non-Euclidean optimization and parameterization protocols adapted to the underlying manifold:

• Riemannian gradient descent: For SPD parameters (e.g., inertia matrices in (Aboussalah et al., 21 Jul 2025) or transition matrices in (Conti et al., 19 Feb 2025)), the gradient and update are defined via the manifold's metric; e.g., for affine-invariant geometry,

$$\mathrm{grad}\,\mathcal{J}(X) = X\,(\nabla_{\mathrm{Euc}}\mathcal{J})\,X,$$

with updates via the Riemannian exponential map. Parameters are retracted to the manifold after every update.

• Manifold-aware autoencoders and convolution: Operations such as convolution, attention, and nonlinearity are performed with SPD-preserving (or more generally structure-preserving) parameterizations; e.g., kernel slices $\widehat K = W^\top W + \epsilon I$ to ensure SPD, and geometric attention blocks leverage the matrix exponential to remain on-manifold (Dan et al., 20 Jan 2026).
• Group actions and symplectic integration: In Poincaré disk–based architectures, layer transformations are group actions (e.g., $$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$0 rotations), and the activation function corresponds to a lift respecting the symplectic form (Assandje et al., 30 Sep 2025). In symplectic neural ODEs, integration schemes such as symplectic Euler ensure the states remain evolves symplectically (Celledoni et al., 2022).
• Geometric bottlenecks and sparse manifolds: In reinforcement learning with continuous state/action spaces, the intrinsic dimension of the learned reachable manifold is governed by the action dimension. Sparse, bottleneck, or locally low-rank layers are inserted to match this geometry explicitly, reducing sample and parameter inefficiency (Tiwari et al., 28 Jul 2025).

5. Empirical Properties, Robustness, and Theoretical Guarantees

Preservation of physical and geometric invariants: Geometric state-space neural networks empirically preserve trajectory stability, energy conservation, invariance under group action, and uniqueness of representation. GeoHNN achieves trajectory error and energy drift an order of magnitude lower than classical HNNs on multi-body and high-dimensional mechanical systems, matching physical conservation laws (Aboussalah et al., 21 Jul 2025).

Sample efficiency and inductive bias: Explicit geometric encoding (e.g., via geometric step-size in STREAM (Schöne et al., 2024)) yields strong generalization and sample efficiency. STREAM attains 100% accuracy on DVS128-Gestures and outperforms baseline Mamba models on sparse point-cloud and event datasets.

Structural generalization and overfitting detection: Hull-based classifiers in activation space exhibit smaller train-test accuracy gaps and sharper overfitting indicators than Softmax; hull gap $$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$1 grows with overfitting (Jia et al., 2019). Monitoring folding measure $$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$2 in ReLU nets quantifies expressivity and warns of excessive geometric complexity that may underlie adversarial vulnerabilities (Lewandowski et al., 14 Feb 2025).

Universality: GSSNNs constructed from ODE discretization, symplectic splitting, and Hamiltonian embedding possess universal approximation capabilities for flows of vector fields and compositions of $$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$3 diffeomorphisms (with theorems substantiated in (Celledoni et al., 2022)).

Intrinsic dimension of attainable sets: In RL, the reachable set under deep policy dynamics concentrates around a low-dimensional submanifold; measured intrinsic dimension matches the theoretical bound of $$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$4, much lower than raw state-space dimension, across MuJoCo environments (Tiwari et al., 28 Jul 2025).

6. Open Questions, Design Strategies, and Future Directions

Current limitations and open questions focus on multi-dimensional geometric step-size injection (simultaneous $$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$5, $$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$6, $$\dot\theta = -I(\theta)^{-1}\nabla_\theta\Phi(\theta)$$7 in STREAM (Schöne et al., 2024)), learned sequence ordering, extension to non-linear SSMs, and manifold adaptivity in reinforcement learning (Tiwari et al., 28 Jul 2025). Hybrid models combining geometric SSMs and attention, or integrating continuous-time nonlinearity, remain to be fully explored.

Design recommendations:

• Match architectural bottlenecks, layer widths, or manifold structure to the task’s intrinsic geometry (e.g., input point sparsity, action dimension, known physical invariants).
• Regularize geometric invariants (e.g., volume, hull size, folding measure) to balance expressivity and robustness.
• Use explicit manifold parameterizations and Riemannian optimization in layers where geometric structure is required.
• Monitor geometric quantities (folding, hull-gap, Fréchet mean variance) during training as diagnostic signals.

Outlook: The geometric state-space paradigm leverages advances from modern geometry, mathematical physics, and topological data analysis to provide transparent, robust, and generalizable neural architectures. By directly respecting the geometric backbone of the data and domain, these architectures promise both theoretical tractability and leading empirical performance across physical modeling, brain dynamics analysis, geometric vision, and high-dimensional control. The synthesis of continuous manifold learning, discrete combinatorial geometry, and structure-preserving numerical schemes defines the state-of-the-art toolkit for learning in non-Euclidean environments (Assandje et al., 30 Sep 2025, Aboussalah et al., 21 Jul 2025, Schöne et al., 2024, Dan et al., 20 Jan 2026, Celledoni et al., 2022, Jia et al., 2019, Lewandowski et al., 14 Feb 2025, Tiwari et al., 28 Jul 2025).