Hyperbolic State-Space Models

• Hyperbolic state-space models utilize negatively curved manifolds or GH Lévy processes to enhance sequence modeling through hierarchical and heavy-tail dynamics.
• Geometric models capture tree-like structures by mapping sequence states onto the Poincaré ball or Lorentz space, preserving multi-scale hierarchy and entailment.
• GH-based models employ generalized hyperbolic Lévy processes, offering flexible non-Gaussian dynamics with heavy tails and jump-sensitive filtering for complex systems.

Searching arXiv for the cited papers and closely related work on hyperbolic state-space models. arXiv search query: (Patil et al., 6 Sep 2025) Hyperbolic LLMs Hyperbolic state-space models are state-space systems in which the term hyperbolic enters in two technically distinct ways. In one line of work, latent states evolve in a negatively curved manifold such as the Poincaré ball or the Lorentz model, so that linear-time sequence modeling can be coupled to hierarchy-preserving representation learning, multi-scale structure, and entailment-sensitive geometry (Patil et al., 6 Sep 2025). In another line of work, hyperbolic refers to the generalised hyperbolic (GH) family of probability laws: the latent dynamics of a continuous-time state-space model are then driven by a GH Lévy process, yielding non-Gaussian, heavy-tailed, and jump-sensitive filtering models rather than manifold-valued neural sequence models (Kındap et al., 2023).

1. Two meanings of “hyperbolic” in state-space modeling

In recent arXiv literature, the geometric meaning of hyperbolicity is associated with negatively curved latent spaces for sequence models. The motivation is that language and many structured sequences are inherently hierarchical rather than flat, with examples including syntactic trees in language, ontological relations, multi-hop semantic reasoning, recommendation sequences with latent taxonomy, and audio-visual or biological sequences with branching structure. Standard Euclidean SSMs are strong at long-range dependencies and linear-time sequence processing, but they do not naturally encode tree-like expansion, multi-scale hierarchy, or entailment structure (Patil et al., 6 Sep 2025).

The probabilistic meaning of hyperbolicity is different. In GH state-space models, the latent process is a continuous-time linear SDE driven by a generalised hyperbolic Lévy process rather than Brownian motion. This produces a flexible non-Gaussian framework capable of skewness, heavy tails, and impulsive jumps, with applications to filtering problems in dynamic systems such as Langevin dynamics and intraday financial series (Kındap et al., 2023).

A recurrent source of confusion is that these two families share the word hyperbolic but not the same mathematical object. The first family is about representation geometry and manifold-constrained state evolution; the second is about the choice of non-Gaussian driving noise.

2. Geometric hyperbolic state spaces: manifolds, hierarchy, and recurrence

The geometric family is built on the claim that negative curvature is well matched to tree growth and hierarchy because hyperbolic space expands exponentially with radius. A central geometric intuition is that points near the origin encode coarse, abstract, or high-level concepts, whereas points near the boundary encode fine-grained, specific, or leaf-level concepts. This radial semantics makes hyperbolic state-space models especially suitable for tasks requiring both long context and hierarchical reasoning (Patil et al., 6 Sep 2025).

Two manifolds recur in this literature. The Poincaré ball is bounded and conformal, and the Lorentz model is a hyperboloid in Minkowski space that is often more numerically stable. In the Lorentz formulation, the inner product is

$$\langle x,y\rangle_L = -x_0y_0 + \sum_{i=1}^{n} x_i y_i,$$

with distance

$$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$

When Euclidean and hyperbolic computation are mixed, exponential and logarithmic maps at the origin are used to move between tangent-space and manifold representations (Patil et al., 6 Sep 2025).

Hyperbolic SSMs differ from Euclidean SSMs not only in where states live but also in how recurrence is implemented. Euclidean SSMs propagate state in flat latent space. Hyperbolic SSMs must ensure that updates stay on-manifold, using exponential and logarithmic maps, curvature-aware discretization, parallel transport, and Riemannian optimization. The hyperbolic contribution is therefore not merely representational; it geometrically constrains state evolution itself (Patil et al., 6 Sep 2025).

3. Representative neural architectures

The recent literature organizes hyperbolic SSMs as a distinct category within hyperbolic LLMs and related sequence architectures, combining linear-time state-space dynamics from Mamba/SSMs with hyperbolic latent geometry for hierarchy (Patil et al., 6 Sep 2025).

Model	Task domain	Defining mechanism
HiM / Hierarchical Mamba	Mixed-hop prediction / multi-hop inference	Fully hyperbolic Mamba2 selective SSM with hyperbolic projections and hierarchy-aware losses
HMamba-Full / HMamba-Half	Sequential recommendation	Curvature-normalized discretization; full vs partial hyperbolic processing
SHMamba	Audio-visual question answering	Adaptive curvature hyperbolic alignment with Mamba-style structured SSM

HiM is presented as a fully hyperbolic state-space LLM. Sequence representations are projected into either a Poincaré or Lorentz manifold, and the model learns a curvature parameter $\mathcal{K}$ together with a norm-scaling factor $\gamma$. Its training objective includes a centripetal loss, which enforces that parents should lie closer to the origin than children, and a clustering loss, which pulls related nodes together and pushes unrelated nodes apart. The model is evaluated on WordNet, FoodOn, DOID, and SNOMED for mixed-hop prediction or multi-hop inference. In the reported table, HiM-Poincaré reaches WordNet: 85.9 and DOID: 90.2, outperforming a Euclidean Mamba baseline and improving over the hierarchical Transformer comparator (Patil et al., 6 Sep 2025).

HMamba adapts hyperbolic geometry to sequential recommendation. The fully hyperbolic variant applies a curvature-normalized discretization

$$\bar A = \exp(\Delta A \odot K(\mathcal{K})), \qquad K(\mathcal{K})=\mathrm{diag}(\sqrt{|\mathcal{K}|},1,\dots,1),$$

with learnable $\mathcal{K}<0$, and uses Lorentz parallel transport for hidden-state evolution. The partially hyperbolic HMamba-Half keeps part of the sequence processing and scoring in flat space. The comparison between the two variants illustrates a direct trade-off: full hyperbolicity offers better hierarchy preservation and greater expressiveness, whereas the half-hyperbolic version is more efficient and easier to deploy. Reported results indicate improvements in top-K recommendation accuracy by about 3–11% over Transformer-based methods while retaining $\mathcal{O}(L)$ efficiency for long interaction streams (Patil et al., 6 Sep 2025).

SHMamba targets AVQA by combining a structured state-space backbone with hyperbolic multimodal alignment. Audio and visual features are projected into hyperbolic space using an adaptive curvature module

$k = k_0 \cdot \text{sigmoid}(\text{MLP}(K_{av})),$

and temporal dynamics are modeled with Mamba-style SSM blocks. The system uses pretrained VGGish for audio and pretrained CLIP for visual and question features, three linear encoders, a hyperbolic alignment module, a cross fusion block, and a classifier. On MUSIC-AVQA, SHMamba reaches 74.12, compared with 71.52 for ST-AVQA; on AVQA, it reaches 90.8\% total accuracy. Relative to ST-AVQA, the model uses 4.033M parameters rather than 18.480M, and 1.086 GFLOPs rather than 3.188 GFLOPs, corresponding to 78.12% fewer parameters and 65.93% fewer FLOPs (Yang et al., 2024).

A frequent misconception is that such systems are merely Euclidean SSMs with a hyperbolic output layer. The discussion of HiM explicitly rejects that characterization: hyperbolic structure is injected into the sequence modeling process and the training objective, not only into the final representation layer (Patil et al., 6 Sep 2025).

4. Hyperbolic state spaces in state-embedding augmentation

A related but distinct use of hyperbolic state spaces appears in "Learning Fine-grained Domain Generalization via Hyperbolic State Space Hallucination" (Bi et al., 10 Apr 2025). This method is built on Vision Mamba / VMamba, a selective SSM backbone, but the principal intervention is not a fully hyperbolic recurrent dynamics. Instead, it manipulates internal state embeddings and constrains them in hyperbolic space.

The method, Hyperbolic State Space Hallucination (HSSH), has two components: State Space Hallucination (SSH) and Hyperbolic Manifold Consistency (HMC). SSH estimates channel-wise style statistics of state embeddings, extrapolates beyond the observed style range, and forms hallucinated features by an AdaIN-like transformation. HMC then projects both the original and hallucinated state embeddings into a Poincaré ball

$$\mathcal{B}_c^n = \{x \in \mathbb{R}^n : c\|x\|^2 < 1\},$$

with default curvature parameter $c=0.1$, and minimizes hyperbolic distance between pre- and post-hallucination embeddings while also drawing together samples from the same coarse-grained category (Bi et al., 10 Apr 2025).

The optimization objective combines classification on original and hallucinated features with a hyperbolic consistency penalty,

$$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$0

with $$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$1. The method is evaluated on CUB-Paintings, RS-FGDG, and Birds-31 using top-1 accuracy on unseen target domains. On CUB-Paintings, the average accuracy increases from 63.47 for VMamba to 66.03 for HSSH; on RS-FGDG, from 66.85 to 69.65; on Birds-31, from 88.24 to 90.69. The ablation on CUB-Paintings shows VMamba baseline: 63.47 avg, +SSH: 64.86 avg, and +SSH + HMC: 66.03 avg (Bi et al., 10 Apr 2025).

This line of work suggests a broader interpretation of hyperbolic state spaces in neural systems: the manifold need not parameterize the recurrent transition itself in order to shape how SSM-derived internal states preserve hierarchical and high-order statistics.

5. Generalised hyperbolic Lévy state-space models

The GH family uses hyperbolic in the distributional sense. A GH state-space model is a continuous-time linear SDE state-space system

$$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$2

with observations

$$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$3

where $$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$4 is a GH Lévy process and $$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$5 is continuous-time white Gaussian measurement noise with covariance $$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$6 (Kındap et al., 2023).

The GH distribution is a five-parameter family that is infinitely divisible and therefore defines a Lévy process. Its special cases include the normal inverse-Gaussian (NIG), variance-gamma (VG), and Student-$$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$7 processes. A central construction is the Gaussian variance-mean mixture representation: a GH increment can be written as

$$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$8

with $$d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).$$9 a subordinator jump size and $\mathcal{K}$0. This conditional Gaussian structure is the foundation of both the simulation and inference procedures (Kındap et al., 2023).

Because the GH Lévy process has infinite activity, exact simulation is intractable. The proposed approach uses a generalised shot-noise series representation, adaptive truncation, and residual approximation. GIG jumps are simulated via tractable dominating processes and thinning, using either a tempered stable dominating process or a gamma dominating process. The omitted small-jump residual is approximated by a Gaussian term with matching moments (Kındap et al., 2023).

The SDE solution is

$\mathcal{K}$1

Conditional on the latent jumps, the stochastic integral is Gaussian, and the state transition over $\mathcal{K}$2 becomes

$\mathcal{K}$3

This enables conditional Kalman filtering inside a sequential Metropolis–Hastings MCMC scheme. After sampling jump sequences and running Kalman recursions conditionally, the filtering distribution is approximated by a Gaussian mixture and then collapsed back to a single Gaussian by moment matching (Kındap et al., 2023).

The paper studies a Langevin dynamics example with

$\mathcal{K}$4

and applies the method to intraday GBP/JPY data. On synthetic Langevin data, the method accurately estimates both position and latent velocity and recovers a large jump in velocity around $\mathcal{K}$5. On the financial series, the inferred latent velocity exhibits rapid shifts, especially near $\mathcal{K}$6, and the estimated price state tracks the observed price closely (Kındap et al., 2023).

6. Comparative properties, limitations, and open problems

The geometric family of hyperbolic SSMs is motivated by hierarchy, long-range dependencies, and multi-scale structure. Compared with Euclidean SSMs, these models better preserve tree structure, multi-hop relations, fine/coarse semantic scale, and long-range hierarchical dependencies. At the same time, they introduce additional complexity: more delicate optimization, numerical stability issues near boundaries or at large radii, overhead from hyperbolic operations, and the need to tune fixed or learnable curvature and scaling parameters (Patil et al., 6 Sep 2025).

The trade-off between full hyperbolic and partially hyperbolic designs is explicit in HMamba. Full hyperbolicity offers better hierarchy preservation and greater expressiveness, but hybrid versions are more efficient and easier to stabilize. The SHMamba results likewise show that hyperbolic alignment and SSM recurrence can be combined with substantially lower parameter count and FLOPs than Transformer baselines, but the paper still treats adaptive curvature, tangent-space mapping, and multimodal fusion as essential engineering components rather than incidental details (Yang et al., 2024).

The HSSH results indicate that hyperbolic state spaces can also act as a consistency geometry for SSM-derived state embeddings under style perturbation, especially in fine-grained domain generalization. A plausible implication is that hyperbolic structure can be useful even when the recurrent core itself is not fully manifold-valued, provided that internal states are organized to preserve hierarchical and high-order statistics (Bi et al., 10 Apr 2025).

The GH family has a different profile of strengths and weaknesses. Its advantages are continuous-time modeling under irregular sampling, explicit handling of non-Gaussian, heavy-tailed, and asymmetric dynamics, and the ability to embed exact Kalman-style updates inside MCMC through conditional Gaussianity. Its limitations are equally clear: GH/GIG jump simulation remains approximate because of infinite activity, inference uses prior-proposal MCMC rather than a fully optimal jump proposal, and parameter learning is not fully solved, with experiments relying on grid search (Kındap et al., 2023).

Open directions in the geometric literature include learnable and mixture-of-curvature SSMs, more stable manifold discretization, better Riemannian optimizers, unified benchmarks for hierarchy-aware sequence modeling, extensions to additional modalities and dynamic graphs, and more efficient fully hyperbolic implementations (Patil et al., 6 Sep 2025). Across both meanings of the term, the central issue is not whether state-space modeling is useful, but what kind of hyperbolicity is being exploited: negative curvature for structured representation and sequence reasoning, or GH Lévy structure for non-Gaussian stochastic dynamics.