---
title: Hyperbolic State-Space Models
url: https://www.emergentmind.com/topics/hyperbolic-state-space-models
type: topic
---

# Hyperbolic State-Space Models

Searching arXiv for the cited papers and closely related work on hyperbolic state-space models.
arXiv search query: 2509.05757 Hyperbolic Large Language Models
Hyperbolic state-space models are state-space systems in which the term *hyperbolic* enters in two technically distinct ways. In one line of work, latent states evolve in a negatively curved manifold such as the Poincaré ball or the Lorentz model, so that linear-time sequence modeling can be coupled to hierarchy-preserving representation learning, multi-scale structure, and entailment-sensitive geometry [2509.05757]. In another line of work, *hyperbolic* refers to the generalised hyperbolic (GH) family of probability laws: the latent dynamics of a continuous-time state-space model are then driven by a GH Lévy process, yielding non-Gaussian, heavy-tailed, and jump-sensitive filtering models rather than manifold-valued neural sequence models [2309.11422].

## 1. Two meanings of “hyperbolic” in state-space modeling

In recent arXiv literature, the geometric meaning of hyperbolicity is associated with negatively curved latent spaces for sequence models. The motivation is that language and many structured sequences are inherently hierarchical rather than flat, with examples including syntactic trees in language, ontological relations, multi-hop semantic reasoning, recommendation sequences with latent taxonomy, and audio-visual or biological sequences with branching structure. Standard Euclidean SSMs are strong at long-range dependencies and linear-time sequence processing, but they do not naturally encode tree-like expansion, multi-scale hierarchy, or entailment structure [2509.05757].

The probabilistic meaning of hyperbolicity is different. In GH state-space models, the latent process is a continuous-time linear SDE driven by a generalised hyperbolic Lévy process rather than Brownian motion. This produces a flexible non-Gaussian framework capable of skewness, heavy tails, and impulsive jumps, with applications to filtering problems in dynamic systems such as Langevin dynamics and intraday financial series [2309.11422].

A recurrent source of confusion is that these two families share the word *hyperbolic* but not the same mathematical object. The first family is about representation geometry and manifold-constrained state evolution; the second is about the choice of non-Gaussian driving noise.

## 2. Geometric hyperbolic state spaces: manifolds, hierarchy, and recurrence

The geometric family is built on the claim that negative curvature is well matched to tree growth and hierarchy because hyperbolic space expands exponentially with radius. A central geometric intuition is that points near the origin encode coarse, abstract, or high-level concepts, whereas points near the boundary encode fine-grained, specific, or leaf-level concepts. This radial semantics makes hyperbolic state-space models especially suitable for tasks requiring both long context and hierarchical reasoning [2509.05757].

Two manifolds recur in this literature. The **Poincaré ball** is bounded and conformal, and the **Lorentz model** is a hyperboloid in Minkowski space that is often more numerically stable. In the Lorentz formulation, the inner product is
$$
\langle x,y\rangle_L = -x_0y_0 + \sum_{i=1}^{n} x_i y_i,
$$
with distance
$$
d_{\mathcal{L}}(x,y) = \mathrm{arcosh}\!\big(-\langle x,y\rangle_L\big).
$$
When Euclidean and hyperbolic computation are mixed, exponential and logarithmic maps at the origin are used to move between tangent-space and manifold representations [2509.05757].

Hyperbolic SSMs differ from Euclidean SSMs not only in where states live but also in how recurrence is implemented. Euclidean SSMs propagate state in flat latent space. Hyperbolic SSMs must ensure that updates stay on-manifold, using exponential and logarithmic maps, curvature-aware discretization, parallel transport, and Riemannian optimization. The hyperbolic contribution is therefore not merely representational; it geometrically constrains state evolution itself [2509.05757].

## 3. Representative neural architectures

The recent literature organizes hyperbolic SSMs as a distinct category within hyperbolic large language models and related sequence architectures, combining linear-time state-space dynamics from Mamba/SSMs with hyperbolic latent geometry for hierarchy [2509.05757].

| Model | Task domain | Defining mechanism |
|---|---|---|
| HiM / Hierarchical Mamba | Mixed-hop prediction / multi-hop inference | Fully hyperbolic Mamba2 selective SSM with hyperbolic projections and hierarchy-aware losses |
| HMamba-Full / HMamba-Half | Sequential recommendation | Curvature-normalized discretization; full vs partial hyperbolic processing |
| SHMamba | Audio-visual question answering | Adaptive curvature hyperbolic alignment with Mamba-style structured SSM |

HiM is presented as a fully hyperbolic state-space LLM. Sequence representations are projected into either a Poincaré or Lorentz manifold, and the model learns a curvature parameter \(\mathcal{K}\) together with a norm-scaling factor \(\gamma\). Its training objective includes a **centripetal loss**, which enforces that parents should lie closer to the origin than children, and a **clustering loss**, which pulls related nodes together and pushes unrelated nodes apart. The model is evaluated on WordNet, FoodOn, DOID, and SNOMED for mixed-hop prediction or multi-hop inference. In the reported table, **HiM-Poincaré** reaches **WordNet: 85.9** and **DOID: 90.2**, outperforming a Euclidean Mamba baseline and improving over the hierarchical Transformer comparator [2509.05757].

HMamba adapts hyperbolic geometry to sequential recommendation. The fully hyperbolic variant applies a curvature-normalized discretization
$$
\bar A = \exp(\Delta A \odot K(\mathcal{K})), \qquad K(\mathcal{K})=\mathrm{diag}(\sqrt{|\mathcal{K}|},1,\dots,1),
$$
with learnable \(\mathcal{K}<0\), and uses Lorentz parallel transport for hidden-state evolution. The partially hyperbolic **HMamba-Half** keeps part of the sequence processing and scoring in flat space. The comparison between the two variants illustrates a direct trade-off: full hyperbolicity offers better hierarchy preservation and greater expressiveness, whereas the half-hyperbolic version is more efficient and easier to deploy. Reported results indicate improvements in **top-K recommendation accuracy by about 3–11%** over Transformer-based methods while retaining **\(\mathcal{O}(L)\)** efficiency for long interaction streams [2509.05757].

SHMamba targets AVQA by combining a structured state-space backbone with hyperbolic multimodal alignment. Audio and visual features are projected into hyperbolic space using an adaptive curvature module
$$
k = k_0 \cdot \text{sigmoid}(\text{MLP}(K_{av})),
$$
and temporal dynamics are modeled with Mamba-style SSM blocks. The system uses pretrained **VGGish** for audio and pretrained **CLIP** for visual and question features, three linear encoders, a hyperbolic alignment module, a cross fusion block, and a classifier. On MUSIC-AVQA, SHMamba reaches **74.12**, compared with **71.52** for ST-AVQA; on AVQA, it reaches **90.8\% total accuracy**. Relative to ST-AVQA, the model uses **4.033M parameters** rather than **18.480M**, and **1.086 GFLOPs** rather than **3.188 GFLOPs**, corresponding to **78.12% fewer parameters** and **65.93% fewer FLOPs** [2406.09833].

A frequent misconception is that such systems are merely Euclidean SSMs with a hyperbolic output layer. The discussion of HiM explicitly rejects that characterization: hyperbolic structure is injected into the sequence modeling process and the training objective, not only into the final representation layer [2509.05757].

## 4. Hyperbolic state spaces in state-embedding augmentation

A related but distinct use of hyperbolic state spaces appears in "Learning Fine-grained Domain Generalization via Hyperbolic State Space Hallucination" [2504.08020]. This method is built on **Vision Mamba / VMamba**, a selective SSM backbone, but the principal intervention is not a fully hyperbolic recurrent dynamics. Instead, it manipulates internal state embeddings and constrains them in hyperbolic space.

The method, **Hyperbolic State Space Hallucination (HSSH)**, has two components: **State Space Hallucination (SSH)** and **Hyperbolic Manifold Consistency (HMC)**. SSH estimates channel-wise style statistics of state embeddings, extrapolates beyond the observed style range, and forms hallucinated features by an AdaIN-like transformation. HMC then projects both the original and hallucinated state embeddings into a Poincaré ball
$$
\mathcal{B}_c^n = \{x \in \mathbb{R}^n : c\|x\|^2 < 1\},
$$
with default curvature parameter \(c=0.1\), and minimizes hyperbolic distance between pre- and post-hallucination embeddings while also drawing together samples from the same coarse-grained category [2504.08020].

The optimization objective combines classification on original and hallucinated features with a hyperbolic consistency penalty,
$$
\mathcal{L} = \mathcal{L}_{cls} + \widetilde{\mathcal{L}_{cls}} + \lambda \mathcal{L}_{HMC},
$$
with \(\lambda = 0.5\). The method is evaluated on **CUB-Paintings**, **RS-FGDG**, and **Birds-31** using **top-1 accuracy** on unseen target domains. On CUB-Paintings, the average accuracy increases from **63.47** for VMamba to **66.03** for HSSH; on RS-FGDG, from **66.85** to **69.65**; on Birds-31, from **88.24** to **90.69**. The ablation on CUB-Paintings shows **VMamba baseline: 63.47 avg**, **+SSH: 64.86 avg**, and **+SSH + HMC: 66.03 avg** [2504.08020].

This line of work suggests a broader interpretation of hyperbolic state spaces in neural systems: the manifold need not parameterize the recurrent transition itself in order to shape how SSM-derived internal states preserve hierarchical and high-order statistics.

## 5. Generalised hyperbolic Lévy state-space models

The GH family uses *hyperbolic* in the distributional sense. A GH state-space model is a continuous-time linear SDE state-space system
$$
d \mathbf{x}(t) = \mathbf{A}\mathbf{x}(t)\,dt + \mathbf{L}\,dW(t),
$$
with observations
$$
\mathbf{y}(t) = \mathbf{H}\mathbf{x}(t) + \varepsilon(t),
$$
where \(W(t)\) is a GH Lévy process and \(\varepsilon(t)\) is continuous-time white Gaussian measurement noise with covariance \(\sigma_\varepsilon^2\) [2309.11422].

The GH distribution is a five-parameter family that is infinitely divisible and therefore defines a Lévy process. Its special cases include the **normal inverse-Gaussian (NIG)**, **variance-gamma (VG)**, and **Student-\(t\)** processes. A central construction is the Gaussian variance-mean mixture representation: a GH increment can be written as
$$
W = \mu_W Z + \sigma_W \sqrt{Z}\,U,
$$
with \(Z\) a subordinator jump size and \(U\sim\mathcal N(0,1)\). This conditional Gaussian structure is the foundation of both the simulation and inference procedures [2309.11422].

Because the GH Lévy process has **infinite activity**, exact simulation is intractable. The proposed approach uses a generalised shot-noise series representation, **adaptive truncation**, and **residual approximation**. GIG jumps are simulated via tractable dominating processes and thinning, using either a **tempered stable** dominating process or a **gamma** dominating process. The omitted small-jump residual is approximated by a Gaussian term with matching moments [2309.11422].

The SDE solution is
$$
\mathbf{x}(t)=e^{\mathbf{A}t}\mathbf{x}(0)+\int_0^t e^{\mathbf{A}(t-u)}\mathbf{L}\,dW(u).
$$
Conditional on the latent jumps, the stochastic integral is Gaussian, and the state transition over \((s,t]\) becomes
$$
p(\mathbf{x}_t\mid \mathbf{x}_s,\{(V_i,Z_i)\})= \mathcal N\!\left(e^{\mathbf{A}(t-s)}\mathbf{x}_s+\mathbf{m},\,\mathbf{S}\right).
$$
This enables **conditional Kalman filtering** inside a **sequential Metropolis–Hastings MCMC** scheme. After sampling jump sequences and running Kalman recursions conditionally, the filtering distribution is approximated by a Gaussian mixture and then collapsed back to a single Gaussian by moment matching [2309.11422].

The paper studies a Langevin dynamics example with
$$
d\dot{x}(t)=\theta \dot{x}(t)\,dt + dW(t), \qquad \theta<0,
$$
and applies the method to intraday **GBP/JPY** data. On synthetic Langevin data, the method accurately estimates both position and latent velocity and recovers a large jump in velocity around \(t\approx 78\). On the financial series, the inferred latent velocity exhibits rapid shifts, especially near \(t\approx 4200\), and the estimated price state tracks the observed price closely [2309.11422].

## 6. Comparative properties, limitations, and open problems

The geometric family of hyperbolic SSMs is motivated by hierarchy, long-range dependencies, and multi-scale structure. Compared with Euclidean SSMs, these models better preserve tree structure, multi-hop relations, fine/coarse semantic scale, and long-range hierarchical dependencies. At the same time, they introduce additional complexity: more delicate optimization, numerical stability issues near boundaries or at large radii, overhead from hyperbolic operations, and the need to tune fixed or learnable curvature and scaling parameters [2509.05757].

The trade-off between **full hyperbolic** and **partially hyperbolic** designs is explicit in HMamba. Full hyperbolicity offers better hierarchy preservation and greater expressiveness, but hybrid versions are more efficient and easier to stabilize. The SHMamba results likewise show that hyperbolic alignment and SSM recurrence can be combined with substantially lower parameter count and FLOPs than Transformer baselines, but the paper still treats adaptive curvature, tangent-space mapping, and multimodal fusion as essential engineering components rather than incidental details [2406.09833].

The HSSH results indicate that hyperbolic state spaces can also act as a consistency geometry for SSM-derived state embeddings under style perturbation, especially in fine-grained domain generalization. A plausible implication is that hyperbolic structure can be useful even when the recurrent core itself is not fully manifold-valued, provided that internal states are organized to preserve hierarchical and high-order statistics [2504.08020].

The GH family has a different profile of strengths and weaknesses. Its advantages are continuous-time modeling under irregular sampling, explicit handling of non-Gaussian, heavy-tailed, and asymmetric dynamics, and the ability to embed exact Kalman-style updates inside MCMC through conditional Gaussianity. Its limitations are equally clear: GH/GIG jump simulation remains approximate because of infinite activity, inference uses prior-proposal MCMC rather than a fully optimal jump proposal, and parameter learning is not fully solved, with experiments relying on grid search [2309.11422].

Open directions in the geometric literature include **learnable and mixture-of-curvature SSMs**, **more stable manifold discretization**, **better Riemannian optimizers**, **unified benchmarks for hierarchy-aware sequence modeling**, **extensions to additional modalities and dynamic graphs**, and **more efficient fully hyperbolic implementations** [2509.05757]. Across both meanings of the term, the central issue is not whether state-space modeling is useful, but what kind of hyperbolicity is being exploited: negative curvature for structured representation and sequence reasoning, or GH Lévy structure for non-Gaussian stochastic dynamics.

Source: https://www.emergentmind.com/topics/hyperbolic-state-space-models