---
title: Hyperbolic Embedding Reservoir (HypER)
url: https://www.emergentmind.com/topics/hyperbolic-embedding-reservoir-hyper
type: topic
---

# Hyperbolic Embedding Reservoir (HypER)

Searching arXiv for the cited HypER paper and closely related hyperbolic-geometry work to ground the article.
Hyperbolic Embedding Reservoir (HypER) is an Echo State Network (ESN) architecture in which the reservoir’s latent geometry and recurrent wiring are explicitly matched to the stretch-and-fold structure and exponential divergence of chaotic dynamics. In HypER, reservoir neurons are embedded in the unit Poincaré ball with curvature fixed at $K=-1$, and recurrent couplings decay exponentially with hyperbolic geodesic distance. The design preserves standard ESN mechanisms—sparsity, leaky integration, spectral-radius normalization, and training restricted to a Tikhonov-regularized readout—while introducing a curvature-aware inductive bias intended to align local reservoir expansion and contraction with Lyapunov growth and the stable and unstable directions of chaotic flows [2508.18196].

## 1. Concept and motivation

HypER was introduced to address a specific mismatch between conventional reservoir computing and chaotic dynamics. Chaotic flows exhibit local stretch-and-fold behavior: infinitesimal perturbations separate exponentially along unstable directions and are subsequently folded back toward a fractal attractor. In the formulation used for HypER, if two initial conditions differ by $\|\delta x_0\|$, then their separation grows as $\|\delta x_t\| \approx \|\delta x_0\| e^{\lambda_{\max} t}$ when the largest Lyapunov exponent satisfies $\lambda_{\max} > 0$ [2508.18196].

Classical ESNs typically operate in Euclidean latent spaces with polynomial volume and metric growth. According to the HypER formulation, such reservoirs can reproduce short-term structure but often struggle to sustain forecasts beyond approximately $5$–$8$ Lyapunov times because their latent geometry does not encode an inductive bias for exponential expansion and contraction [2508.18196]. HypER addresses this by using negative curvature, so that exponential metric growth is built directly into the reservoir geometry.

The central intuition is geometric. Hyperbolic spaces have exponential expansion of volume and geodesic distance, so small coordinate changes near the boundary of the Poincaré ball can induce large geodesic increments. HypER exploits this by assigning many reservoir units to near-boundary regions under hyperbolic-uniform sampling, thereby representing strongly expanding directions, while more central units capture contracting modes. This suggests that the reservoir is not merely a random nonlinear dynamical system, but a geometry-shaped surrogate whose local gain structure is intended to mirror Lyapunov behavior [2508.18196].

## 2. Geometric foundation in the Poincaré ball

HypER uses the $d$-dimensional unit Poincaré ball
$$
B^d = \{ x \in \mathbb{R}^d : \|x\| < 1 \},
$$
with constant negative sectional curvature fixed at $K=-1$ [2508.18196]. The conformal Riemannian metric is
$$
ds^2 = \frac{4 \|dx\|^2}{(1 - \|x\|^2)^2}.
$$

The hyperbolic geodesic distance between $x,y \in B^n$ is defined as
$$
d_{\mathbb{H}}(x,y) = \operatorname{arcosh}\!\left( 1 + \frac{2 \|x-y\|^2}{(1-\|x\|^2)(1-\|y\|^2)} \right),
$$
with
$$
\operatorname{arcosh}(\zeta) = \ln\!\left(\zeta + \sqrt{\zeta^2 - 1}\right).
$$
Geodesics are straight lines through the origin or circular arcs orthogonal to the boundary, and angles are preserved because the metric is conformal [2508.18196].

In HypER, curvature is not itself tuned. The construction fixes $K=-1$ and uses the unit ball, while the effective expansion and contraction are controlled primarily by the hyperbolic kernel width $\sigma$ and the sparsity parameter $\kappa$ [2508.18196]. This distinguishes HypER from formulations that treat curvature as a trainable parameter. A plausible implication is that the method emphasizes architectural simplicity and ESN-style fixed-reservoir design over manifold learning.

The use of hyperbolic geometry in HypER is conceptually consistent with broader machine-learning work exploiting exponential volume growth. Hyperbolic embeddings have been used to encode latent structure in networks through the $\mathbb{S}^1/\mathbb{H}^2$ framework, where radial position represents popularity or degree and angular position represents similarity [1904.10814]. More generally, hyperbolic representation learning has been motivated by the ability of negatively curved spaces to embed tree-like and hierarchical structure with low distortion, although this comes with well-documented numerical constraints in both the Poincaré and Lorentz models [2211.00181]. HypER differs in using hyperbolic geometry not for static hierarchy modeling, but for recurrent dynamics and chaotic forecasting [2508.18196].

## 3. Reservoir construction and state dynamics

HypER defines each reservoir neuron $i$ by a coordinate $p_i \in B^d$, written as $p_i = r_i \omega_i$, where $r_i \in [0,1)$ and $\omega_i \in S^{d-1}$ [2508.18196]. Two sampling schemes are specified.

The Euclidean-isotropic baseline samples $r_i$ uniformly in Euclidean volume within a ball of radius $R<1$, using
$$
r_i = R u^{1/d}, \qquad u \sim \mathrm{Uniform}(0,1),
$$
with $\omega_i$ uniform on $S^{d-1}$ [2508.18196]. This baseline ignores the native hyperbolic volume measure.

HypER uses hyperbolic-uniform sampling. The hyperbolic radius $\rho_i \ge 0$ is drawn with density
$$
p(\rho) \propto \sinh^{d-1}(\rho),
$$
and related to the Euclidean radius by
$$
\rho = 2\,\operatorname{artanh}(r), \qquad r_i = \tanh(\rho_i/2).
$$
Using the normalized cumulative distribution
$$
\frac{F(\rho_i)}{F(\rho_{\max})}
=
\frac{\cosh \rho_i - 1}{\cosh \rho_{\max} - 1}
=
u_i,
$$
with $u_i \sim \mathrm{Uniform}(0,1)$, the radius is sampled as
$$
\rho_i = \operatorname{arcosh}\!\left(u_i(\cosh \rho_{\max} - 1) + 1\right),
$$
followed by $r_i = \tanh(\rho_i/2)$ and uniform $\omega_i$ on $S^{d-1}$ [2508.18196]. This sampling places more nodes near the boundary, matching the exponential hyperbolic volume element.

Pairwise hyperbolic distances $d_{ij} = d_{\mathbb{H}}(p_i,p_j)$ determine the recurrent weight matrix through the kernel
$$
W_{ij} = \exp(-d_{ij}/\sigma),
$$
where $\sigma > 0$ is the kernel width [2508.18196]. Small hyperbolic distances yield stronger coupling, whereas near-boundary nodes, which are far apart in hyperbolic distance, have weaker direct coupling.

Sparsity is enforced by retaining only the top-$\kappa$ entries in each row, producing a sparse matrix $\widetilde{W}$. This typically yields a directed, asymmetric graph; symmetric variants are possible but are not used in the reported work [2508.18196]. The sparse matrix is then scaled to a target spectral radius $\varrho \in (0,1)$:
$$
W = \left(\frac{\varrho}{\rho(\widetilde{W})}\right)\widetilde{W},
$$
where $\rho(\cdot)$ denotes the spectral radius. The purpose is to preserve the geometry-induced structure while keeping the internal dynamics near the edge of stability and maintaining the echo-state property (ESP) [2508.18196].

The state update follows the standard leaky ESN form
$$
x_{t+1} = (1-\alpha)x_t + \alpha\,\phi(Wx_t + Uu_t),
$$
where $\alpha \in (0,1]$ is the leak rate, $U \in \mathbb{R}^{N\times m}$ is a random input-weight matrix, and $\phi$ is a $1$-Lipschitz pointwise activation such as $\tanh$ [2508.18196]. Mixed node-wise activations, specifically tanh–sine–linear combinations, are also explored and are reported to help most when combined with hyperbolic connectivity.

The readout uses a second-order polynomial feature map
$$
\xi_t = [x_t^1,\ldots,x_t^N,(x_t^1)^2,\ldots,(x_t^N)^2,1]^\top \in \mathbb{R}^{2N+1},
$$
with prediction
$$
\hat{u}_{t+1} = W_{\mathrm{out}} \xi_t.
$$
Only $W_{\mathrm{out}}$ is trained, preserving the characteristic fixed-reservoir workflow of ESNs [2508.18196].

## 4. Training protocol and theoretical analysis

HypER trains only the readout by Tikhonov-regularized least squares. After washout $T_w$, the method collects feature-target pairs $\{\xi_t, u_{t+1}\}$ for $t=T_w+1,\ldots,T$ and solves
$$
\min_{W_{\mathrm{out}}}
\sum_t \|u_{t+1} - W_{\mathrm{out}}\xi_t\|^2 + \lambda \|W_{\mathrm{out}}\|_F^2,
$$
with closed-form solution
$$
W_{\mathrm{out}}^* = Y \Xi^\top (\Xi\Xi^\top + \lambda I)^{-1},
$$
where $Y$ stacks row targets, $\Xi$ stacks column features, and $I$ is the identity matrix [2508.18196]. Cholesky or SVD is used for numerical stability.

Two operating modes are distinguished. During training and validation, teacher forcing (open loop) feeds the true input $u_t$ at every step, ensuring stability and single-step error learning. During testing, autoregressive forecasting (closed loop) feeds $\hat{u}_{t+1}$ back as input, so long-horizon fidelity depends on whether the reservoir dynamics suppress or amplify relevant perturbations in a task-appropriate manner [2508.18196].

The paper develops a formal analysis of curvature-controlled divergence. For the row-sparse hyperbolic-kernel matrix $\widetilde{W}$ with distinct points $\{p_i\}$ and minimum pairwise hyperbolic distance
$$
\delta = \min_{i\ne j} d_{\mathbb{H}}(p_i,p_j),
$$
Lemma 1 gives
$$
\lambda_{\min}(\widetilde{W}) \ge 1 - (N-1)e^{-\delta/\sigma},
\qquad
\rho(\widetilde{W}) \le 1 + (N-1)e^{-\delta/\sigma}.
$$
As $\sigma \downarrow 0$, the ratio $\lambda_{\min}(\widetilde{W})/\rho(\widetilde{W})$ increases, which the paper interprets as lifting the spectral floor [2508.18196].

For a $C^1$ activation $\phi$ with $0 < m \le \phi'(z) \le L < \infty$ on the reachable domain, the one-step Jacobian is
$$
J(x) = \frac{\partial x_{t+1}}{\partial x_t}
=
(1-\alpha)I + \alpha D_\phi(x)W,
$$
where
$$
D_\phi(x)=\operatorname{diag}(\phi'(Wx + Uu)).
$$
Lemma 2 provides
$$
s_{\min}(J(x))
\ge
\sqrt{
\frac{m}{L}
\left[(1-\alpha)+\alpha m \lambda_{\min}(W)\right]
},
$$
and
$$
\|J(x)\|
\le
\sqrt{
\frac{L}{m}
\left[(1-\alpha)+\alpha L \rho(W)\right]
}.
$$
Since spectral normalization fixes $\rho(W)=\varrho<1$, the upper gain remains controlled, while the lower gain depends on the hyperbolic-kernel structure [2508.18196].

The main state-divergence result defines
$$
\beta(\sigma)
:=
\sqrt{
\frac{m}{L}
\left[
(1-\alpha)+\alpha m \varrho
\left(
\frac{\lambda_{\min}(\widetilde{W}(\sigma))}{\rho(\widetilde{W}(\sigma))}
\right)
\right]
}.
$$
If two input streams are identical up to time $t_0-1$, differ at $t_0$, and coincide thereafter, then for the resulting state difference $e_t = x_t - y_t$, if $\beta(\sigma)>1$,
$$
\|e_{t_0+\tau}\| \ge \alpha m \|\Delta u\| \beta(\sigma)^{\tau-1}
\qquad
\text{for all } \tau \ge 1,
$$
with $\Delta u = U(u_{t_0}-v_{t_0}) \ne 0$ [2508.18196]. The paper interprets this as a lower bound on reservoir state divergence that mirrors Lyapunov growth while preserving ESP via spectral normalization.

This analysis establishes the conceptual distinction from standard ESNs and graph-structured reservoirs. Standard ESNs and engineered topologies such as small-world or cycle reservoirs alter connectivity patterns in flat geometry, whereas HypER uses negative curvature and hyperbolic-distance decay to shape the local expansion–contraction spectrum itself [2508.18196].

## 5. Empirical evaluation

HypER is evaluated on synthetic chaotic systems and real-world time series. The synthetic benchmarks are Lorenz–63 with parameters $(\sigma,\rho,\beta)=(10,28,8/3)$ and largest Lyapunov exponent approximately $0.905$, the Rössler system with $(a,b,c)=(0.2,0.2,5.7)$ and largest Lyapunov exponent approximately $0.071$, and the hyperchaotic Chen–Ueta system with $(a,b,c)=(35,3,28)$ [2508.18196]. Trajectories are generated with LSODA at $\Delta t=0.02$ for $12{,}500$ samples; the first $2{,}000$ are discarded as washout, the next $80\%$ are used for training, and the final $20\%$ for testing.

The real-world datasets are MIT–BIH Arrhythmia (ECG), Santa Fe Dataset B (chaotic laser), and the Sunspot monthly index (SILSO v2.0). Signals are normalized to $[0,1]$ and represented through a $3$D delay embedding using standard $80/20$ splits [2508.18196].

Baselines include Euclidean ESN, SCR, CRJ, SW-ESN, MCI-ESN, and DeepESN. Reservoir sizes are $300$ units except for DeepESN, which uses $3\times 100$. Global hyperparameters are tuned with $U \sim \mathcal{N}(0,0.2^2)$, target $\rho(W)=0.99$, leak $\alpha=0.8$, and ridge regularization $\lambda=10^{-5}$. Results are averaged over $30$ runs [2508.18196].

The reported metrics are normalized root mean squared error (NRMSE), Valid Prediction Time (VPT), Attractor Deviation (ADev), and Welch power spectral density (PSD) [2508.18196].

| Benchmark | Result reported for HypER | Comparison reported |
|---|---:|---:|
| Lorenz VPT | $12.215 \pm 1.23$ | best baseline $8.049 \pm 2.36$ |
| Rössler VPT | $5.142 \pm 0.76$ | best baseline $\approx 3.427 \pm 0.68$ |
| Chen–Ueta VPT | $5.067 \pm 0.70$ | best baseline $\approx 2.628 \pm 0.44$ |
| Lorenz ADev | $19.67 \pm 6.82$ | best baseline $\approx 33.50 \pm 14.57$ |
| Rössler ADev | $1.27 \pm 1.17$ | best baseline $1.80 \pm 1.52$ |

On Lorenz–63, HypER attains NRMSE $0.0002 \pm 0.0001$ at $200$ steps and $1.2580 \pm 0.2721$ at $1000$ steps, outperforming MCI-ESN ($1.6795 \pm 0.2205$), SW-ESN ($1.8956 \pm 0.2003$), CRJ ($2.0490 \pm 0.3540$), Euclidean ESN ($2.1093 \pm 0.2033$), and DeepESN ($2.0663 \pm 0.1326$) at the $1000$-step horizon [2508.18196].

On Rössler, HypER maintains an order-of-magnitude lower error than Euclidean baselines at every horizon up to $1000$ steps; the paper gives the example of $0.0061 \pm 0.0031$ for HypER versus $0.0312 \pm 0.0509$ for MCI-ESN at $1000$ steps [2508.18196]. On Chen–Ueta, HypER reports $2.0413 \pm 0.1697$ at $1000$ steps, lower than SW-ESN at $2.3884 \pm 0.1746$ and lower than the other listed baselines [2508.18196].

PSD analysis is used to assess frequency-domain fidelity. The paper reports that HypER preserves the Lorenz power spectrum, whereas flat ESNs inject spurious high-frequency noise [2508.18196]. This suggests that the benefit is not limited to pointwise trajectory error, but extends to attractor-level dynamical statistics.

On real-world datasets, the gains are strongest for signals treated as chaotic or nonstationary. On MIT–BIH, HypER reports NRMSE $0.7528 \pm 0.1217$ at $1000$ steps compared with the best baseline MCI-ESN at $1.1262 \pm 0.0156$, corresponding to an improvement of approximately $33$–$35\%$ across horizons [2508.18196]. On Santa Fe, HypER gives $0.2451 \pm 0.0074$ at $1000$ steps versus the best baseline $0.2529 \pm 0.0011$. On sunspots, HypER is competitive but not superior, with $0.3488 \pm 0.0033$ at $1000$ steps versus MCI-ESN at $0.3361 \pm 0.0002$ [2508.18196]. The paper interprets this as consistent with theory: when $\lambda_{\max} \approx 0$, raising $\lambda_{\min}(W)$ is unnecessary.

## 6. Ablations, implementation, and relation to adjacent work

The ablation results identify geometry, sampling, kernel scale, and latent dimension as the main determinants of performance. For Lorenz at the $1000$-step horizon, moving from Euclidean to hyperbolic geometry with Euclidean-volume sampling reduces NRMSE by approximately $3\%$, and switching to hyperbolic-uniform sampling yields an additional approximately $7\%$ reduction, together giving the best VPT and ADev [2508.18196]. Increasing the latent dimension from the common Poincaré disk setting $d=2$ to $d=3$ or $4$ weakens gains, which the paper attributes to curvature dilution.

The kernel width $\sigma$ has a clear optimum around $\sigma \approx 0.1$. If $\sigma$ is too small, the graph becomes under-connected and VPT decreases; if $\sigma$ is too large, curvature effects are diluted and $\beta(\sigma)\to 1$ [2508.18196]. Performance plateaus for sparsity $\kappa \in [40,80]$, with denser graphs slightly improving ADev, but NRMSE gains attributed mainly to geometry rather than overall connectivity scale.

Mixed activations improve all reservoir types, but the improvement is strongest for HypER. For Lorenz, the reported mixed-activation comparison gives HypER NRMSE $1.2580$ versus the best graph baseline at $1.4974$, and VPT $12.215$ versus $9.636$ [2508.18196].

A typical implementation uses $N=300$ units, $d=2$, $\sigma \approx 0.1$, $\kappa \in [40,80]$, $\varrho = 0.99$, $\alpha = 0.8$, $\lambda=10^{-5}$, and $U \sim \mathcal{N}(0,0.2^2)$ [2508.18196]. Dense adjacency construction is $O(N^2)$; row-wise sparsification reduces subsequent structure to $O(\kappa N)$. Readout training is dominated by solving linear systems of size $(2N+1)\times(2N+1)$ [2508.18196]. No manifold optimization is used because the reservoir is fixed after sampling and normalization.

HypER’s use of the Poincaré ball also places it near a broader numerical discussion in hyperbolic ML. Work on the numerical stability of hyperbolic representation learning has shown that hyperbolic models can suffer catastrophic NaNs and floating-point pathologies near the Poincaré boundary or at large Lorentz time coordinates, with different trade-offs between representational capacity and optimization behavior [2211.00181]. HypER avoids many of these issues because it does not train manifold coordinates directly; neuron positions are sampled once, and only the Euclidean readout is optimized [2508.18196]. This suggests that fixed-geometry reservoir computing may be an attractive regime for exploiting hyperbolic structure without incurring the full optimization burden of end-to-end hyperbolic learning.

The method is also distinct from static hyperbolic embedding frameworks such as Mercator, which infer latent $\mathbb{H}^2$ coordinates of network nodes under the Popularity$\times$Similarity model [1904.10814], and from hyperbolic dense retrieval systems such as HypRAG, where hyperbolic geometry is used to encode semantic hierarchy and preserve specificity through norm- or radius-based separation [2602.07739]. The shared principle is the use of exponential metric growth as inductive bias, but HypER applies it to dynamical forecasting rather than network embedding or retrieval.

## 7. Limitations and prospective directions

HypER fixes curvature at $K=-1$ and uses hyperbolic-uniform sampling with a maximum radius parameter $\rho_{\max}$. Although effective in the reported experiments, performance still depends on $\rho_{\max}$, $\sigma$, $\kappa$, $\alpha$, $\varrho$, and input scaling [2508.18196]. Careful tuning remains necessary, so the geometric prior does not eliminate hyperparameter sensitivity.

The method’s computational construction is naively $O(N^2)$ in the number of reservoir units, which limits straightforward scaling to very large reservoirs. The paper explicitly identifies scalable approximations such as hyperbolic $k$-nearest neighbors and fast kernel evaluation as useful directions [2508.18196]. It also notes that gains diminish as latent dimension increases, leaving open the question of how to balance representational capacity against curvature concentration.

Several extensions are proposed: adaptive or learned node placement, task-specific curvature or variable metrics, hybrid knowledge-based plus reservoir schemes, online adaptation, output feedback, and multi-layer hyperbolic reservoirs [2508.18196]. A systematic relation between Lyapunov exponents and geometric control parameters such as curvature and $\sigma$ is also identified as an open problem. This suggests a possible future program in which reservoir geometry is calibrated directly to the Lyapunov spectrum of the target system rather than chosen through heuristic tuning.

A common misconception would be to view HypER as simply an ESN with a different graph topology. The paper argues for a stronger claim: the essential modification is not merely graph sparsity or nonrandom wiring, but the introduction of a negative-curvature metric into the latent state space, so that the reservoir’s local expansion–contraction properties can track the geometry of chaotic divergence [2508.18196]. Whether this principle generalizes beyond the studied chaotic and real-world benchmarks remains an empirical question, but within the reported scope, HypER presents a geometrically explicit formulation of reservoir computing in which hyperbolic structure functions as a dynamical inductive bias rather than a static embedding device.

Source: https://www.emergentmind.com/topics/hyperbolic-embedding-reservoir-hyper