HyperbolicRAG: Hierarchy-Aware Retrieval

• HyperbolicRAG is a retrieval-augmented generation framework that uses the Poincaré ball model to encode both semantic similarity and hierarchical abstraction.
• It fuses Euclidean and hyperbolic retrieval signals, enabling fine-grained matching and global, hierarchy-aware reasoning for complex knowledge graphs.
• Empirical evaluations on multi-hop question answering tasks show robust improvements in evidence retrieval over standard and graph-based methods.

HyperbolicRAG is a retrieval-augmented generation (RAG) framework that employs hyperbolic geometry to jointly model semantic similarity and hierarchical abstraction in retrieval pipelines for LLMs. By embedding information units within a shared Poincaré manifold and fusing retrieval signals from both Euclidean and hyperbolic space, HyperbolicRAG enables fine-grained semantic matching together with global hierarchy-aware reasoning for complex knowledge graphs. Empirical results show robust gains in evidence retrieval and multi-hop question answering tasks over standard and graph-based RAG approaches (Linxiao et al., 24 Nov 2025).

1. Hyperbolic Geometry in Representation Learning

HyperbolicRAG leverages the Poincaré ball model of hyperbolic space to encode latent hierarchy. The $d$-dimensional Poincaré ball of negative curvature $-c$ is defined as $$\mathbb{H}^c_d = \{x\in\mathbb{R}^d : c\|x\|^2 < 1\}$$, with the metric $g_x = \lambda_x^2 g^E$, where $\lambda_x = 2/(1 - c\|x\|^2)$ and $g^E$ is the Euclidean metric. The geodesic distance between two points $u, v \in \mathbb{H}_d^c$ is

$$d_{\mathbb{H}^c}(u, v) = \frac{1}{\sqrt{c}} \operatorname{arcosh}\left(1 + 2c\frac{\|u-v\|^2}{(1 - c\|u\|^2)(1 - c\|v\|^2)}\right).$$

Hierarchical containment is represented by the radial coordinate: nodes near the origin encode high-level abstractions, and nodes near the boundary encode specific facts. The exponential volume growth of hyperbolic space enables the representation of large trees and hierarchies with low distortion (Linxiao et al., 24 Nov 2025).

2. Depth-Aware Representation Learning Pipeline

For each candidate node (fact, passage, entity), an initial Euclidean embedding $z_v^E$ is computed via a pretrained encoder. These features pass through a multi-stage transformation:

• Hierarchy Feature Extraction: $u_v = \phi(z_v^E)$ via a small MLP.
• Depth Prediction: $d_v = \psi_{\mathrm{mode}(v)}(u_v)$, predicting radial depth within $[0, 1]$ adaptively for passages, facts, or entities.
• Fusion and Radial Alignment: A gated feature fusion combines $z_v^E$ and hierarchy features; the result is rescaled such that the radial norm encodes depth:

$$\|\hat z_v^E\| = \alpha + \beta d_v, \quad \hat z_v^E = \frac{\alpha + \beta d_v}{\|z_v^*\|} z_v^*,$$

with hyperparameters $\alpha, \beta > 0$, $\alpha + \beta \leq 1$.

• Poincaré Mapping: The rescaled embedding is mapped into hyperbolic space by:

$$z_v^H = \exp^c_0(\hat z_v^E) = \tanh(\sqrt{c} \|\hat z_v^E\|) \frac{\hat z_v^E}{\sqrt{c} \|\hat z_v^E\|}.$$

This architecture achieves joint semantic-containment optimization: direction encodes local semantics, while norm encodes abstraction depth. The design allows passage and fact embeddings to simultaneously capture fine-grained similarity and strict containment relationships (Linxiao et al., 24 Nov 2025).

3. Unsupervised Contrastive Regularization

Contrastive learning enforces cross-level geometric consistency. For each passage, positives are facts present within that passage, and negatives are facts outside of it (and vice versa for facts). Hyperbolic margin-based losses are used:

• Passage-to-Fact:

$$\mathcal{L}_{p \to f} = \sum_p \sum_{f^+ \in \mathcal{F}(p)} [ d_H(z_p^H, z_{f^+}^H) - d_H(z_p^H, z_{f^-}^H) + \gamma ]_+$$

• Fact-to-Passage:

$$\mathcal{L}_{f \to p} = \sum_f \sum_{p^+ \in \mathcal{P}(f)} [ d_H(z_f^H, z_{p^+}^H) - d_H(z_f^H, z_{p^-}^H) + \gamma ]_+$$

• Total Loss:

$$\mathcal{L}_{\mathrm{align}} = \mathcal{L}_{p \to f} + \mathcal{L}_{f \to p}$$

This strict alignment pushes each passage closer to its own facts (and vice versa) than to negatives, encoding containments in hyperbolic space (Linxiao et al., 24 Nov 2025).

4. Mutual-Ranking Fusion Mechanism

Retrieval aggregates ranked signals from both Euclidean and hyperbolic branches:

• Euclidean Branch: Queries encoded in $\mathbb{R}^d$ are used for top-$k$ fact retrieval by cosine similarity, with personalized PageRank (PPR) propagation over a passage–entity graph.
• Hyperbolic Branch: Queries are projected into $\mathbb{H}_d^c$, and top-$k$ facts are ranked by negative hyperbolic distance, followed by PPR.
• Mutual Fusion: For each passage, combined rank scores from both branches are fused, with a “reciprocal rank” bonus for agreement:

  for each passage p: r_E := position in Euclidean ranking r_H := position in Hyperbolic ranking s_E := 1/(r_E+1) s_H := 1/(r_H+1) b := 1/(r_E + r_H + 2) score(p) := (s_E + s_H) * (1 + b)

  This late-fusion rewards consistency and enables retrieval of passages that are salient in both spaces (Linxiao et al., 24 Nov 2025).

5. Empirical Results and Ablations

Comprehensive evaluation on QA benchmarks (NQ, PopQA, MuSiQue, 2WikiMultihopQA, HotpotQA) with 1,000 queries each demonstrated the following Recall@5 values (average over all datasets): BM25 55.1%, NV-Embed-v2 73.4%, HippoRAG2 78.2%, HyperbolicRAG 79.0% (best). On multi-hop subsets, HyperbolicRAG achieved 76.2%, 92.1%, 96.3% for MuSiQue, 2Wiki, and Hotpot, respectively. End-to-end QA scores (Exact Match / F1 average) were 49.0/60.0 (NV-Embed-v2), 51.0/62.7 (HippoRAG2), and 51.4/63.3 (HyperbolicRAG) (Linxiao et al., 24 Nov 2025). Ablation studies showed that:

• Removing the hyperbolic branch lowered Recall@5 (to 74.7%, 90.4%, 96.2% on multi-hop).
• Hyperbolic branch alone also reduced recall versus fusion, but outperformed standard Euclidean-only alignment.
• The dual-branch fusion is essential for state-of-the-art evidence recall, especially on multi-hop reasoning.

Performance was stable across diverse embedding backbones (GTE-Qwen2-7B, GritLM-7B, NV-Embed-v2-7B, Qwen3-8B), LLM generators (LLaMA-3 70B, QWQ-32B), and robust to curvature parameter $c$ variation (Linxiao et al., 24 Nov 2025).

6. Implementation Guidelines and Hyperparameters

Key hyperparameters include curvature $c$ (default 1.0); radial alignment coefficients $\alpha, \beta$ (e.g., $\alpha=0.1, \beta=0.8$); margin $\gamma$ (default 1.0); PPR restart probability (e.g., $0.15$); number of top-$k$ facts for initialization ($k=20$); embedding dimension $d=768$; hierarchy MLP dimension $d'=128$; and learning rates for encoder ($1\times 10^{-5}$) and depth predictor ($1\times 10^{-4}$). Recommended implementation practices include precomputing embeddings, building the passage–entity graph once, enforcing numerical stability in distance/activation calculations, and monitoring radial norms to prevent boundary saturation ($\|z\| < 1$). The use of the same graph adjacency and PPR parameters in both branches ensures comparability (Linxiao et al., 24 Nov 2025).

7. Significance and Connections to Graph Geometry

HyperbolicRAG bridges advances in geometric graph modeling and neural retrieval. Its use of the Poincaré ball for containment-aware embeddings is motivated by findings in hyperbolic random graph models (Aldecoa et al., 2015), which showed that hyperbolic geometry naturally reproduces complex network phenomena such as power-law degree distributions and hierarchical clustering. By combining this with modern contrastive learning and dual-space mutual ranking, HyperbolicRAG provides a principled and empirically validated approach to hierarchy-aware retrieval for LLMs. This suggests broad applicability for hierarchical and multi-hop reasoning tasks where information structure is inherently non-Euclidean (Linxiao et al., 24 Nov 2025).