---
title: 'HyperbolicRAG: Hierarchy-Aware Retrieval'
url: https://www.emergentmind.com/topics/hyperbolicrag
type: topic
---

# HyperbolicRAG: Hierarchy-Aware Retrieval

HyperbolicRAG is a retrieval-augmented generation (RAG) framework that employs hyperbolic geometry to jointly model semantic similarity and hierarchical abstraction in retrieval pipelines for large language models (LLMs). By embedding information units within a shared Poincaré manifold and fusing retrieval signals from both Euclidean and hyperbolic space, HyperbolicRAG enables fine-grained semantic matching together with global hierarchy-aware reasoning for complex knowledge graphs. Empirical results show robust gains in evidence retrieval and multi-hop question answering tasks over standard and graph-based RAG approaches [2511.18808].

## 1. Hyperbolic Geometry in Representation Learning

HyperbolicRAG leverages the Poincaré ball model of hyperbolic space to encode latent hierarchy. The $d$-dimensional Poincaré ball of negative curvature $-c$ is defined as $\mathbb{H}^c_d = \{x\in\mathbb{R}^d : c\|x\|^2 < 1\}$, with the metric $g_x = \lambda_x^2 g^E$, where $\lambda_x = 2/(1 - c\|x\|^2)$ and $g^E$ is the Euclidean metric. The geodesic distance between two points $u, v \in \mathbb{H}_d^c$ is
\[
d_{\mathbb{H}^c}(u, v) = \frac{1}{\sqrt{c}} \operatorname{arcosh}\left(1 + 2c\frac{\|u-v\|^2}{(1 - c\|u\|^2)(1 - c\|v\|^2)}\right).
\]
Hierarchical containment is represented by the radial coordinate: nodes near the origin encode high-level abstractions, and nodes near the boundary encode specific facts. The exponential volume growth of hyperbolic space enables the representation of large trees and hierarchies with low distortion [2511.18808].

## 2. Depth-Aware Representation Learning Pipeline

For each candidate node (fact, passage, entity), an initial Euclidean embedding $z_v^E$ is computed via a pretrained encoder. These features pass through a multi-stage transformation:
- **Hierarchy Feature Extraction:** $u_v = \phi(z_v^E)$ via a small MLP.
- **Depth Prediction:** $d_v = \psi_{\mathrm{mode}(v)}(u_v)$, predicting radial depth within $[0, 1]$ adaptively for passages, facts, or entities.
- **Fusion and Radial Alignment:** A gated feature fusion combines $z_v^E$ and hierarchy features; the result is rescaled such that the radial norm encodes depth:
  \[
  \|\hat z_v^E\| = \alpha + \beta d_v, \quad \hat z_v^E = \frac{\alpha + \beta d_v}{\|z_v^*\|} z_v^*,
  \]
  with hyperparameters $\alpha, \beta > 0$, $\alpha + \beta \leq 1$.
- **Poincaré Mapping:** The rescaled embedding is mapped into hyperbolic space by:
  \[
  z_v^H = \exp^c_0(\hat z_v^E) = \tanh(\sqrt{c} \|\hat z_v^E\|) \frac{\hat z_v^E}{\sqrt{c} \|\hat z_v^E\|}.
  \]

This architecture achieves joint semantic-containment optimization: direction encodes local semantics, while norm encodes abstraction depth. The design allows passage and fact embeddings to simultaneously capture fine-grained similarity and strict containment relationships [2511.18808].

## 3. Unsupervised Contrastive Regularization

Contrastive learning enforces cross-level geometric consistency. For each passage, positives are facts present within that passage, and negatives are facts outside of it (and vice versa for facts). Hyperbolic margin-based losses are used:
- **Passage-to-Fact:**
  \[
  \mathcal{L}_{p \to f} = \sum_p \sum_{f^+ \in \mathcal{F}(p)} [ d_H(z_p^H, z_{f^+}^H) - d_H(z_p^H, z_{f^-}^H) + \gamma ]_+
  \]
- **Fact-to-Passage:**
  \[
  \mathcal{L}_{f \to p} = \sum_f \sum_{p^+ \in \mathcal{P}(f)} [ d_H(z_f^H, z_{p^+}^H) - d_H(z_f^H, z_{p^-}^H) + \gamma ]_+
  \]
- **Total Loss:**
  \[
  \mathcal{L}_{\mathrm{align}} = \mathcal{L}_{p \to f} + \mathcal{L}_{f \to p}
  \]
This strict alignment pushes each passage closer to its own facts (and vice versa) than to negatives, encoding containments in hyperbolic space [2511.18808].

## 4. Mutual-Ranking Fusion Mechanism

Retrieval aggregates ranked signals from both Euclidean and hyperbolic branches:
- **Euclidean Branch:** Queries encoded in $\mathbb{R}^d$ are used for top-$k$ fact retrieval by cosine similarity, with personalized PageRank (PPR) propagation over a passage–entity graph.
- **Hyperbolic Branch:** Queries are projected into $\mathbb{H}_d^c$, and top-$k$ facts are ranked by negative hyperbolic distance, followed by PPR.
- **Mutual Fusion:** For each passage, combined rank scores from both branches are fused, with a “reciprocal rank” bonus for agreement:
  ```
  for each passage p:
    r_E := position in Euclidean ranking
    r_H := position in Hyperbolic ranking
    s_E := 1/(r_E+1)
    s_H := 1/(r_H+1)
    b   := 1/(r_E + r_H + 2)
    score(p) := (s_E + s_H) * (1 + b)
  ```
This late-fusion rewards consistency and enables retrieval of passages that are salient in both spaces [2511.18808].

## 5. Empirical Results and Ablations

Comprehensive evaluation on QA benchmarks (NQ, PopQA, MuSiQue, 2WikiMultihopQA, HotpotQA) with 1,000 queries each demonstrated the following Recall@5 values (average over all datasets): BM25 55.1%, NV-Embed-v2 73.4%, HippoRAG2 78.2%, HyperbolicRAG 79.0% (best). On multi-hop subsets, HyperbolicRAG achieved 76.2%, 92.1%, 96.3% for MuSiQue, 2Wiki, and Hotpot, respectively. End-to-end QA scores (Exact Match / F1 average) were 49.0/60.0 (NV-Embed-v2), 51.0/62.7 (HippoRAG2), and 51.4/63.3 (HyperbolicRAG) [2511.18808]. Ablation studies showed that:
- Removing the hyperbolic branch lowered Recall@5 (to 74.7%, 90.4%, 96.2% on multi-hop).
- Hyperbolic branch alone also reduced recall versus fusion, but outperformed standard Euclidean-only alignment.
- The dual-branch fusion is essential for state-of-the-art evidence recall, especially on multi-hop reasoning.

Performance was stable across diverse embedding backbones (GTE-Qwen2-7B, GritLM-7B, NV-Embed-v2-7B, Qwen3-8B), LLM generators (LLaMA-3 70B, QWQ-32B), and robust to curvature parameter $c$ variation [2511.18808].

## 6. Implementation Guidelines and Hyperparameters

Key hyperparameters include curvature $c$ (default 1.0); radial alignment coefficients $\alpha, \beta$ (e.g., $\alpha=0.1, \beta=0.8$); margin $\gamma$ (default 1.0); PPR restart probability (e.g., $0.15$); number of top-$k$ facts for initialization ($k=20$); embedding dimension $d=768$; hierarchy MLP dimension $d'=128$; and learning rates for encoder ($1\times 10^{-5}$) and depth predictor ($1\times 10^{-4}$). Recommended implementation practices include precomputing embeddings, building the passage–entity graph once, enforcing numerical stability in distance/activation calculations, and monitoring radial norms to prevent boundary saturation ($\|z\| < 1$). The use of the same graph adjacency and PPR parameters in both branches ensures comparability [2511.18808].

## 7. Significance and Connections to Graph Geometry

HyperbolicRAG bridges advances in geometric graph modeling and neural retrieval. Its use of the Poincaré ball for containment-aware embeddings is motivated by findings in hyperbolic random graph models [1503.05180], which showed that hyperbolic geometry naturally reproduces complex network phenomena such as power-law degree distributions and hierarchical clustering. By combining this with modern contrastive learning and dual-space mutual ranking, HyperbolicRAG provides a principled and empirically validated approach to hierarchy-aware retrieval for LLMs. This suggests broad applicability for hierarchical and multi-hop reasoning tasks where information structure is inherently non-Euclidean [2511.18808].

Source: https://www.emergentmind.com/topics/hyperbolicrag