Hyperbolic Dense Retrieval

• Hyperbolic dense retrieval is a method that encodes queries and items in hyperbolic space to represent tree-like, hierarchical structures effectively.
• It employs fully hyperbolic and hybrid architectures that project Euclidean embeddings into curved spaces, thereby alleviating issues like hubness and similarity crowding.
• Empirical evaluations demonstrate improved performance and scalability on benchmarks such as CodeSearchNet and RAGBench, validating its efficiency in multi-modal retrieval tasks.

Hyperbolic dense retrieval is a class of information retrieval methods that encode queries and candidate items (text, code, multimodal instances) as points in hyperbolic space, typically modeled via the Poincaré ball or Lorentz hyperboloid. The key rationale is that negative curvature affords exponentially growing volume with increasing radius, naturally mirroring the hierarchical and branching structures pervasive in natural language, code, and structured data. In contrast to conventional Euclidean dense retrieval, which suffers from hubness and crowding when encoding taxonomies or multi-level semantics, hyperbolic approaches maintain separation between semantically distant or specific entities, improving retrieval fidelity, efficiency, and scalability in settings such as code search, retrieval-augmented generation, and cross-modal matching.

1. Geometric Foundations of Hyperbolic Dense Retrieval

Let $d$ denote the embedding dimension. Two principal models of hyperbolic space are deployed:

• Poincaré Ball Model: $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$, endowed with the metric $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$ (where $g^E$ is the Euclidean metric tensor). Hyperbolic distance is given by

$$d(q, a) = \operatorname{arcosh}\left(1 + 2 \frac{\|q - a\|^2}{(1-\|q\|^2)(1-\|a\|^2)}\right).$$

The exponential map at the origin reduces to $\exp_0(v) = \tanh(\|v\|)\frac{v}{\|v\|}$, enabling Euclidean features to be retracted into the ball.

• Lorentz Model: $$\mathbb{H}^d_c = \{u = (\tilde u, u_{d+1}) \in \mathbb{R}^{d+1} \mid \langle u, u\rangle_{\mathcal{L}} = -1/c,\, u_{d+1} > 0\}$$, with Lorentzian inner product $$\langle u, v\rangle_{\mathcal{L}} = \langle \tilde u, \tilde v\rangle_E - u_{d+1} v_{d+1}$$. Geodesic distance is

$$d_{\mathbb{H}}(u,v) = \frac{1}{\sqrt{c}}\operatorname{arccosh}[ -c \langle u, v\rangle_{\mathcal{L}} ].$$

Crucially, the exponential volume growth of hyperbolic balls provides sufficient representational capacity for tree-like and hierarchical data, with general concepts near the origin and specific concepts near the boundary. This property underpins the suitability of hyperbolic spaces for embedding structured semantics (Tang et al., 2023, Madhu et al., 8 Feb 2026, Li et al., 14 Nov 2025).

2. Model Architectures and Pooling Strategies

Contemporary hyperbolic dense retrieval methods instantiate hyperbolic geometry either via fully hyperbolic encoders or hybrid projections:

• Fully Hyperbolic Transformer (HyTE-FH): All encoder layers operate directly in the Lorentz model, including Lorentz-linear transforms, hyperbolic LayerNorm, Lorentz residual connections, and Lorentz self-attention mechanisms. Attention aggregates via the Lorentz-weighted midpoint:

$$\operatorname{Att}_L(q_i) = \frac{\sum_j \nu_{ij} \lambda_j v_j}{\sqrt{-K}\|\sum_j \nu_{ij} \lambda_j v_j\|_L}, \quad \lambda_j = v_{j,0}$$

with weights $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$0 (Madhu et al., 8 Feb 2026).

• Hybrid Hyperbolic Projection (HyTE-H, HyCoQA): Pre-trained Euclidean encoders (e.g., BERT or bi-encoders) output $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$1 features, which are projected into hyperbolic space—for the Lorentz model, via

$$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$2

where $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$3 (Madhu et al., 8 Feb 2026).

• Geometry-Aware Pooling:
  • Outward Einstein Midpoint (OEM): To preserve hierarchical separation during aggregation, OEM applies a radial amplification factor and pools in the ambient space before projecting back to the manifold, mathematically defined as

  $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$4

  then projects $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$5 onto $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$6 (Madhu et al., 8 Feb 2026). - Contribution-Aware Aggregation (H²ARN): Local features are weighted according to Lorentzian distance from a mean anchor, ensuring semantically salient components have higher influence:

  $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$7

  leading to a denoised, hierarchical global embedding (Li et al., 14 Nov 2025).

Pooling operators with outward bias provably amplify or at least preserve the radial (hierarchical) depth, in contrast with naïve Euclidean mean pooling, which contracts hierarchical separation.

3. Loss Functions, Optimization, and Self-Organization

Hyperbolic dense retrievers typically adopt contrastive or hinge-based training objectives, adapted to hyperbolic geometry:

• Pairwise Hinge Loss (Tang et al., 2023):

$$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$8

with $$\mathcal{B}^d = \{x \in \mathbb{R}^d \mid \|x\| < 1\}$$9 a (linearly parameterized) function of the hyperbolic distance.

• Contrastive InfoNCE Loss (Li et al., 14 Nov 2025):

$g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$0

with (multi-positive) batch-based normalization.

• Hierarchical Ordering Loss (Entailment Cone) (Li et al., 14 Nov 2025): For text-3D matching, a loss term maintains a cone of half-angle $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$1 around $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$2 such that its paired instance $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$3 must fall within:

$g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$4

where $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$5 measures the angular separation (in Lorentzian sense).

• Riemannian Optimization: All gradients are converted from the Euclidean to the Riemannian setting, and parameters are updated via retraction (e.g., $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$6) (Tang et al., 2023).

Latent hierarchy emerges during model training. In Poincaré or Lorentz geometry, low-norm points represent general/root concepts, while specific/leaf concepts are mapped near the boundary—without explicit tree labels, models learn to encode hierarchical entailment and semantic specificity in embedding norms (Tang et al., 2023, Madhu et al., 8 Feb 2026, Li et al., 14 Nov 2025).

4. Empirical Evaluations and Comparative Performance

Empirical validation demonstrates consistent superiority of hyperbolic dense retrievers over Euclidean baselines across modalities:

Benchmark	Euclidean Baseline	Hyperbolic (Hybrid)	Hyperbolic (Fully)
MTEB (mean score)	54.11 (EucBERT)	54.57 (HyTE-H^Euc)	56.41 (HyTE-FH)
MTEB (mean type)	51.31	53.71	53.75
RAGBench F/CR/AR	0.596/0.798/0.647	0.706/0.814/0.739	0.732/0.848/0.765

• On CodeSearchNet, the HyCoQA framework achieves average MRR improvement of 3.5–4.0% over competitive baselines (CodeBERT, CoCoSoDa, CodeRetriever), with ablation showing a –4.38% MRR drop when the hyperbolic layer is removed (Tang et al., 2023).
• On RAGBench, HyTE-H (149M parameters) surpasses state-of-the-art 300–500M parameter multilingual Euclidean models (GTE, Gemma, KaLM) with up to 29% higher context/answer relevance (Madhu et al., 8 Feb 2026).
• In text–3D retrieval (T3DR-HIT v2), the Lorentzian H²ARN method alleviates hierarchy representation collapse and redundancy-induced saliency dilution, outperforming previous Euclidean and semantic alignment models (Li et al., 14 Nov 2025).

A prominent analytical observation is that embedding radial depth (norm or $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$7 coordinate) increases by 20–25% from root to leaf in controlled taxonomies for hyperbolic models, while remaining flat or decreasing in Euclidean variants (Madhu et al., 8 Feb 2026).

5. Computational and Practical Considerations

Hyperbolic dense retrieval is optimized for scalability in both training and inference:

• Per-Pair Complexity: Hyperbolic distance computations are $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$8; scoring $g_x = \left(\frac{2}{1-\|x\|^2}\right)^2 g^E$9 candidates is $g^E$0.
• Indexing: Approximate nearest neighbor structures (e.g., HNSW) under hyperbolic (Lorentzian) distance are usable; empirical deployments achieve sub-millisecond pairwise retrieval, with millions of daily queries feasible (Tang et al., 2023, Li et al., 14 Nov 2025).
• Storage: Storing $g^E$1 dense $g^E$2-dimensional vectors scales as $g^E$3; for $g^E$4, this requires $g^E$510GB—tractable via vector quantization or sharding (Tang et al., 2023).
• Pooling Efficiency: The Outward Einstein Midpoint and contribution-aware aggregation are efficient, with provable properties preserving hierarchy without significant compute overhead.

HyCoQA dispenses with complex query-document interaction layers, requiring only distance computations and simple pooling. Both the hybrid and fully hyperbolic approaches remain compatible with frozen or partially trainable feature encoders (e.g., BERT).

6. Addressing Typical Failure Modes in Euclidean Dense Retrieval

Several hierarchical and retrieval-specific phenomena challenge Euclidean methods:

• Hierarchy Representation Collapse (HRC): Polynomial volume growth in Euclidean space induces crowding of abstract and specific nodes, demolishing semantic entailment and deteriorating performance for hierarchical reasoning (Li et al., 14 Nov 2025).
• Redundancy-Induced Saliency Dilution (RISD): Averaging over local representations, as is typical in Euclidean pooling, obscures discriminative cues and hard negatives; hyperbolic contribution-aware aggregation reweights features using Lorentzian distances to suppress noise.
• Spurious Similarity and Hubness: Lacking curvature, Euclidean encoders assign similar distances to semantically distant objects, increasing false positives in retrieval and risk of hallucination in downstream systems (Madhu et al., 8 Feb 2026).
• Loss of Specificity Encoding: Euclidean norms do not separate general/specific concepts; hyperbolic norms reliably reflect concept specificity, aiding both relevance ranking and entailment.

By intrinsically aligning the embedding geometry to the underlying data structure, hyperbolic dense retrieval mitigates these effects, enabling higher recall, stronger generalization, and faithful context selection for downstream consumption (Tang et al., 2023, Li et al., 14 Nov 2025, Madhu et al., 8 Feb 2026).

7. Applications and Future Directions

Hyperbolic dense retrieval architectures have been validated in code search (HyCoQA on CodeSearchNet) (Tang et al., 2023), retrieval-augmented generation (HyTE/FH, HyTE-H on MTEB, RAGBench) (Madhu et al., 8 Feb 2026), and text–3D retrieval (H²ARN on T3DR-HIT v2) (Li et al., 14 Nov 2025). They are suited to any domain with tree-like or hierarchical semantics, including knowledge graphs, biomedical ontologies, or taxonomic search.

A plausible implication is the broad applicability of geometric inductive bias to future retriever-generator pipelines, supporting higher faithfulness, lower hallucination rates, and improved performance even with smaller models. Theoretical and practical extensions include scaling to extremely large retrieval corpora, further optimization of hyperbolic pooling operators, and direct coupling with generative models in the hyperbolic embedding domain.

Empirical results and the geometric rationale both underscore embedding geometry as a critical design axis for dense retrieval—shaping new approaches to efficient, faithful, and hierarchy-aware information access systems.