ReTreever: Hierarchical Tree-based Retrieval

• ReTreever is a tree-based retrieval paradigm that employs a learned perfect binary tree to generate both coarse and fine representations.
• It utilizes specialized split functions and propagation schemes, including cross-attention, to achieve logarithmic retrieval latency and enhanced accuracy.
• The approach demonstrates competitive performance and interpretability across diverse tasks such as document retrieval, recommendation, and cross-modal applications.

ReTreever is a tree-based retrieval paradigm in which retrieval is mediated by an explicit hierarchy rather than by a single flat embedding space. In its most specific usage, it denotes a method for organizing and representing reference documents at various granular levels through a learned perfect binary tree of assignment distributions, so that coarse representations of size $2^h$ and a fine leaf-level representation are derived from the same trained structure (Gupta et al., 11 Feb 2025). In adjacent literature, the same name or an explicitly mapped concept is also used for tree-structured retrieval in token-efficient cross attention, recommendation, syntax-guided question answering, repository-level code retrieval, and textual-graph retrieval, making ReTreever both a specific model family and a broader design pattern for hierarchical retrieval (Feng et al., 2023, Liu et al., 2024, Zhang et al., 31 May 2025, Fehr et al., 30 May 2025, Wei et al., 8 Jan 2026).

1. Terminological scope and research lineage

The explicit use of the name “ReTreever” appears in at least two distinct but related lines of work. In “Tree Cross Attention,” ReTreever is a flexible architecture built around Tree Cross Attention (TCA), where a balanced tree over context tokens supports logarithmic retrieval during inference (Feng et al., 2023). In “ReTreever: Tree-based Coarse-to-Fine Representations for Retrieval,” the name is used for a document retriever that replaces a single dense vector with a hierarchy of assignment distributions over a learned binary tree, directly targeting retrieval in QA and RAG settings (Gupta et al., 11 Feb 2025).

A broader conceptual usage is also explicit in several papers. “Learning Deep Tree-based Retriever for Efficient Recommendation” states that Deep Tree-based Retriever (DTR) is mapped to the concept “ReTreever,” emphasizing a learnable tree-based retriever for recommendation that jointly optimizes a neural scoring model and a hierarchical index (Liu et al., 2024). “TreeRare” states that its retrieval-only variant, “Tree-Retrieval,” can serve as a general-purpose “ReTreever” module for syntax-tree-guided retrieval (Zhang et al., 31 May 2025). The CoRet summary likewise presents a “ReTreever” as a structure-aware dense retriever for repository-level code editing, while T-Retriever is described as embodying tree-based indexing and retrieval explicitly in textual attributed graphs (Fehr et al., 30 May 2025, Wei et al., 8 Jan 2026).

Work	Retrieval object	Tree role
“Tree Cross Attention” (Feng et al., 2023)	Context tokens	Balanced tree with top-down search
“ReTreever: Tree-based Coarse-to-Fine Representations for Retrieval” (Gupta et al., 11 Feb 2025)	Reference documents	Learned perfect binary tree of assignment distributions
“Learning Deep Tree-based Retriever for Efficient Recommendation” (Liu et al., 2024)	Recommendation items	Learned tree with beam-search-aligned retrieval
“TreeRare” (Zhang et al., 31 May 2025)	Passages for syntax-tree nodes	Question syntax tree guides retrieval
“CoRet” (Fehr et al., 30 May 2025)	Repository code chunks	Repository structure and call graph guide retrieval
“T-Retriever” (Wei et al., 8 Jan 2026)	Textual graph clusters	Encoding tree over attributed graphs

This distribution of usages suggests that ReTreever is best understood not as a single invariant architecture, but as a family resemblance among systems that use explicit trees to trade off retrieval cost, accuracy, structural control, and interpretability.

2. Core architecture of the document-retrieval ReTreever

In the document-retrieval formulation, ReTreever begins with a frozen encoder $E$ such as BAAI/bge-large-en-v1.5 and introduces a learnable perfect binary tree $T$ of depth $D$ (Gupta et al., 11 Feb 2025). Internal nodes implement routing through split functions, and leaves represent terminal groups. For an input $x$, the leaf-level assignment $T(x)\in[0,1]^{|T_L|}$ is the fine representation, while the assignments at intermediate depth $h$ are coarse representations of size $2^h$.

For an internal node $t$, the split function outputs a scalar score $s_{\theta_t}(x)\in\mathbb{R}$, which is converted into left and right routing probabilities by

$E$0

The paper’s strongest split function is a cross-attention module with learnable node embeddings $E$1 and shared projections,

$E$2

followed by

$E$3

A node-specific scorer then aggregates the attention outputs into $E$4, and the best-performing variant further refines this with a per-node MLP that incorporates ancestor scores.

Two propagation schemes are used. In product propagation, the leaf probability is

$E$5

where $E$6. The induced node reach probability is

$E$7

which enforces $E$8 and $E$9. In learned propagation, a leaf-specific function $T$0 maps ancestor split probabilities to a leaf score $T$1, after which normalization across leaves yields $T$2. The paper reports that learned propagation is better than product propagation at leaf level (Gupta et al., 11 Feb 2025).

This design makes the hierarchy itself the representation. Rather than compressing documents post hoc, ReTreever learns routing functions so that queries and contexts are assigned to similar branches, and the same trained tree can be queried at different depths without retraining.

3. Similarity, training objective, and retrieval procedure

The document-retrieval ReTreever uses negative Total Variation Distance as its similarity between leaf-level assignment distributions:

$T$3

Given a batch of positive query-context pairs $T$4, with $T$5 and $T$6, the training loss is a symmetric InfoNCE objective,

$T$7

This directly encourages positive pairs to have similar routes and leaf distributions while using in-batch negatives to prevent collapse (Gupta et al., 11 Feb 2025).

A central training device is stochastic depth scheduling. At each iteration, a level $T$8 is randomly selected and the contrastive loss is applied at that level, with a bias toward deeper levels. The paper contrasts this with constant depth, linear or exponential depth growth, and Matryoshka-style sums of losses across levels, and reports that stochastic depth performs best overall for coarse representations. Optimization uses AdamW with learning rate $T$9, $D$0k steps, $D$1k warmup steps, batch size $D$2, encoder input truncated to $D$3 tokens, tree depth $D$4, cross-attention split with $D$5 heads and head dimension $D$6, and a learned propagation module implemented as a 2-layer MLP with ReLU and dropout (Gupta et al., 11 Feb 2025).

Inference is explicitly coarse-to-fine. For a chosen level $D$7, each context is assigned a representation $D$8 over the $D$9 nodes at depth $x$0, and an index is built over these assignments. At query time, ReTreever computes $x$1, retrieves top-$x$2 contexts by nearest neighbors in assignment space using nTVD, and optionally refines by reranking with leaf-level assignments $x$3 or with cosine similarity in the original encoder space. The index size is therefore controlled by $x$4, while leaf-level retrieval uses $x$5 (Gupta et al., 11 Feb 2025).

The practical implication is a single trained retriever that can expose multiple operating points. Coarser levels provide smaller representations and lower-latency search; finer levels recover the highest available fidelity from the same hierarchy.

4. Empirical behavior, latency, and interpretability

The document-retrieval ReTreever is evaluated on Natural Questions, HotpotQA, TopiOCQA, and RepLiQA, using Recall@k and NDCG@k, primarily at $x$6 (Gupta et al., 11 Feb 2025). At leaf level, the paper reports that ReTreever generally preserves the encoder’s representational power. On NQ, ReTreever reaches NDCG@10 $x$7 and Recall@10 $x$8, compared with BGE at $x$9. On HotpotQA, BGE remains stronger at $T(x)\in[0,1]^{|T_L|}$0 versus ReTreever’s $T(x)\in[0,1]^{|T_L|}$1. On TopiOCQA, BGE also wins, with $T(x)\in[0,1]^{|T_L|}$2 against ReTreever’s $T(x)\in[0,1]^{|T_L|}$3. On RepLiQA, ReTreever improves NDCG@10 to $T(x)\in[0,1]^{|T_L|}$4 from BGE’s $T(x)\in[0,1]^{|T_L|}$5, while BGE is slightly higher in Recall@10, $T(x)\in[0,1]^{|T_L|}$6 versus $T(x)\in[0,1]^{|T_L|}$7.

The strongest coarse representations come from the stochastic-depth variant. The paper states that ReTreever-Stochastic delivers the strongest coarse embeddings up to size $T(x)\in[0,1]^{|T_L|}$8 on NQ and HotpotQA and remains competitive or favorable on finer levels. This supports the central claim that coarse-to-fine retrieval can be trained as a single hierarchy rather than as a collection of separately compressed models (Gupta et al., 11 Feb 2025).

Latency is a major empirical advantage relative to hierarchical baselines. On NQ test contexts, ReTreever reports $T(x)\in[0,1]^{|T_L|}$9 ms and NDCG@10 $h$0, compared with Hier-Kmeans at $h$1 ms and $h$2, Hier-GMM at $h$3 ms and $h$4, and RAPTOR at $h$5 ms and $h$6. On RepLiQA test contexts, ReTreever reports $h$7 ms and NDCG@10 $h$8, compared with Hier-Kmeans at $h$9 ms and $2^h$0, Hier-GMM at $2^h$1 ms and $2^h$2, and RAPTOR at $2^h$3 ms and $2^h$4. The paper attributes this partly to parallel split evaluation and the use of assignment-space retrieval rather than recursive traversal at query time (Gupta et al., 11 Feb 2025).

Interpretability is treated as a first-class property. The paper reports that node embeddings’ cosine similarity decreases with tree distance, while pairwise context cosine similarity increases with deeper lowest common ancestor depths. Topic modeling and keyword extraction over a subtree’s assigned contexts can be used to label nodes; one example reports that a subtree rooted at node $2^h$5 in an NQ depth-$2^h$6 model groups “media” contexts, with children specializing into “publishing” and “TV,” and a deeper path refining to “Television seasons.” The hierarchy therefore serves not only as an index but also as a semantic organization of the corpus (Gupta et al., 11 Feb 2025).

5. Cross-domain ReTreever formulations

In recommendation, the mapped ReTreever is DTR, or Deep Tree-based Retriever, which organizes items as leaves of a balanced tree and performs layer-wise beam search (Liu et al., 2024). DTR reframes training as level-wise softmax over tree nodes at the same level, replacing one-versus-all binary node training with explicit horizontal competition. It also rectifies non-leaf labels so that, in expectation, they align with the max-over-subtree probabilities needed by beam search, and uses sampled softmax with a tree-based sampling distribution. The paper states that, if rank consistency holds at all levels with the rectified targets, the model is Bayes optimal under beam search. Empirically, DTR(T-RL), combining tree-based sampling and rectified labels, is best overall across MovieLens-10M, MIND, Amazon Books, and Tmall Click; for F-measure@20 it improves over OTM from $2^h$7 to $2^h$8 on Movie, from $2^h$9 to $t$0 on MIND, from $t$1 to $t$2 on Amazon, and from $t$3 to $t$4 on Tmall (Liu et al., 2024).

In token-efficient inference, ReTreever is the architecture built on Tree Cross Attention (Feng et al., 2023). TCA organizes context tokens into a balanced tree, computes internal summaries bottom-up, and then uses a learned policy to descend top-down, adding siblings to the retrieved set. Cross attention is restricted to this $t$5 subset. The paper reports that on the Copy Task, TCA reaches approximately $t$6 accuracy while using approximately $t$7, $t$8, and $t$9 of tokens for $s_{\theta_t}(x)\in\mathbb{R}$0, respectively, whereas Perceiver IO with the same token budget reaches approximately $s_{\theta_t}(x)\in\mathbb{R}$1, $s_{\theta_t}(x)\in\mathbb{R}$2, and $s_{\theta_t}(x)\in\mathbb{R}$3. On Human Activity classification, ReTreever achieves $s_{\theta_t}(x)\in\mathbb{R}$4 accuracy with approximately $s_{\theta_t}(x)\in\mathbb{R}$5 of tokens, compared with Transformer+CA at $s_{\theta_t}(x)\in\mathbb{R}$6 using $s_{\theta_t}(x)\in\mathbb{R}$7 of tokens (Feng et al., 2023).

In syntax-guided question answering, TreeRare presents a retrieval-only “Tree-Retrieval” that can act as a general-purpose ReTreever module (Zhang et al., 31 May 2025). It parses a question into a dependency tree or constituency tree, traverses the tree bottom-up, and uses node spans as retrieval queries. The retrieval-only variant uses BM25 with top-$s_{\theta_t}(x)\in\mathbb{R}$8 passages per node and a cross-encoder reranker to select top-$s_{\theta_t}(x)\in\mathbb{R}$9 passages across each subtree. With a GPT-4o-mini backbone, Tree-Retrieval (DT) reaches AVG $E$00 on the multi-hop QA setting against BM25 at $E$01 and DPR at $E$02, while Tree-Retrieval (CT) reaches AmbigDoc Answer Recall $E$03 and Entity Recall $E$04, improving over BM25 at $E$05 (Zhang et al., 31 May 2025).

In repository-level code editing, the mapped ReTreever is a structure-aware dense retriever derived from CoRet (Fehr et al., 30 May 2025). The repository is decomposed into chunks such as functions, classes, and class methods; each chunk is prefixed with its relative file path and augmented with selected call-graph neighbors, especially downstream callees. Training is repository-level, with in-instance negatives and a maximum-likelihood loss over chunks from the same repository. On SWE-bench Verified, CoRet reaches chunk-level perfect-recall@20 of $E$06 and MRR $E$07, compared with CodeSage Small at $E$08 and $E$09. File-path information is a major contributor: removing file paths at inference reduces performance from @20 $E$10 and MRR $E$11 to @20 $E$12 and MRR $E$13 (Fehr et al., 30 May 2025).

A related graph-RAG line is T-Retriever, which reformulates attributed graph retrieval as tree-based retrieval using a semantic and structure-guided encoding tree (Wei et al., 8 Jan 2026). Its Adaptive Compression Encoding optimizes a tree of height $E$14 under a Semantic-Structural Entropy objective,

$E$15

thereby replacing rigid layer-specific compression quotas with global optimization. Internal nodes hold summaries of related subgraphs and leaves retain fine-grained units. The retrieval pipeline indexes all nodes uniformly, allowing multi-resolution retrieval. On SceneGraphs, WebQSP, and BookGraphs, the paper reports accuracy improvements over the best baseline of $E$16, $E$17, and $E$18, respectively (Wei et al., 8 Jan 2026).

Taken together, these systems show that the defining ReTreever move is not tied to one modality. It is the insertion of a meaningful tree between query and corpus, so that retrieval is performed by routing, aggregation, or constrained expansion over that hierarchy.

6. Limitations, failure modes, and open directions

The document-retrieval ReTreever inherits the usual trade-offs of learned hierarchies (Gupta et al., 11 Feb 2025). Misrouting at coarse levels can reduce recall for tail queries, highly compositional questions, or rare entities. Very deep trees can overfit final-level routing, while a fixed tree may route new distributions incorrectly under domain shift. The paper notes that collapse risk is handled by contrastive negatives, but inadequate negatives or skewed data can still bias routing, motivating monitoring of leaf usage distribution and continued training under distribution shift.

Other ReTreever-style systems expose additional structural failure modes. In DTR, the max-heap property can be violated in practice because of model error, label-estimation noise in $E$19, or dynamic catalogs, which can degrade beam search accuracy; the paper lists improved $E$20 estimation, adaptive beam widths, temperature-scaled softmax, dynamic tree learning, learned branching factors, and advanced sampling distributions as open directions (Liu et al., 2024). In TCA-based ReTreever, tree construction quality is critical: random organization degrades performance, and REINFORCE-based routing introduces training variance even though the resulting retrieval is $E$21 (Feng et al., 2023). In TreeRare, parser errors propagate through the syntax tree and constituency trees increase token usage and cost relative to dependency trees (Zhang et al., 31 May 2025). In the repository-level code setting, current implementation is Python-centric, and cross-repo dependencies, very large call graphs, and dynamic dispatch remain difficult (Fehr et al., 30 May 2025). In graph retrieval, performance depends on LM embeddings, the KDE bandwidth $E$22, the weighting parameter $E$23, and the height bound $E$24; very high $E$25 can overemphasize semantics, very low $E$26 can reduce semantic coherence, and extremely dense graphs complicate clean splits (Wei et al., 8 Jan 2026).

A broader implication is that ReTreever remains an active design space rather than a settled architecture. The recurrent research questions are how trees should be built, how routing should be supervised, how much structure should be exposed to the user, and how to preserve retrieval quality while exploiting coarse representations, beam-searchable indices, or logarithmic token selection. Across these formulations, the tree is not merely an acceleration device: it is also the main representational object through which cost, utility, and interpretability are jointly negotiated.