Hierarchical Trees in RAPTOR

• Hierarchical trees are structured models that recursively abstract and summarize data using techniques like GMM clustering and GPT-based summarization for efficient retrieval.
• In RAPTOR, documents are split into chunks, embedded with SBERT, and organized into a multi-level tree that enables precise, context-aware question answering.
• The HO-Tree in ST-Raptor extends these ideas to semi-structured tables by integrating meta and body trees to maintain structural integrity and improve answer accuracy.

Hierarchical trees in retrieval and question answering constitute a structured approach for organizing, abstracting, and querying complex information spaces, particularly in the RAPTOR framework for text documents (Sarthi et al., 2024) and the HO-Tree representation for semi-structured tables in ST-Raptor (Tang et al., 25 Aug 2025). These models utilize recursive or orthogonal tree structures to recursively abstract, summarize, and enable context-sensitive reasoning over large, heterogeneous corpora and tables, optimizing both accuracy and efficiency for retrieval-augmented LLMs.

1. RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval

The RAPTOR model applies hierarchical tree construction to lengthy documents, facilitating multi-range retrieval for LLMs. Let $D$ be a document of $T$ tokens. RAPTOR begins by segmenting $D$ into $N$ contiguous, sentence-respecting text “chunks” of at most 100 tokens. Each chunk $t_j$ is embedded using SBERT: $e_0(t_j)=\varphi(t_j)\in\mathbb{R}^d$, with $d=768$.

At each recursive level $\ell$, embeddings $e_{\ell}(n_{\ell,i})$ corresponding to nodes $n_{\ell,i}$ are clustered using a Gaussian Mixture Model (GMM). To address high-dimensionality, UMAP-based dimensionality reduction may optionally reduce $\mathbb{R}^d\to\mathbb{R}^{d'}$. The GMM fits $K$ Gaussians to maximize

$$L(\pi, \mu, \Sigma) = \sum_{i=1}^M \log \sum_{k=1}^K \pi_k \mathcal{N}(e_\ell(n_{\ell,i}); \mu_k, \Sigma_k),$$

with $K$ chosen to minimize BIC:

$BIC = \ln M\cdot |\theta| - 2\ln\hat{L},$

where $|\theta|=K[d+d(d+1)/2+1]$. Nodes are assigned soft cluster memberships via posteriors $\gamma_{i,k}$, with cluster sets $C_k = \{ n_{\ell,i} : \gamma_{i,k}\geq \tau \}$ for a threshold $\tau$.

Child texts in each cluster $C_k$ are concatenated and summarized using GPT-3.5-turbo, generating new summary nodes $s_{\ell+1,k}$. Each summary is embedded: $e_{\ell+1}(s_{\ell+1,k}) = \varphi(s_{\ell+1,k})$. This process recurses until a single root node (level $L$) remains or clusters become too small to split. The resulting hierarchical summary tree enables retrieval at multiple abstraction levels.

2. Formal Tree Structure, Inference, and Query Algorithms

Formally, the tree comprises $L$ levels indexed $\ell=0\ldots L$. Level-0 nodes correspond to original text chunks, with each higher level built from GMM-clustered, abstractive summaries. A node $n_{\ell+1,k}$ at level $\ell+1$ has children $C_k\subseteq\{n_{\ell,i}\}$.

Querying in RAPTOR proceeds by embedding the query $q$ to $q̂=\varphi(q)$. Each node is scored by cosine similarity:

$$\text{score}(n_{\ell,i}) = \cos(q̂, e_\ell(n_{\ell,i})) \equiv \frac{q̂\cdot e_\ell(n_{\ell,i})}{\|q̂\|\|e_\ell(n_{\ell,i})\|}.$$

Two retrieval modes are defined:

• Tree traversal: Starting at the root ($\ell=L$), top-$k$ nodes by score are selected, then recursively their children are scored at each lower level, selecting top-$k$ at each step. The context for the LLM is the union of texts from selected nodes across all levels.
• Collapsed-tree retrieval: All nodes across all levels are pooled; top nodes are chosen by score in descending order until a global token budget is reached.

Empirical evidence indicates collapsed-tree retrieval achieves higher answer accuracy, while tree traversal provides deterministic per-level quota and lower computational overhead when $kL\ll M$ (with $M$ the total number of tree nodes), especially in large document scenarios (Sarthi et al., 2024).

3. Computational Complexity and Trade-Offs

The RAPTOR build time consists of $O(N\cdot E)$ for chunk embedding (where $E$ is per-encoder cost), $O(N_\ell\log N_\ell)$ per-level for clustering, and a summarization overhead proportional to LLM-invocation token counts per level. Empirically, wall-clock and token costs scale linearly with document length $T$.

During retrieval:

• Flat-retrieval (baseline, e.g., DPR/BM25): $O(N)$ for scoring/sorting.
• Tree traversal: $O(L\cdot k\cdot\log(\text{child-count}))$, typically much less than $N$ for small $k$ and moderate $L$.
• Collapsed-tree retrieval: $O(M)$ or $O(M\log M)$ with $M<2N$ to score and sort all nodes.

FAISS or approximate $k$-NN can accelerate all approaches.

Key trade-offs are summarized in the following table:

Retrieval Mode	Granularity	Speed	Accuracy (Empirical)
Collapsed-tree	Flexible	Moderate	Highest
Tree traversal	Fixed per-level	Fast (when $kL\ll M$)	Lower (but scalable)

Collapsed-tree retrieval is most accurate; tree traversal is optimal when deterministic quotas and speed are required (Sarthi et al., 2024).

4. Hierarchical Trees for Semi-Structured Tables: HO-Tree in ST-Raptor

ST-Raptor generalizes hierarchical tree frameworks to semi-structured tables, formulating the Hierarchical Orthogonal Tree (HO-Tree) representation (Tang et al., 25 Aug 2025). For a table $T$, the HO-Tree is a triple

$$HO\text{-}Tree(T) = (M, B, \varphi: M.\mathsf{Leaves} \to B.\mathsf{Levels})$$

where:

• $M$: Meta-Tree, representing headers and their hierarchical containment.
• $B$: Body-Tree, representing content cells as paths (rows) in a row-oriented trie.
• $\varphi$: a pointer from each meta-tree leaf (resolved header) to a body-tree level.

This design encodes both hierarchical header structure and side-by-side orthogonal table sections, accommodating multi-row/column spans, arbitrary merged cells, and recursive subtables.

Algorithmically, HO-Tree construction proceeds by meta-information detection (via VLMs and embedding-based header identification), recursive table partitioning according to merged-cell and header orientation principles, and depth-first construction—producing a forest of HO-Trees under a synthetic root as needed. Each cell is processed $O(n)$ times (with $n$ the total number of cells), and embedding dominates the cost at $O(nd)$.

5. Operations and Pipelines over Hierarchical Trees

ST-Raptor exposes a formal language of atomic tree operations, composable into complex pipelines for LLM-guided question answering. The basic operations include:

• $\mathrm{CHL}(v)$: Child subtree retrieval for meta-node $v$.
• $\mathrm{FAT}(v)$: Ancestor subtree retrieval for meta-node $v$.
• $\mathrm{EXT}(m, b)$: Value extraction, returning all body-nodes at meta-node $m$’s associated level with row-ancestor value $b$.
• $\mathrm{Cond}(D, f)$: Data filtering by predicate $f$.
• $\mathrm{Math}(D, g)$: Numeric aggregation.
• $\mathrm{Cmp}(D_1, D_2, h)$: Set comparison.
• $\mathrm{Foreach}(D, u\mapsto u')$: Map operation.
• $\mathrm{Align}(P, HO\text{-}Tree)$: Parameter alignment.
• $\mathrm{Rea}(Q,D)$: LLM-based reasoning on data $D$ for query $Q$.

Given a natural language question $Q$, ST-Raptor (1) decomposes $Q$ into sub-questions using few-shot prompting and retrieved exemplars, (2) generates atomic operation statements for each, and (3) sequentially executes these over the HO-Tree, invoking forward and backward verification mechanisms to ensure correctness and stability of answers (Tang et al., 25 Aug 2025).

6. Empirical Results and Impact

Controlled experiments confirm the efficacy of the hierarchical tree approach. RAPTOR achieves state-of-the-art results on multi-step reasoning question answering tasks. Notably:

• On QuALITY (5k-token passages), DPR+GPT-4 yields 60.4% accuracy, RAPTOR+GPT-4 62.4% (+2.0 pp); on the QuALITY-HARD subset, performance improves from 54.7% to 56.6%.
• On QASPER, RAPTOR+GPT-4 obtains 55.7 F1 (DPR+GPT-4: 53.0 F1).
• On NarrativeQA, RAPTOR+UnifiedQA improves ROUGE-L, BLEU-1, and METEOR metrics by 1–0.7 pp over strong baselines.

Coupling RAPTOR retrieval with GPT-4 yields an absolute 20-point lift in QuALITY accuracy over the previous SOTA (62.3% to 82.6%), demonstrating RAPTOR’s utility for multi-step, thematic reasoning over long contexts (Sarthi et al., 2024).

ST-Raptor, leveraging the HO-Tree, attains up to 20% higher answer accuracy than nine other baselines in the semi-structured table setting, as measured on the SSTQA dataset with 764 questions across 102 real-world tables (Tang et al., 25 Aug 2025).

7. Significance and Generalizations

Hierarchical trees in RAPTOR and ST-Raptor provide an explicit, recursive abstraction of the underlying information space, decoupling summary granularity and query expressivity from fixed chunking strategies. This enables robust retrieval and reasoning, supporting complex question decomposition, abstraction, and compositional generalization. The models natively accommodate multi-modal, recursive, and thematic content, and their modular tree operations and layouts offer principled mechanisms for aligning LLM inference with the original data structure. This suggests that recursive trees may remain foundational for scalable, interpretable retrieval-augmented language modeling in both textual and tabular domains, particularly where multi-step reasoning and layout complexity are central.

Further developments may refine dynamic tree construction, LLM-guided pipeline generation, and cross-modal schema induction, leveraging the demonstrated empirical effectiveness and formal flexibility of hierarchical tree representations (Sarthi et al., 2024, Tang et al., 25 Aug 2025).