MoDora: LLM-Powered Tree-Based Document QA

• MoDora is an LLM-powered system that restructures OCR outputs into a hierarchical Component-Correlation Tree to capture layout, semantic grouping, and cross-modal evidence.
• It employs local-alignment aggregation and a question-type-aware retrieval strategy to effectively align interleaved text, tables, charts, and images for precise Q&A.
• Empirical evaluations show MoDora outperforms baselines by up to 61.07%, highlighting its innovative structured representation and hybrid reasoning approach.

Searching arXiv for the specified paper and closely related document-analysis context. MoDora is an LLM-powered system for question answering over semi-structured documents, introduced as a tree-based framework for documents that contain interleaved text, tables, charts, images, and supplementary elements arranged in irregular layouts (Xu et al., 26 Feb 2026). It is designed for settings in which plain-text representations and page-level visual representations are both inadequate, because they fail to preserve semantic grouping, hierarchical structure, layout distinctions, and cross-region evidence alignment. The system combines local-alignment aggregation for OCR-parsed elements, a Component-Correlation Tree (CCTree) for hierarchical organization, and a question-type-aware retrieval strategy that supports both semantic and locational reasoning. Experiments report that MoDora outperforms baselines by 5.97% to 61.07% in accuracy (Xu et al., 26 Feb 2026).

1. Problem setting and motivation

Semi-structured documents, in the formulation used by MoDora, contain heterogeneous elements such as paragraphs, tables, charts, images, and supplements including headers, footers, sidebars, and page numbers (Xu et al., 26 Feb 2026). Such documents occur in scientific reports, financial statements, technical manuals, magazines, and presentation slides, and they pose a distinct question-answering problem because the answer often depends not only on textual semantics but also on layout, hierarchy, and cross-modal alignment.

The paper formalizes document question answering as a mapping

$f: (\mathcal{D}, \mathcal{Q}) \mapsto \mathcal{A}$

where $\mathcal{D}$ is the document, $\mathcal{Q}$ the question, and $\mathcal{A}$ the answer (Xu et al., 26 Feb 2026). The answer is assumed to depend on a relevant subset of document components,

$$\mathcal{C}_{rel} = \{CO_{k_1}, CO_{k_2}, \dots, CO_{k_m} \mid CO_{k_i} \subseteq \mathcal{D}, CO_{k_i} is\ relevant\ to\ \mathcal{Q}\}.$$

This formulation emphasizes that answering is fundamentally a retrieval-and-alignment problem over structured document units rather than a purely sequential reading task.

The paper identifies three technical challenges. First, OCR-parsed elements are often fragmented and stripped of semantic context, so a title, its governed paragraphs, and an associated table may appear as disconnected units. Second, existing approaches lack effective representations for both hierarchical structure and layout-specific distinctions, such as differentiating sidebars from main content or linking tables to nested chapter titles. Third, many questions require retrieving and aligning information across multiple regions or pages, including cross-modal evidence such as a paragraph on one page and a table cell on another (Xu et al., 26 Feb 2026).

This motivates MoDora’s central design choice: documents should be represented neither as flat OCR text nor as undifferentiated whole-page images, but as layout-aware components organized into a tree structure that preserves hierarchical and inter-component relations (Xu et al., 26 Feb 2026).

2. System architecture and overall workflow

MoDora consists of three major stages: document preprocessing, tree construction, and tree-based document analysis (Xu et al., 26 Feb 2026). The first stage converts OCR outputs into enriched document components. The second stage organizes those components into the Component-Correlation Tree. The third stage performs question-type-aware retrieval and final answer generation.

The system is explicitly OCR-based. The paper states that this design supports digital-native PDFs, scanned PDFs, screenshots, and rasterized documents, and notes that around 20% of DUDE documents lack native PDF structure (Xu et al., 26 Feb 2026). OCR is therefore treated as a practical substrate rather than a fallback.

Multiple language and vision-LLMs are used in specialized roles. GPT-5 is used for hierarchy detection, question parsing, and final QA; Qwen2.5-VL-7B-Instruct is used locally for information enrichment, summarization, forward search, and backward verification; Qwen3-Embedding-8B is used for embeddings; and PaddleOCR is used for OCR (Xu et al., 26 Feb 2026). The framework is therefore not an end-to-end trained neural model but a structured system pipeline.

A plausible implication is that MoDora’s contribution lies primarily in representation, orchestration, and retrieval design rather than in proposing a new pretrained backbone. The paper indeed provides no end-to-end training or fine-tuning procedure for MoDora itself (Xu et al., 26 Feb 2026).

3. Component construction from OCR outputs

MoDora begins from OCR-parsed elements with type, text content when available, location information such as page indices and bounding boxes, and reading order (Xu et al., 26 Feb 2026). It then applies local-alignment aggregation to construct larger, semantically coherent components. A component is defined as

$CO=\{e_k^x\}, \quad CO\subseteq \mathcal{D}.$

The aggregation procedure is rule-based. For textual content, a title and the paragraphs until the next title are merged into one component: $$A_1=\{e^t_{title},e^t_{para,1},\dots,e^t_{para,n}\}.$$ For non-text elements such as tables, charts, and images, MoDora aggregates an optional nearby title with the non-text element: $$A_2=\{[e^t_{title}], e^x\} \cup \{e^x,[e^t_{title}]\}, \quad x\in\{b,c,i\}.$$ For supplementary elements, components are grouped by type per page: $A_3=\{e^s\}.$ These constructions preserve units such as “section title + body paragraphs” or “table/chart + caption,” which would otherwise be fragmented in OCR output (Xu et al., 26 Feb 2026).

For components with hierarchical titles, MoDora performs format-aware hierarchy relation detection. The title texts and cropped title images are provided to an MLLM, which predicts the hierarchical level of each title using numbers of # symbols (Xu et al., 26 Feb 2026). Because both title text and title appearance are used, the method incorporates semantics together with typographic layout cues. This suggests that title hierarchy is treated as a document-level latent structure that cannot be recovered reliably from text alone.

For non-text components, MoDora generates a triplet representation

$$\mathcal{T}(CO)=\big(title(CO), metadata(CO), data(CO)\big), \quad CO\subset A_2.$$

The prompt is type-specific: charts emphasize axes, labels, and trends; tables emphasize schema and representative cell values; images are also covered, though the paper gives fewer details (Xu et al., 26 Feb 2026). The goal is not merely captioning but structure-aware serialization of non-text content into forms suitable for retrieval and reasoning.

The paper reports only about 3% hallucinated enriched outputs in manual analysis, and states that generation is skipped when an existing matching caption is found by heuristics (Xu et al., 26 Feb 2026). This indicates that enrichment is treated as a controlled augmentation step rather than unrestricted generation.

4. The Component-Correlation Tree

The Component-Correlation Tree, or CCTree, is the central structural representation in MoDora (Xu et al., 26 Feb 2026). Each node $\mathcal{D}$0 corresponds to one document component and stores component information, a unique node index $\mathcal{D}$1, and child nodes $\mathcal{D}$2. Edges $\mathcal{D}$3 encode three kinds of relation.

Text-to-text relations connect text components through title hierarchy: $\mathcal{D}$4 Text-to-other relations connect a text component to nearby non-text components: $\mathcal{D}$5 Supplementary relations connect supplementary components in separate branches: $\mathcal{D}$6

This organization preserves several distinctions that are frequently collapsed in flat retrieval systems. Nested sections become explicit parent-child chains. Tables, charts, and images are attached under governing text sections rather than treated as isolated blocks. Sidebars, headers, footers, and page numbers are isolated from the main content hierarchy rather than mixed into it (Xu et al., 26 Feb 2026).

The tree is constructed in two stages. The overall algorithm first splits components into main-body components $\mathcal{D}$7 and supplement components $\mathcal{D}$8, creates a root $\mathcal{D}$9, and attaches a main branch and a supplementary branch (Xu et al., 26 Feb 2026). Main-body construction is stack-based: text nodes are attached according to title levels, while non-text nodes are attached as children of the current text node but are not pushed onto the stack. This yields a hierarchy in which text structure drives the backbone of the tree and associated non-text content attaches to the active textual scope.

A notable feature is bottom-up cascade summarization. Leaf nodes receive metadata consisting of several nominal phrases. Non-leaf metadata is generated from both the node’s own content and the metadata of its children (Xu et al., 26 Feb 2026). The paper provides the following attenuation formula:

$\mathcal{Q}$0

where $\mathcal{Q}$1 is the maximum depth of the tree including node $\mathcal{Q}$2, $\mathcal{Q}$3 is the depth of node $\mathcal{Q}$4, $\mathcal{Q}$5 is the number of summaries for node $\mathcal{Q}$6, $\mathcal{Q}$7 is the total number of summaries generated by the children of node $\mathcal{Q}$8, and $\mathcal{Q}$9 controls information growth rate (Xu et al., 26 Feb 2026). The stated effect is controlled compression of child information as it propagates upward. With $\mathcal{A}$0, the paper notes that when a parent aggregates 12 child summaries, it may generate 7 summaries after attenuation (Xu et al., 26 Feb 2026).

This suggests that CCTree is not merely a static hierarchy but a retrieval-oriented abstraction: higher nodes summarize subtrees compactly enough to support pruning while retaining enough semantic signal to avoid losing cross-region evidence.

5. Question-type-aware retrieval

MoDora’s retrieval stage is explicitly conditioned on question type (Xu et al., 26 Feb 2026). The paper lists representative categories including Text-Only Questions, Hybrid Data Questions, Structural Hierarchy Questions, and Location-Based Questions. Question parsing is performed with GPT-5. Retrieval then proceeds differently depending on whether the question contains locational cues, semantic cues, or both.

For location-based retrieval, each page is partitioned into a $\mathcal{A}$1 grid (Xu et al., 26 Feb 2026). Locational phrases in the question are mapped heuristically to grid cells. For example, “bottom of the page” maps to $\mathcal{A}$2. Nodes whose stored locations overlap the target region are retrieved. No ranking formula is given; the mechanism is overlap-based. This directly addresses questions such as those involving “top of page 1” or “bottom-right of page 1,” where semantic similarity alone is insufficient.

For semantic retrieval, MoDora performs iterative tree traversal using two alternating steps: forward search and backward verification (Xu et al., 26 Feb 2026). Nodes are represented as $\mathcal{A}$3 and $\mathcal{A}$4 pairs, where the path is the sequence of node indices from root to node. The retrieval state machine includes Initial State $\mathcal{A}$5, Checking State $\mathcal{A}$6, Fallback State $\mathcal{A}$7, Confirmed State $\mathcal{A}$8, and Excluded State $\mathcal{A}$9.

In forward search, the LLM receives the question together with each child’s index and metadata, and selects relevant child indices (Xu et al., 26 Feb 2026). This constitutes LLM-guided pruning. Selected children enter checking state. Unselected subtrees are not simply discarded; instead, they enter a fallback process. Each such subtree is flattened and chunked, embeddings are computed, and the top-$$\mathcal{C}_{rel} = \{CO_{k_1}, CO_{k_2}, \dots, CO_{k_m} \mid CO_{k_i} \subseteq \mathcal{D}, CO_{k_i} is\ relevant\ to\ \mathcal{Q}\}.$$0 relevant chunks are retrieved and merged into a fusion node with no children (Xu et al., 26 Feb 2026). The implementation retrieves 3 chunks or pages via embedding search (Xu et al., 26 Feb 2026).

Backward verification then evaluates candidate nodes in detail. For real tree nodes, the MLLM receives both textual content and a cropped document region based on node location; for fusion nodes, only textual content is used (Xu et al., 26 Feb 2026). Relevant nodes are marked confirmed and emitted as evidence; irrelevant ones are excluded. Manual analysis reports that the selector plus embedding search recalled all required evidence at 91% rate, while the verifier filtered 84% of irrelevant nodes but excluded 12% of necessary evidence (Xu et al., 26 Feb 2026).

This retrieval design reflects a hybrid philosophy. LLM-guided traversal provides structure-sensitive pruning; embeddings provide recall insurance when summarization or pruning misses details; verification restores precision. A plausible implication is that MoDora treats retrieval not as a single ranking operation but as a staged search process over a structured hypothesis space.

6. Answer generation and evidence fusion

After retrieval, MoDora constructs the final QA input from three evidence channels: textual evidence, locational evidence, and hierarchical information (Xu et al., 26 Feb 2026). Textual evidence is represented as $$\mathcal{C}_{rel} = \{CO_{k_1}, CO_{k_2}, \dots, CO_{k_m} \mid CO_{k_i} \subseteq \mathcal{D}, CO_{k_i} is\ relevant\ to\ \mathcal{Q}\}.$$1 pairs, preserving both content and structural provenance. Locational evidence consists of cropped document regions corresponding to retrieved nodes, thereby preserving layout, table structure, chart appearance, and typographic cues that may not survive text serialization. Hierarchical information is provided through the index schema of the CCTree.

The final answer is then generated by prompting an MLLM with the original question plus these evidence forms (Xu et al., 26 Feb 2026). The paper does not specify the exact prompt template. It also states a fallback condition: if retrieval results are empty or deemed irrelevant, all nodes in the CCTree are used to ensure completeness (Xu et al., 26 Feb 2026).

This suggests that MoDora is best described as a retrieval-augmented generative QA system whose defining feature is not the generation model itself but the structured evidence construction pipeline.

7. Empirical evaluation

MoDora is evaluated on seven benchmarks, with emphasis on DUDE, MMDA, MP-DocVQA, and M3DocVQA, and additional evaluation on Civic, Qasper, and Contract (Xu et al., 26 Feb 2026). The MMDA benchmark is newly introduced and contains 537 documents and 1,065 QA pairs, with question categories including 444 text-based, 250 hybrid, 159 hierarchy-related, 151 location-based, and 61 formatted-text questions (Xu et al., 26 Feb 2026).

The paper also reports document structural statistics. MMDA is the most structurally complex among the reported datasets, with 49.2 nodes, 37.7 leaves, and 4.23 levels on average, compared with 17.2 nodes, 12.6 leaves, and 3.36 levels for DUDE (Xu et al., 26 Feb 2026). This supports the claim that MMDA was designed to stress hierarchical and layout-rich reasoning.

The headline result is that MoDora outperforms baselines by 5.97% to 61.07% in accuracy (Xu et al., 26 Feb 2026). The paper further reports AIC-Acc improvements over the best baseline of +9.00% on DUDE, +8.46% on M3DocVQA, +5.97% on MP-DocVQA, and +16.24% on MMDA (Xu et al., 26 Feb 2026). On DUDE, MoDora achieves 83.70 AIC-Acc and 77.24 ACNLS, compared with 25.06 AIC-Acc and 23.43 ACNLS for LayoutLMv3 (Xu et al., 26 Feb 2026). On MMDA, it achieves 73.33 AIC-Acc and 55.23 ACNLS, compared with 17.37 and 13.96 for LayoutLMv3 and 57.09 and 37.10 for DocAgent (Xu et al., 26 Feb 2026).

Performance by question type on MMDA shows especially large gains for structural and hybrid reasoning. MoDora scores 76.73 on hierarchy questions, 79.95 on text, 68.00 on hybrid, and 68.21 on location, outperforming GPT-5, M3DocRAG, ZenDB, DocAgent, and other baselines across all categories (Xu et al., 26 Feb 2026). This distribution is consistent with the system’s design: CCTree primarily targets hierarchy, local-alignment aggregation and non-text enrichment target hybrid questions, and grid-based retrieval targets location questions.

8. Ablations, interpretation, and limitations

The ablation study provides the strongest evidence for which components matter most (Xu et al., 26 Feb 2026). Removing component construction causes the largest drop: from 83.70 to 55.96 on DUDE and from 73.33 to 36.24 on MMDA. Removing tree structure reduces performance to 68.37 on DUDE and 57.84 on MMDA. Removing textual evidence or locational evidence also causes substantial degradation, and removing forward search lowers performance to 78.35 on DUDE and 70.89 on MMDA (Xu et al., 26 Feb 2026).

These results support a clear interpretation. Component construction is foundational, because raw OCR elements are too fragmented for effective retrieval. Tree structure is critical, because flat representations cannot preserve title scope and component association. Forward search contributes less than those two but still matters, which suggests that structured pruning is useful even when embedding fallback is present.

The paper does not provide a standalone ablation for non-text triplet extraction or for bottom-up cascade summarization alone (Xu et al., 26 Feb 2026). Therefore, any claim that one is quantitatively more important than the other would be inferential rather than explicit. What can be said is that hybrid-question performance and the overall CCTree retrieval gains are consistent with these mechanisms playing a substantial role.

The paper also discusses limitations. OCR errors may misclassify titles as plain text; hierarchy detection can assign incorrect levels; enrichment hallucinations occur in about 3% of cases; the backward verifier excludes 12% of necessary evidence in manual analysis; and final answer generation may still fail even with correct evidence (Xu et al., 26 Feb 2026). The authors also note that the current OCR-based setup does not fully exploit native PDF structure and suggest future extensions to multi-document forests and more dynamic retrieval with backtracking or skipping (Xu et al., 26 Feb 2026).

A common misconception would be to treat MoDora as simply another OCR-RAG pipeline. The paper argues against that reading: the central novelty is not OCR plus retrieval, but the combination of component-level restructuring, tree-based relational encoding, and question-type-aware search over that structure (Xu et al., 26 Feb 2026). Another misconception would be to view it as a generic multimodal document VQA model; in fact, it is a structured analysis system whose main advantage derives from explicit representation of document hierarchy and layout.

9. Position and practical significance

MoDora occupies a distinct position between flat OCR-based RAG pipelines and page-level multimodal QA systems (Xu et al., 26 Feb 2026). Compared with flat chunking, it preserves better units of analysis, title scope, text-table association, and page-coordinate information. Compared with page-level multimodal QA, it retrieves finer evidence, separates supplementary from main content, and models hierarchy explicitly.

The system is most appropriate for documents that are semi-structured rather than purely textual, especially when answers depend on mixed modalities, cross-page evidence, structural hierarchy, or locational references (Xu et al., 26 Feb 2026). The paper explicitly notes that for ordinary text QA, MoDora may be excessive. This suggests a division of labor: the system targets the long tail of real-world PDFs and reports where visual organization is part of the semantics.

MoDora also introduces a cost–accuracy tradeoff characteristic of system-level pipelines. With local-model offloading, the paper reports an average cost of \$\mathcal{C}_{rel} = \{CO_{k_1}, CO_{k_2}, \dots, CO_{k_m} \mid CO_{k_i} \subseteq \mathcal{D}, CO_{k_i} is\ relevant\ to\ \mathcal{Q}\}.$20.01 for direct GPT-5 and \$0.09 for DocAgent (Xu et al., 26 Feb 2026). On MMDA, token usage totals 7,023k prompt tokens and 1,688k completion tokens, and the system uses 1 preprocessing API call, 0 tree-construction API calls, and 4 analysis API calls per query (Xu et al., 26 Feb 2026). This suggests that the method is more expensive than naïve prompting but cheaper than more heavily agentic alternatives while delivering materially better accuracy on structure-heavy tasks.

In summary, MoDora is a tree-based semi-structured document analysis system whose defining contribution is to recast document QA as structured retrieval over layout-aware components organized in a Component-Correlation Tree (Xu et al., 26 Feb 2026). Its technical core lies in local-alignment aggregation, type-specific enrichment, hierarchical tree construction, bottom-up cascade summarization, and hybrid location/semantic retrieval. The empirical evidence indicates that this representation is especially effective for hierarchy questions, hybrid text-and-table/chart questions, and complex multi-page documents, making MoDora a notable systems contribution in document intelligence research.