MultiDocFusion: Hierarchy-Aware Multimodal Chunking

• MultiDocFusion is a hierarchy-aware multimodal chunking pipeline that reconstructs visual and logical document structures to enhance retrieval precision by 8–15%.
• It integrates document region detection, OCR, and LLM-based hierarchical parsing to produce context-rich, readable chunks aligned with document organization.
• The pipeline employs DFS-based chunk construction to preserve parent–child relations, leading to a 2–3% improvement in ANLS QA scores over baseline methods.

MultiDocFusion is a multimodal, hierarchy-aware chunking pipeline for retrieval-augmented generation over long industrial documents. Rather than segmenting documents solely by token length or shallow semantic similarity, it reconstructs visual layout and logical section hierarchy, then forms chunks that preserve reading order and parent–child context. The pipeline integrates document region detection, OCR, LLM-based document section hierarchical parsing, and DFS-based hierarchical chunk construction; across industrial benchmarks, it is reported to improve retrieval precision by 8–15% and ANLS QA scores by 2–3% relative to baselines, with the central claim that explicit document hierarchy materially improves multimodal document QA (Shin et al., 14 Apr 2026).

1. Problem formulation and motivation

MultiDocFusion addresses a specific failure mode in long-document RAG: industrial documents are structured artifacts rather than flat text sequences. The target corpus includes PDFs, scans, technical reports, contracts, manuals, forms, and financial statements. These documents contain titles, section headers, nested subsections, tables, figures, page-dependent layout cues, and OCR noise. In this setting, conventional chunking can separate a header from its content, mix the end of one section with the start of another, or detach a table from the paragraph that interprets it (Shin et al., 14 Apr 2026).

The paper distinguishes four baseline chunking families. Length chunking uses a fixed token budget; semantic chunking groups semantically similar sentences; LLM-based chunkers such as LumberChunker or Perplexity chunking improve over fixed-length segmentation but remain largely text-sequence-centric; structure-based chunking uses detected layout segments but, without explicit hierarchy, still cannot represent nested organization. The argument is that these approaches systematically miss three properties that are decisive in industrial documents: layout, hierarchy, and OCR noise. Layout affects how tables, figures, captions, sidebars, and multi-column regions should be grouped. Hierarchy encodes semantic relations such as 1, 1.1, and 1.1.1, which are not reducible to local sequence adjacency. OCR noise makes text-only similarity signals brittle, especially in scanned or low-quality documents (Shin et al., 14 Apr 2026).

This framing positions MultiDocFusion as a hierarchy-first alternative. Its goal is not merely better segmentation granularity, but recovery of the document’s latent structural units so that retrieval operates over semantically coherent evidence rather than arbitrary spans. A plausible implication is that the method is best understood as a preprocessing and indexing strategy for multimodal document corpora, rather than as a standalone QA model.

2. Pipeline architecture

The pipeline comprises four stages: vision-based document region detection, OCR-based text extraction, DSHP-LLM document section hierarchical parsing, and DFS-based hierarchical chunk construction (Shin et al., 14 Apr 2026).

The first stage, Document Parsing (DP), detects page regions such as title, section headers, text blocks, tables, and figures. Each region is assigned a segment ID, segment type, bounding box, page number, and spatial coordinates, producing a page-level Layout Structure. The paper states that this stage uses object-detection-style models such as DETR, DiT, and VGT, fine-tuned on DocLayNet. The second stage applies OCR to each detected region. The cropped segment image is sent to OCR, the recognized text is attached back to the corresponding bounding box, and the result is an Annotated Layout combining geometry with text. OCR engines used include Tesseract, EasyOCR, and TrOCR (Shin et al., 14 Apr 2026).

The third stage is the central architectural novelty: DSHP-LLM, or Document Section Hierarchical Parsing with LLM. It consumes a Header List extracted from the annotated layout and reconstructs a Document Hierarchical Tree with explicit parent–child relations. The fourth stage performs depth-first traversal over that tree to generate hierarchical chunks. These chunks preserve a header with its descendant content, keep subsection context attached, enforce a token budget, and remain readable through Markdown-style heading markers (Shin et al., 14 Apr 2026).

The significance of this design lies in the coupling between visual structure and logical structure. DP and OCR provide spatially anchored textual segments, while DSHP-LLM and DFS grouping convert those segments into retrievable units aligned with document organization. The resulting chunks are therefore neither purely layout-derived nor purely text-derived; they are multimodal in the sense that geometry, reading order, and section semantics are fused before indexing.

3. Document hierarchical tree construction

The tree representation uses several node types: Root, Title, Section Headers, and General Nodes, where general nodes include text blocks, tables, and figures. In the paper’s figure, Root and Title are colored yellow, Section Headers red, and general nodes green. The hierarchical structure can take forms such as Root → Title → Section 1 → Section 1.1 → Section 1.1.1, making explicit the nested parent–child relations that conventional chunking usually ignores (Shin et al., 14 Apr 2026).

DSHP-LLM reconstructs this hierarchy from the header list. The prompt instructs the model to infer parentage from numbering when available—for example, 3.1 as a child of 3, and 3.1.1 as a child of 3.1—and otherwise to infer hierarchy from textual context. Top-level headings receive parent = null. The output format is JSON with fields of the form:

[ {"id": "<id from the original header_list>", "parent": "<id of the parent node or null if root>"} ]

The paper also states that general-purpose LLMs such as GPT-4 are not sufficient out of the box for this task; fine-tuning is required for reliable document-specific section-structure inference. To make training efficient, DSHP-LLM is instruction-tuned on hierarchical document datasets using LoRA/QLoRA. The evaluated backbones include Llama-3.2-3B, Qwen-2.5-3B, Mistral-8B, and Qwen-2.5-7B, with fine-tuned Mistral-8B selected as the backbone for the final pipeline (Shin et al., 14 Apr 2026).

After header hierarchy is built, the system attaches general nodes by traversing the header tree while scanning the spatially sorted segment list. Any general segment encountered before the next header is attached as a child of the current header node. This mechanism aligns logical structure from headers with reading order and spatial order from the layout parser. The result is therefore not merely a section tree but a full document structure tree incorporating textual, tabular, and figure content.

4. Hierarchical chunking and formalization

Chunk construction uses DFS over the hierarchical tree. A virtual node named FAKE_ROOT points to the actual root, and traversal begins there. Along each root-to-leaf path, the algorithm accumulates the current node’s text with inherited parent context. If the accumulated text exceeds the budget, it is split; otherwise, it is appended to the chunk list. The split condition is written as

$\text{length}(temp) > max\_len,$

with max_len = 550 tokens in the implementation (Shin et al., 14 Apr 2026).

Conceptually, this means that chunk content is path-conditioned: parent context is carried into descendants rather than discarded at section boundaries. The paper explicitly notes that this yields more chunks than standard methods because context is duplicated across descendants, but that duplication is deliberate. It keeps local context intact, preserves semantic anchors supplied by parent headers, and reduces fragmentation during retrieval. A plausible implication is that the indexing granularity is intentionally redundant in order to improve evidence coherence at retrieval time.

The resulting chunks are readable as Markdown-style hierarchical text, for example through nested heading markers such as document title, section, and subsection headings followed by the associated body text. In retrieval experiments, the system indexes all test documents jointly and retrieves top-$k$ chunks from the whole corpus, with the default $k = 4$. This corpus-level setting is important because it avoids assuming that the relevant document is already known, making the benchmark closer to realistic industrial retrieval (Shin et al., 14 Apr 2026).

5. Experimental setting and reported results

The hierarchy-parsing component is trained and evaluated on DocHieNet and HRDH. DocHieNet contains 1,673 documents drawn from mixed reports, papers, and industrial documents, with many scanned images. HRDH contains about 1,500 academic PDFs with complex layouts and many hierarchical labels. RAG and multi-page VQA evaluation uses DUDE, MPVQA, CUAD, and MOAMOB. DUDE contains 3,000+ documents and 7,000+ QA pairs across financial reports, manuals, and mixed domains. MPVQA contains about 17,000 documents and 48,000+ questions. CUAD contains 510 legal contracts and 13,000+ QA pairs. MOAMOB contains 2 industrial technical documents and 71 QA pairs, described as very challenging, nuclear-domain-style documents (Shin et al., 14 Apr 2026).

Evaluation uses hierarchy-parsing metrics of Accuracy, F1, and TEDS; retrieval metrics of Precision, Recall, and nDCG; and QA metrics of ANLS, ROUGE-L, and METEOR. The implementation uses a single NVIDIA A100 40GB GPU, an Intel Xeon 32-core CPU, 256GB RAM, PyTorch 2.0 with Transformers, DSHP-LLM training for 5 epochs with batch size 16 and learning rate $1\times 10^{-5}$, LoRA plus 4-bit quantization, BM25 parameters $k_1 = 1.2$ and $b = 0.75$, and maximum chunk length of 550 tokens (Shin et al., 14 Apr 2026).

On DocHieNet and HRDH, fine-tuning substantially improves hierarchical parsing over general-purpose LLMs. Reported TEDS gains include +16.71% for Mistral-8B + DSHP-LLM and +20.85% for Llama-3.2-3B + DSHP-LLM on DocHieNet, and +52.25% for Mistral-8B + DSHP-LLM and +49.24% for Qwen-2.5-3B + DSHP-LLM on HRDH. In retrieval, the headline claim is an 8–15% precision improvement relative to baselines. Reported average retrieval precision is 0.2001 on DUDE, 0.1759 on MPVQA, 0.8651 on CUAD, and 0.6184 on MOAMOB. The paper further states that, compared with structure-based chunking on DUDE, MultiDocFusion improves Recall by 7.08% and Precision by 5.51% (Shin et al., 14 Apr 2026).

For QA, the reported overall gain is 2–3% in ANLS. Main-table ANLS values are 0.1859 for DUDE, 0.1615 for MPVQA, 0.2738 for CUAD, and 0.2596 for MOAMOB. On official test splits where retrieval metrics are hidden, the paper reports DUDE ANLS of 0.1793 and MPVQA ANLS of 0.1544, again exceeding baseline chunking methods. The repeated pattern in these results is that the largest gains appear when explicit hierarchy is introduced on top of layout structure, rather than from layout segmentation alone (Shin et al., 14 Apr 2026).

6. Interpretation, misconceptions, and relation to adjacent research

A recurrent misconception in long-document RAG is that structure-aware chunking is equivalent to hierarchy-aware chunking. MultiDocFusion rejects that equivalence. Structure-based chunking without DSHP-LLM uses detected layout segments but does not encode parent–child relations among sections; the paper identifies the addition of explicit hierarchical parsing as the decisive improvement. Another misconception is that a strong general-purpose LLM can infer document hierarchy without task-specific adaptation. The reported weakness of GPT-4 without fine-tuning, especially on HRDH, is used as counter-evidence (Shin et al., 14 Apr 2026).

The robustness studies indicate that the advantage is not tied to one upstream component. MultiDocFusion remains best across different DP models, OCR models, and embedding models, which the paper interprets as evidence that the chunking strategy itself is the source of improvement. The practical scope described in the paper extends beyond the named benchmarks to enterprise PDFs, scanned reports, multi-page manuals, legal contracts, technical standards, and corpora with many interlinked documents. The authors also note that the induced hierarchy could support GraphRAG-like retrieval in the future because the output is already a typed document graph with parent–child and reading-order relations (Shin et al., 14 Apr 2026).

The method is adjacent to, but distinct from, document parsing systems such as DocFusion, which is a unified generative model for Document Layout Analysis, Mathematical Expression Recognition, Table Recognition, and OCR (Chai et al., 2024). MultiDocFusion operates at a later stage of the document-processing stack: it assumes region detection and OCR outputs, then reconstructs section hierarchy and builds retrieval chunks. This suggests a broader research trend in document intelligence toward unification at multiple layers—parsing, structure induction, and retrieval preparation—while keeping the technical meaning of “fusion” task-specific across those layers.