--- title: 'Document MAP (DMAP): Structural Framework' url: https://www.emergentmind.com/topics/document-map-dmap type: topic --- # Document MAP (DMAP): Structural Framework A Document MAP (DMAP) is a formal structural representation for multimodal documents that encodes hierarchical, relational, and spatial organization of content such as text, figures, tables, and charts. The DMAP paradigm explicitly models document structure in a human-aligned fashion, supporting evidence-grounded retrieval and reasoning for tasks such as multimodal document question answering (MMDocQA). By transcending flat, chunk-based document representations, DMAPs enable comprehensive navigation of logical, spatial, and semantic dependencies that are vital for complex information extraction and interpretability [2601.18203]. ## 1. Formal Definition and Core Representation A DMAP is defined by the tuple \[ \mathrm{DMAP} = (V, E, \phi, \psi, \mathcal{T}) \] where: - \(V\) is the set of atomic document elements, including page content (full-page screenshot and OCR text), figures, tables, and charts. If \(D\) consists of pages \(P_1, \dots, P_n\), each \(P_i\) has elements \(e_{i1}, \dots, e_{im}\), with \(V = \{ e_{ij} \mid 1 \leq i \leq n,\,1 \leq j \leq m_i \}\). - \(E \subseteq V \times V\) encodes relations among elements: hierarchical (section→page, page→element), cross-reference (e.g., “Table 3”→table), and figure–text alignment. \(E = E_{\mathrm{hier}}\cup E_{\mathrm{ref}}\cup E_{\mathrm{align}}\). - \(\phi: V \to \mathbb{R}^d\) is a semantic embedding function assigning each element \(e\) a text embedding \(\mathbf{v}^T_e\) (e.g., ColBERTv2) and a visual embedding \(\mathbf{v}^V_e\) (e.g., ColPali): \(\phi(e) = (\mathbf{v}^T_e, \mathbf{v}^V_e)\). - \(\psi: V \to \mathbb{R}^4\) is the layout mapping function, giving bounding-box coordinates \((x_{\min},y_{\min},x_{\max},y_{\max})\) for each element. - \(\mathcal{T}\) is a hierarchical tree organizing sections, pages, and elements: sections → pages → elements. This formalism provides comprehensive coverage of both logical (hierarchical, referential) and spatial relationships, mirroring human document navigation [2601.18203]. ## 2. Structured-Semantic Understanding Agent The Structured-Semantic Understanding Agent (SSUA) is responsible for constructing the DMAP from a raw multimodal document through three principal stages: 1. **Element Extraction**: The document is split into pages. For each page \(P_i\), atomic elements \(e_{i0}\) (page content) and \(e_{i1}, \ldots, e_{im}\) (figures, tables, charts) are extracted via OCR and detection tools (e.g., pdffigure2). Semantic and visual embeddings are computed. 2. **Hierarchical Population**: SSUA incrementally builds the section→page→element hierarchy. New section headings are detected using font/layout/keyword heuristics, and section numbering is maintained. Elements are linked to their enclosing page and section. 3. **Summary Tree Generation**: Summaries for each section and page are generated and stored for fast, structured retrieval. All section parsing is rule-based, using layout and OCR cues, without additional supervised losses. This approach encodes not only content and layout but also higher-level semantic organization, producing lightweight, navigable summaries for use in downstream retrieval and reasoning [2601.18203]. ## 3. Reflective Reasoning Agent and Retrieval Methodology The Reflective Reasoning Agent (RRA) answers queries using structure-aware, evidence-driven retrieval and generation over DMAP. Its operation proceeds as follows: ### 3.1 Tri-Path Retrieval RRA fuses three distinct retrieval signals: - **Structured Semantic Retrieval**: Leveraging the summary tree, top-k candidate sections/pages are selected semantically relevant to the query. - **Textual Feature Retrieval**: The query's text embedding is used to retrieve elements with high textual similarity. - **Visual Feature Retrieval**: The query's visual embedding is used for visual match retrieval. The union \(R = R^{(s)} \cup R^{(t)} \cup R^{(v)}\) yields the full retrieval set, integrating logical, visual, and textual context. ### 3.2 Iterative, Reflective Generation Leveraging features from retrieved elements, a multimodal generator produces a candidate answer. An LLM-based evaluator assesses answer sufficiency (completeness and consistency). If evidence is insufficient (\(\mathrm{done} = \mathtt{no}\)), the agent broadens retrieval via DMAP’s structure (e.g., parent/neighbor nodes) and regenerates the answer. Iteration continues until sufficiency is affirmed or a step budget is exhausted. This strategy prevents premature answer speculation, reduces hallucination, and supports negative acknowledgments when supporting evidence is absent [2601.18203]. ## 4. Empirical Evaluation and Ablation The DMAP framework was evaluated on five MMDocQA benchmarks: MMLongBench, LongDocURL, PaperTab, PaperText, and FetaTab. Performance was compared across large vision–language models (LVLMs), RAG-based systems, and DMAP-augmented variants. Results showed that DMAP with tri-path retrieval achieves the highest average accuracy (54.4%), outperforming both LVLMs (e.g., Qwen-2.5-VL-7B-Instruct at 27.4%) and standard RAG systems (e.g., MDocAgent at 48.4%). Ablation experiments quantified each retrieval path's contribution: | Variant | Text | Image | DMAP | AvgAcc | |-----------------|------|-------|------|---------| | w/o DMAP | ✓ | ✓ | ✗ | 44.7% | | w/o Image | ✓ | ✗ | ✓ | 52.6% | | w/o Text | ✗ | ✓ | ✓ | 39.4% | | Full (all paths)| ✓ | ✓ | ✓ | 54.4% | Eliminating DMAP causes a 9.7 percentage-point drop, highlighting structural retrieval’s unique value. Textual retrieval is the strongest single path but is outperformed by the full multimodal tri-path fusion [2601.18203]. Analysis by question type (MMLongBench) revealed DMAP’s particular strength for “Layout,” “Table,” “Chart,” and “Figure” evidence, with relative improvements over MDocAgent ranging from +16.8% (Tables) to +89.4% (Charts). ## 5. Advantages and Limitations ### Advantages - **Human-Aligned Representation**: DMAP’s hierarchical and relational schema (sections, pages, elements, and cross-references) preserves the document organization strategies humans naturally use. - **Tri-Path Retrieval**: Combining structured, textual, and visual paths yields higher retrieval precision and more consistent evidence aggregation. - **Reflective Generation**: Iterative answer refinement helps avoid unsupported answers and supports explicit abstention when evidence is lacking. ### Limitations - **Preprocessing Overhead**: Construction and indexing (summaries, embeddings, layout graphs) require substantial offline effort. - **Dependency on Heading Detection**: Reliable section parsing is prerequisite; noisy or nonstandard PDF layouts degrade performance. - **Rule-Based Heuristics**: Current SSUA uses rule-based section segmentation, which may not generalize to irregular document formats. ## 6. Future Directions Several research avenues have been identified: - **Learned Structural Parsing**: Replacing heuristic-based section segmentation and relation extraction with end-to-end trainable approaches. - **Incremental and Dynamic Mapping**: Supporting DMAP updates for streaming, evolving, or version-controlled documents. - **Cross-Document DMAPs**: Extending structures to collections (e.g., legal archives, research corpora) with cross-document links. - **Joint Training**: End-to-end optimization of SSUA and RRA, aligning structure construction directly with reasoning and QA objectives [2601.18203]. A plausible implication is that further automation and learning in DMAP construction and utilization may extend its applicability to a broader class of documents and more complex multilingual or cross-domain QA settings. ## 7. Relation to Broader Multimodal Document QA Paradigms DMAP addresses limitations inherent in flat, chunk-based or naïve RAG (retrieval-augmented generation) methods that ignore document structure. Other frameworks such as ARIAL have focused on modular tool orchestration and explicit spatial localization for document VQA, but do not construct document-level structural maps akin to DMAP [2511.18192]. The integration of structure-aware reasoning and evidence-driven answer generation in DMAP positions it as a pathway toward more trustworthy and explainable document AI systems. Source: https://www.emergentmind.com/topics/document-map-dmap