Hierarchical Document Representation

• Hierarchical document representation is a method that structures documents into tokens, sentences, paragraphs, and sections to capture both local details and global context.
• It leverages recursive aggregation, multi-level self-attention, and pooling techniques to efficiently process long, complex documents.
• Applications include improved document classification, extractive summarization, and multimodal analysis by exploiting inherent hierarchical structures.

Hierarchical document representation refers to a family of representation learning techniques, architectures, and algorithms that explicitly encode a document’s multi-level structure—such as tokens, sentences, paragraphs, sections, and entire documents—into compositional, often recursive or multi-stage, vectorial or graph-based forms. These representations exploit the innate linguistic, logical, and/or layout hierarchy of documents to facilitate downstream tasks such as classification, summarization, retrieval, structure recovery, and multimodal understanding. Hierarchical models offer significant computational and inductive-bias advantages over “flat” architectures, especially for long or structured documents.

1. Foundations and Rationale

Hierarchical representations are motivated by the observation that real-world documents—across domains such as scientific papers, legal contracts, news articles, and technical reports—are not simply long unstructured text streams but are organized in multi-level semantic, rhetorical, and/or physical hierarchies. At a minimum, this implies sentences grouped into paragraphs, paragraphs into sections, and sections into the complete document. Furthermore, multimodal documents may interleave text with figures, tables, or listings, and logical relationships such as references, reading order, and parent–child sectioning.

The core principle is to recursively aggregate and propagate information across hierarchical levels, enabling models to capture both fine-grained local context and global cross-span dependencies. This contrasts with “flat” architectures (e.g., standard BERT, CNNs, RNNs) that treat the document as a single monolithic sequence, which is computationally inefficient and often sub-optimal for long-context modeling (Abreu et al., 2019, He et al., 2024).

2. Neural Hierarchical Architectures

Neural hierarchical document representation approaches decompose modeling into multiple levels, often aligned to linguistic boundaries:

• Word/Sentence/Document Hierarchy: Staging encoders at distinct abstraction levels, such as word-level (CNN or RNN), sentence-level (GRU, LSTM, or Transformer), and document-level (attention, classification, or pooling), leveraging each for increasing receptive field and abstraction (Abreu et al., 2019, Al-Sabahi et al., 2018, Luo et al., 2019).
• Tree-Structured Models: Explicitly leveraging the document’s syntactic or typographic parse tree using recursive neural networks (such as Structure Tree-LSTM), with attention mechanisms over parent–child or sibling nodes and blockwise merges (Mrini et al., 2019).
• Multi-level Self-Attention and Pooling: Composing local representations by self-attention (often at word and sentence levels), then aggregating via global attention or pooling for final document embeddings (Al-Sabahi et al., 2018).
• Transformer-based Hierarchies: Two-stage or multi-stage Transformer models, with word-level/context-level Transformer blocks followed by sentence-level/document-level Transformer blocks (e.g., HIBERT, HDT, HMT) (Zhang et al., 2019, He et al., 2024, Liu et al., 2024).
• Hierarchical Metric Learning: Non-parametric modeling of documents as mixtures over topics or latent semantic blocks, and employing multi-stage optimal transport metrics (e.g., the Hierarchical Optimal Transport—HOTT—distance) (Yurochkin et al., 2019).

A selection of concrete architectures and key properties is presented below.

Model	Key Architectural Elements	Supported Tasks
HAHNN	Word-level CNN → BiGRU + Attention → Sent-level BiGRU + Attn	Classification
HSSAS	Word BiLSTM + Attention → Sent BiLSTM + Attention	Extractive Summarization
HIN-SR	Char-level BERT → Segment-GRU → Attn pool → Softmax	Sentiment Analysis
Structure Tree-LSTM	Recursive LSTM over doc tree + hierarchical attention	Classification, Interpretability
HDT	Hierarchical sparse Transformer, section/sent anchors	Long-context processing
HMT	Hierarchical multimodal Transformers with cross-level fusion	Multimodal doc classification

3. Mathematical Formalisms and Algorithmic Paradigms

Hierarchical encoders typically compose blockwise representations bottom-up and optionally top-down, using parametric or non-parametric methods.

• Recurrent/Recursive Aggregation: At each non-leaf node, merge child vectors using parametric (e.g., LSTM/GRU gates, attention-weighted sums) or structural operations. For example, Structure Tree-LSTM computes

$$m_j = \sum_{k\in C(j)} \alpha_{jk} h_k, \ c_j = i_j \odot \tilde c_j + \sum_{k\in C(j)} f_{jk}\odot c_k, \ h_j = o_j \odot \tanh(c_j),$$

with attention weights $\alpha_{jk}$ parameterized via level-specific networks (Mrini et al., 2019).

• Attention Mechanisms: Self- and cross-attention at multiple levels, e.g., sentence-level attention for distillation into document embeddings, or attention over segment–summary pairs as in HIN (Wei et al., 2020).
• Hierarchical Pooling: Averaging, max, or learned pooling aggregations at each level, optionally with nonlinearity and adaptation layers (Guo et al., 2019).
• Hierarchical Metric Learning: Two-level OT, where a document is a mixture over topics, topics are mixtures over words, and topic/word distances are used to define a Wasserstein metric on documents (Yurochkin et al., 2019).

For Transformer-based hierarchies, architectures use auxiliary structural tokens (e.g., [SEC], [SENT], [DOC]) and sample-dependent sparse attention masks:

$$M^{\mathrm{SENT}}_{ij} = [p^1_i=p^1_j][p^2_i=p^2_j],$$

which restricts attention to tokens in the same sentence/section combination, yielding nearly linear complexity for long documents (He et al., 2024).

4. Structural Reconstruction and Multimodal Hierarchies

Hierarchical representation is not limited to text but extends to multimodal documents and structure recovery:

• Document Structure Analysis (HDSA): Modeling documents (PDF, scanned, or born-digital) as rooted trees with nodes for text regions, graphical objects, and logical roles, recovering relations such as intra-region order, inter-region order, and hierarchical parent–child (TOC) links (Wang et al., 2024, Wang et al., 20 Mar 2025, Ma et al., 2023).
• Multimodal Fusion: Hierarchical Multi-modal Transformers (HMT) and structural Document MAPs (DMAP) combine textual hierarchy with figures, tables, and explicit alignment/containment/referTo edges, supporting queries that exploit both semantic structure and layout (Liu et al., 2024, Fu et al., 26 Jan 2026).
• Structure Recovery Pipelines: Systems such as Detect-Order-Construct jointly solve page object detection, reading order prediction, and tree assembly, using relation-prediction heads and anchor-pointed transformer attention (Wang et al., 2024, Wang et al., 20 Mar 2025).
• Human-Aligned Schemas: DMAP employs a structural-semantics agent to parse documents into a graph with levels—sections, pages, elements—and labeled edges (contains, align, refersTo), enabling iterative, structure-aware reasoning and retrieval (Fu et al., 26 Jan 2026).

5. Empirical Evidence and Applications

Empirical evaluation consistently demonstrates that hierarchical representations improve model accuracy, efficiency, and interpretability across a spectrum of document understanding tasks:

• Classification and Summarization: Hierarchical attention and hybrid models outperform flat CNN, RNN, or non-structural baselines on document classification, extractive summarization, and sentiment analysis (Abreu et al., 2019, Al-Sabahi et al., 2018, Zhang et al., 2019, Wei et al., 2020).
• Parallel Corpus Mining: Hierarchical document encoders (HiDE) increase robustness to sentence segmentation errors and outperform naive averaging for parallel document retrieval (Guo et al., 2019).
• Multimodal QA and Retrieval: Explicitly encoding figure-caption alignment, section containment, and cross-references sharply increases retrieval and QA performance on MMDocQA and MMLongBench, with large gains on layout- and visual-dependent queries (Fu et al., 26 Jan 2026, Liu et al., 2024).
• Computational Efficiency: Methods such as hierarchical chunking (BERT+LSTM/CNN) and hierarchical sparse attention (HDT) scale to much longer documents by limiting attention windows or masking across levels, dramatically reducing memory and latency (Khandve et al., 2022, He et al., 2024).
• Interpretability and Alignment: Hierarchical attention models enable visualization of per-level attention, tracing decisions to sections, paragraphs, or sentences (e.g., Structure Tree-LSTM) (Mrini et al., 2019).

6. Limitations, Open Challenges, and Future Directions

Current hierarchical representation methods present trade-offs and open challenges:

• Hierarchical Decomposition vs. Cross-Block Information: Purely local chunking or blockwise processing (e.g., hierarchical BERT without inter-chunk attention) risks losing dependencies across chunk or block boundaries (Khandve et al., 2022).
• Scalability with Document Length and Structural Complexity: Quadratic scaling of relation-heads or multi-level attention may present bottlenecks for very long or complex documents, especially when structural elements are numerous and interlinked (Wang et al., 20 Mar 2025).
• Dependence on Accurate Segmentation/Parsing: Structure recovery pipelines and tree-based models may propagate errors from early segmentation stages to higher-level representations (Wang et al., 2024). Robust parsing, especially on layout-heterogeneous documents, remains challenging (Ma et al., 2023).
• Limited Benchmarks and Multimodal Datasets: Large-scale, annotated datasets for cross-page, hierarchical, and multimodal structure are scarce, limiting benchmarking for generic document-type recovery (Ma et al., 2023, Wang et al., 20 Mar 2025).
• Unified Modeling and Joint Pretraining: Advances are emerging toward unified models capable of handling textual, visual, and structural signals, but further progress is needed for seamless multimodal, multi-task, and end-to-end learning (Liu et al., 2024, Fu et al., 26 Jan 2026).

Future directions include joint pre-training of vision–text–structure representations, dynamic or learned chunk/block boundaries, recursive and non-tree (e.g., DAG/graph) modeling of document structure, scalable relation prediction methods, and closer integration with graph-based and multimodal neural architectures (Khandve et al., 2022, He et al., 2024, Fu et al., 26 Jan 2026).