Document-Level Information Extraction

• Document-level Information Extraction (DocIE) is a computational process that identifies and structures entities, relations, and events spread across entire documents.
• It employs diverse architectures such as Transformer-based encoders, graph neural networks, and iterative extraction methods to handle cross-sentence dependencies.
• DocIE systems are evaluated using specialized benchmarks like DocRED and DWIE, focusing on coreference resolution and long-context aggregation challenges.

Document-level Information Extraction (DocIE) refers to the computational identification and structuring of entities, relationships, events, and facts that are distributed across sentences or structural boundaries within full documents. Unlike sentence-level IE, DocIE must resolve cross-sentence coreference, aggregate fragmented evidence, handle long-range dependencies, and confront challenges of event individuation and output variability. This field spans a diverse range of tasks, evaluation paradigms, and modeling strategies, integrating long-context representation learning, multimodal reasoning, and end-to-end structured generation.

1. Task Definition and Problem Formulation

DocIE aims to map an input document $x$ to a structured output $y$, where $y$ may include entities, attributes, coreference clusters, relations, and event templates. For illustrative clarity, core DocIE tasks fall into several types:

• Document-level relation extraction (DocRE): Infer relationships between entity pairs whose mentions may be separated by arbitrary distances within the document (Zheng et al., 2023).
• Document-level event extraction (DocEE): Identify event triggers and argument roles, possibly assembling $N$-ary templates aggregating mentions and spans (Gantt et al., 2022).
• Closed IE (cIE) and Open IE: Extraction is either constrained to a fixed schema or formulated to discover arbitrary salient relations (Bouziani et al., 2024, Dong et al., 2021).
• Template filling: For each event or relation type, extract a potentially unbounded set of slot-filled templates, addressing complex individuation decisions (Gantt et al., 2022).

Mathematically, the output function is

$f: x \mapsto y = \left\{ \text{entity set}, \text{coreference clusters}, \text{relation tuples}, \text{event templates or $N$-ary facts} \right\}$

where slot fillers may be sets of text spans drawn from the input. Key distinguishing features versus sentence-level IE include the necessity for joint modeling of coreference, global reasoning, and variable template cardinality.

2. Major Approaches and Model Architectures

DocIE systems employ a spectrum of architectures, most leveraging deep pre-trained models for token-level encoding enhanced with layers for long-range reasoning or structured prediction (Zheng et al., 2023). The principal solution families include:

• Encoder–Decoder LMs: Direct sequence-to-JSON/text generation, enabling end-to-end mapping from document text (optionally with prompts) to structured outputs (Townsend et al., 2021, Zubillaga et al., 26 Jan 2026).
• Joint Multi-Task Transformers: Simultaneously perform mention detection, coreference resolution, entity typing, relation extraction, and linking within a single forward pass, e.g., REXEL, which unifies all sub-tasks on top of RoBERTa (Bouziani et al., 2024).
• Graph Neural Networks (GNNs): Encode document structure with mention/entity/sentence nodes connected by edges for co-occurrence, coreference, or relation context; refine representations via relational GCNs and attention blocks (e.g., GLRE) (Wang et al., 2020).
• Imitation Learning and Iterative Extraction: Sequential Markov decision process (MDP) frameworks iteratively generate event or relation templates, with memory vectors ensuring inter-template consistency (ITERX) (Chen et al., 2022).
• In-context Learning (ICL): For LLMs not directly fine-tuned, carefully engineered prompt suites (e.g., D3IE, ThinkTwice) guide models to perform DocIE in zero/few-shot settings, often with synthetic or hard demonstration selection (Zubillaga et al., 26 Jan 2026, He et al., 2023, Popovič et al., 8 Jul 2025).

A summary of notable DocIE model types and their characteristics:

Approach	Core Component(s)	Key Strength
Joint Transformer	Single encoder, multi-head	End-to-end, fast, cross-task info
Graph Neural Network	Heterogeneous doc graph + GNN	Multi-hop, cross-sentence links
Encoder–Decoder (LM)	Seq2seq Transformer	Flexible, schema-free
Iterative Extraction	MDP, oracle-based imitation	Consistency, order-agnostic
In-context LLM	Retrieved/demo-prompted LLM	Zero/few-shot, demo agnostic

3. Evaluation Benchmarks and Datasets

DocIE research is enabled by several benchmark datasets, each capturing a specific task profile:

• DocRED: Wikipedia paragraphs for document-level relation extraction, with 96 relation types, requiring significant multi-sentence reasoning. SOTA F1: 67.3–80.9 (Zheng et al., 2023).
• DWIE: News corpus with document-level multi-task annotation for entity-centric extraction (NER, coref, RE, EL), evaluated with novel cluster-based F1 metrics (Zaporojets et al., 2020).
• MUC-4, BETTER: Newswire with template-filling event types and associated arguments; used to foreground issues in event individuation (Gantt et al., 2022, Chen et al., 2022).
• SciREX: Scientific papers requiring $N$-ary (binary and 4-ary) relation extraction, salient entity identification, cross-section aggregation (Jain et al., 2020).
• DocILE: Large-scale business documents (invoices, receipts); focus on key information localization and line-item recognition in visually rich formats (Šimsa et al., 2023).
• FUNSD, CORD, SROIE, LIE: Visually rich documents with spatial structure, facilitating studies of layout-aware, OCR-to-entity extraction (Zhang et al., 2022, He et al., 2023).
• DocOIE: Custom document-level Open IE set with annotations capturing context-dependent tuple extraction (Dong et al., 2021).

These resources include a mixture of human-annotated and hybrid distantly-supervised labels, with an increasing trend toward large, multi-modal, and cross-language benchmarks (e.g., DocILE, MultiMUC (Zubillaga et al., 26 Jan 2026)).

4. Error Analysis, Evaluation Metrics, and Core Challenges

Conventional precision/recall/F1 scores fail to disambiguate model failures in DocIE (Das et al., 2022). Error analyses reveal particular challenges:

• Event individuation: Disagreement among human annotators regarding event boundaries; metrics unduly penalize merges/splits even when arguments are correct (Gantt et al., 2022).
• Label noise and annotation inconsistency: Especially prevalent in distantly supervised corpora, introducing up to 10–15% false negatives, which depresses recall (Zheng et al., 2023).
• Entity coreference and reasoning: Most systems handle within-sentence coreference modestly, but cross-sentence and multi-hop reasoning errors account for 25–40% of extraction failures (Zheng et al., 2023).
• Output variability in generation: Models may generate multiple semantically valid but syntactically different outputs; naive metrics can underestimate capacity; selection modules or agreement-voting can mitigate this (Zubillaga et al., 26 Jan 2026).
• Long document and context window limits: Transformers’ 512–2048 token windows can truncate evidence or prevent global aggregation, challenging cross-sentence attribute assignment (Wang et al., 2023).
• Evidence faithfulness and auditable extraction: Recent work develops “predict-select-verify” pipelines with feature attribution and small evidence supervision to improve model plausibility (Tang et al., 2021).

A condensed taxonomy of dominant error types:

Error Type	Example / Symptom	Source
Coreference Failure	Missed cross-sentence entity links	(Zheng et al., 2023)
Event Individuation Error	Merge/split of event templates	(Gantt et al., 2022)
Reasoning Deficiency	Missed multi-hop/inferential relations	(Zheng et al., 2023)
Spurious Extraction	Relations/events extracted spuriously	(Zheng et al., 2023)

Metrics adapted for DocIE include CEAF-RME, F1 with cluster alignment, slot-level and template-level scoring, as well as entity-driven or soft-cluster F1 for entity-centric annotation (Chen et al., 2022, Zaporojets et al., 2020).

5. Advances, Innovations, and Methodological Directions

Recent methodological advances focus on both modeling and evaluation:

• Decoding and Selection Strategies: ThinkTwice (sampling and selection via unsupervised F1-agreement or supervised reward models) turns LLM output diversity from a drawback into a systematic advantage, consistently outperforming greedy decoding and setting new SOTA on MUC-4 and BETTER (Zubillaga et al., 26 Jan 2026).
• Imitation Learning for Template Extraction: Policy learning with dynamic oracles and iterative span memory updates enables order- and number-agnostic extraction of multiple templates per document (Chen et al., 2022).
• Layout-Aware and Multimodal Pretraining: LayoutLM variants, layout-aware pretraining (MLLM, PHS), and hybrid visual-text approaches yield distinct gains on visually rich and form-like document benchmarks (Zhang et al., 2022, Šimsa et al., 2023).
• In-context and Synthetic Data Methods: ICL-D3IE, DocIE@XLLM25, and similar pipelines emphasize the power of tailored demonstration selection, synthetic data, and prompt engineering for ICL with LLMs in DocIE tasks, particularly in zero/few-shot, cross-lingual, or low-resource regimes (He et al., 2023, Popovič et al., 8 Jul 2025).
• Faithful, Auditable Explanations: The predict-select-verify framework, paired with light evidence supervision, yields models that not only maintain accuracy but more reliably highlight the text fragments supporting each extraction (Tang et al., 2021).

6. Limitations, Open Issues, and Future Directions

Despite progress, several persistent obstacles and research frontiers remain:

• Event individuation and metric design: Current template-filling paradigms confound minor grouping disagreements with major argument errors; future metrics may need plural annotation or aggregation-only scoring (Gantt et al., 2022).
• Coreference/system memory: Cross-sentence and document-level entity resolution is insufficiently robust; explicit multi-level or memory-augmented mechanisms are being explored (Wang et al., 2023, Bouziani et al., 2024).
• Long-context representation: Transformer models must scale better to handle long (10k+) token documents, and merge local/global evidence in memory-efficient fashion (Townsend et al., 2021, Wang et al., 2023).
• Explainability and evidence alignment: Ensuring extracted evidence aligns with model predictions and is human-plausible is a nascent sub-field with importance for clinical, legal, and scientific applications (Tang et al., 2021).
• Interleaving symbolic and neural methods: Incorporation of external knowledge graphs, symbolic inference, and reasoning remains limited, especially for multi-hop and commonsense-rich settings (Zheng et al., 2023, Bouziani et al., 2024).
• Robustness to modality and annotation noise: While synthetic demonstrations and semi-supervision enable flexibility, handling OCR noise, layout artifacts, and annotation inconsistencies is still challenging (Šimsa et al., 2023, He et al., 2023).

A plausible implication is that future DocIE will increasingly emphasize unified joint models, robust metric development, explainable reasoning mechanisms, and the adaptation to semi-structured, multi-modal, and cross-domain document corpora.

7. Comparative Performance and Impact

Head-to-head comparison of contemporary DocIE systems illustrates marked advances over pipeline and naïve baselines. For example, in template extraction on MUC-4 (F1, supervised):

Model	F1	Reference
ITERX (T5 large)	35.2	(Chen et al., 2022)
ThinkTwice+Reward LLM	42.0	(Zubillaga et al., 26 Jan 2026)
Joint RoBERTa+LayoutLMv3	0.70 (LIR)	(Šimsa et al., 2023)

In end-to-end document-level relation extraction (DWIE, full DocIE):

Model	F1	Reference
DWIE Baseline	88.3	(Zaporojets et al., 2020)
REXEL	95.4	(Bouziani et al., 2024)

As benchmarks become larger, more granular, and reflective of real-world complexity, continued gains will require advances in reasoning, layout/multimodal fusion, and cross-lingual adaptation.

---

Document-level Information Extraction is a rapidly evolving field, transitioning from local, pipeline-centric solutions to highly integrated, scalable, and auditable systems capable of deep semantic understanding at the document scale. Its challenges lie at the intersection of advanced machine learning, linguistic and discourse theory, corpus linguistics, and practical system engineering. For further study, a systematic review is provided in "A Survey of Document-Level Information Extraction" (Zheng et al., 2023), and state-of-the-art methods and datasets are detailed in references within this article.