---
title: Document-Level Simplification
url: https://www.emergentmind.com/topics/document-level-simplification
type: topic
---

# Document-Level Simplification

Document-level simplification (DLS) is the automatic transformation of an entire multi-sentence text to improve readability while preserving essential content, discourse coherence, and overall document structure. Unlike sentence-level simplification, DLS explicitly requires handling cross-sentence operations, such as information reordering, addition, deletion, and the maintenance of discourse relations and anaphora across paragraphs and chapters [2110.05071], [2502.08281].

## 1. Definition and Scope of Document-Level Simplification

DLS involves producing a simplified document $D_\mathrm{out} = \{t_1, t_2, ..., t_n\}$ from a complex source document $D_\mathrm{in} = \{s_1, s_2, ..., s_m\}$. The output must meet several criteria:
- High fluency and grammaticality at the paragraph and document level;
- Faithfulness to the source’s semantic content (preservation of facts and relations);
- Significantly increased readability (lexical and syntactic);
- Coherence across discourse units, preserving or enhancing logical flow [2502.08281], [2110.05071].

This task subsumes and extends sentence-level simplification: DLS supports sentence and paragraph splitting/merging, content reorganization, anaphora resolution, global compression (removal of entire sentences/paragraphs), cross-sentence paraphrasing, and discourse marker manipulation [2305.19204], [1911.10384].

## 2. Corpora and Resources for Document-Level Simplification

A major bottleneck in DLS research has been the scarcity and complexity of parallel corpora annotated at the document level. Key resources include:

| Corpus       | Language | Size         | Annotations              | Key Features              |
|--------------|----------|--------------|--------------------------|---------------------------|
| D-Wikipedia  | English  | ~143,000     | Complex–simple docs      | Automatic & human eval    |
| SWiPE        | English  | ~145,000     | Fine-grained edit ops    | 19 edit categories; NLI   |
| Newsela      | English  | >1,500       | Multi-level rewrites     | Grade-based, professional |
| DEPLAIN      | German   | ~600–1,000   | Manual sentence/doc      | News & web, CEFR levels   |
| SAMER        | Arabic   | 15 novels ×3 | Multi-level, word RL     | RL-dependent lexicon      |

- D-Wikipedia focuses on complex/simple Wikipedia lead sections with detailed human ratings [2110.05071].
- SWiPE aligns full revision histories from English and Simple English Wikipedia, annotating over 40,000 specific edits in 19 categories (lexical, syntactic, discourse, semantic, non-simplification) [2305.19204].
- Newsela provides professional multi-level manual simplifications of news articles with detailed readability grading.
- DEPLAIN targets German and provides professional news and web documents aligned at sentence and document granularity, supporting both manual and automatic alignment [2305.18939].
- SAMER builds Arabic multi-level simplification data with rigorous word/document readability labels [2404.18615].

These corpora enabled the development of both supervised and unsupervised DLS models and the establishment of task-appropriate evaluation metrics.

## 3. System Architectures and Methodologies

### Pretraining and Fine-Tuning Paradigms

Large pre-trained seq2seq models (BART, T5, mBART) dominate DLS. Key advances include:
- SimpleBART, which leverages continued pre-training with a simplicity-aware masking objective using simple texts (SimpleWiki, Newsela) and ordinary texts with complex→simple span replacement. This induces strong document-scale simplification ability without architecture changes [2305.12463].
- SimDoc introduces multi-objective training, jointly optimizing simplification, readability, and discourse coherence by integrating external coherence classifiers and per-document readability labels, thus explicitly targeting global document properties [2412.18655].

### Planning, Progressive, and Multi-Stage Approaches

Recognizing limits in single-step generation, several approaches decompose DLS:

- **Progressive, hierarchical systems** (ProgDS) simulate human editing passes: discourse segmentation and subheading assignment → topic-level sentence/paragraph simplification → lexical-level rewriting. This decomposition matches human strategies for coherent, controlled simplification [2501.03857].
- **LLM-Guided pipelines** utilize large language models to first generate concise summaries (semantic scaffolds) and then produce a summary-aligned simplification, improving global coherence for long scientific documents [2508.11816].

### Context-Aware and Plan-Guided Models

- Context-aware architectures (e.g., ConBART, LEDpara) encode explicit inter-sentence and paragraph context during simplification. This is achieved either by processing larger text units (paragraphs) or introducing additional cross-attention over dynamic document windows, ameliorating coherence and anaphora issues that plague sentence-by-sentence approaches [2305.06274].
- Plan-guided pipelines use a first-stage planner (predicts per-sentence operation: copy, rephrase, split, delete) followed by a controlled simplification generator, further enhancing alignment and global consistency [2305.06274], [2404.03278].

### Decoding and Selection

- Minimum Bayes Risk with Optimal Transport (MBR-OT) extends sentence-level utility functions to document level using Wasserstein distances between sets of generated sentences, better capturing structural reordering and non-aligned phenomena characteristic of DLS [2505.23078].

## 4. Evaluation Metrics and Analysis

Traditional simplification metrics (SARI, BLEU) are adapted for documents, but DLS metrics penalize over-deletion, verbosity, and loss of structure:

| Metric           | Definition & Adaptation           | Purpose                         |
|------------------|----------------------------------|----------------------------------|
| D-SARI           | SARI + length/sentence-penalties | Document-scale SARI [2110.05071] |
| FKGL/FRE         | Document-wide readability         | Reading grade/ease [2412.18655]  |
| COH_out          | External classifier on coherence  | Discourse assessment             |
| SLE_doc/ESLE_doc | Simplicity-level estimator        | Match to target grade            |
| BARTScore, BLEU  | Document-wide faithfulness/fluency| Fluency/factuality               |

Reference-based and reference-less metrics are both used; recent work stresses the need to separately analyze simplicity vs. faithfulness, given their tradeoff (more aggressive simplifiers may drop content) [2404.03278]. D-SARI and COH_out were found to align better with human judgments of overall quality than basic SARI or BLEU.

## 5. Empirical Results and State of the Art

Recent strong DLS systems, as evaluated on D-Wikipedia, Newsela, or domain-specific scientific corpora, show marked improvements:

- On D-Wikipedia, SimpleBART achieves 41.64 D-SARI, outperforming fine-tuned BART and plan-guided baselines (+1.8 D-SARI over BART) [2305.12463].
- LLM pipelines, including GPT-4o and Llama3.1-70B, demonstrate that closed/open-source LLMs have caught up or outperformed prior non-LLM and smaller transformer models, achieving highest simplicity and lowest FKGL [2502.08281].
- Hierarchical/progressive systems like ProgDS, especially with CoT or iterative passes, lead D-SARI/SARI by notable margins (e.g., up to ~46.89 SARI on Newsela-A) and nearly match human references in coherence and simplicity judgments [2501.03857].
- SimDoc’s multi-signal optimization yields up to 50.09 D-SARI and FKGL ~3.2 on Newsela, with explicit improvements in both readability and discourse coherence [2412.18655]. Sentence-level baselines lag by over 20 D-SARI points.
- For low-resource languages, DEPLAIN and SAMER introduced new doc-scale corpora, and baseline doc-level models outperform sentence-level in SARI and readability, displaying greater structural handling [2305.18939], [2404.18615].

A representative cross-section of model performance is summarized below:

| Model                | D-SARI (↑) | FKGL (↓) | Coherence/COH (↑) | Dataset        |
|----------------------|------------|----------|-------------------|---------------|
| SimpleBART           | 41.64      | —        | —                 | D-Wikipedia   |
| GPT-4o               | 41.96      | 5.46     | —                 | Newsela (200) |
| LEDpara (context)    | 83.0*      | —        | —                 | Newsela-auto  |
| T5-large (SimDoc, FT)| 50.09      | 3.24     | —                 | NewselaS      |
| mBART (DEPLAIN, doc) | 44.56      | —        | —                 | DEPLAIN-APA   |

*Metric scales may differ; LEDpara reports SARI, others D-SARI; direct metric comparison must account for this.

## 6. Key Scientific and Technological Challenges

The main technical challenges that distinguish DLS from sentence simplification include:
- **Document-scale content selection**: accurate identification of redundant or peripheral content for deletion [1911.10384], [2501.03857].
- **Discourse and coherence modeling**: maintaining logical flow, causal or temporal relations, resolving anaphora, and ensuring sentence ordering is both comprehensible and faithful [2305.06274], [2412.18655].
- **Evaluation**: balancing aggressive simplification (readability, simplicity) with meaning preservation and factual correctness—trade-offs visible in both automatic and human assessments [2404.03278].
- **Data scarcity**: scalability of annotated corpora across genres, languages, and domains; alignment and annotation quality, especially for high-variance transformations [2305.19204], [2305.18939].
- **Computational load**: DLS models may require higher context windows, hierarchical or multi-stage architectures, and expensive decoding approaches such as MBR-OT [2505.23078].

## 7. Open Problems and Future Directions

Several future avenues are proactively identified:
- **Multi-level controllability**: generating simplification at different granularity/readability levels, possibly tailored to individual user profiles [2502.08281], [2412.18655], [2404.18615].
- **Dynamic evaluation and iterative prompting**: integrating automatic metrics like SARI, COH_out inline for iterative model refinement [2508.11816], [2412.18655].
- **Broader and cross-lingual resources**: extending corpora, models, and evaluation to non-English and low-resource settings; leveraging web-scale harvesting, as for DEPLAIN [2305.18939], [2404.18615].
- **Finer-grained coherence and discourse objectives**: moving towards differentiable, learning-based discourse reward or penalty signals [2412.18655].
- **Hybrid and modular architectures**: combining retrieval, planning, and progressive generation, or fusing oracle edit plans into LLM pipelines [2305.19204], [2501.03857].
- **Robustness and factuality**: addressing error cases—hallucination, over/under-compression, loss of document tone—via external knowledge or plan-guided generation.

DLS now benefits from the convergence of data resources, transformer pretraining, and LLM scaling. Contemporary models operating at the document level set a strong baseline but emphasize the need for explicit, interpretable targeting of both readability and coherence, as well as principled evaluation frameworks that respect the complexity of long-form language transformations.

Source: https://www.emergentmind.com/topics/document-level-simplification