DocEditBench: Document-Level Model Editing

Updated 8 July 2026

DocEditBench is a benchmark for document-level model editing that requires transforming entire Wikipedia introductions based on temporal updates.
It emphasizes editing full documents with multiple factual changes, ensuring that updated parts reflect new information while preserving unchanged sections.
The benchmark highlights challenges in long-context extrapolation, multi-fact integration, and controlling unintended side effects in model outputs.

DocEditBench, in the query-aligned usage, denotes DocMEdit (Document-level Model Edit), a benchmark for document-level model editing in LLMs. It was introduced to address a limitation of prior model-editing datasets, which typically require short phrases or single sentences and therefore provide only a weak proxy for realistic long-form updating. DocMEdit instead requires models to transform an input document into an updated full document, under edits that are document-level, extrapolative, multi-fact, and local, using temporally separated Wikipedia introductions as source and target documents (Zeng et al., 26 May 2025). The label DocEditBench is not used uniformly across recent literature: in later work on scientific PDF refinement, it also names a distinct benchmark for multimodal scientific document editing rather than document-level knowledge editing in Wikipedia text (Qian et al., 9 Aug 2025).

1. Name, identity, and scope

The benchmark introduced in "DocMEdit: Towards Document-Level Model Editing" is called DocMEdit. In the abstract it is described as a dataset focused on document-level model editing and referred to generically as a benchmark $\benchmarkname$; in the query-aligned usage, the same benchmark is called DocEditBench. Functionally, these denote the same benchmark in the DocMEdit paper (Zeng et al., 26 May 2025).

Usage in literature	Core task	Source
DocMEdit / query-aligned “DocEditBench”	Document-level model editing over Wikipedia introductions	(Zeng et al., 26 May 2025)
“DocEditBench” in DocRefine	Scientific paper PDF editing under natural language instructions	(Qian et al., 9 Aug 2025)

This naming overlap is a recurrent source of confusion. In the DocMEdit sense, the benchmark is about editing model behavior so that it produces updated full documents. In the DocRefine sense, the benchmark is about end-to-end scientific PDF editing, including text, figures, tables, formulas, and layout-sensitive modifications. The two task definitions, data modalities, and evaluation objectives are different, even though both are document-centric (Zeng et al., 26 May 2025).

2. Document-level model editing as a task

DocMEdit defines document-level model editing in contrast to earlier sentence-level or fact-level settings such as QA or cloze-style triple editing. In the earlier setting, outputs are typically short phrases or single sentences. In DocMEdit, the model is given a full document $x$ and must produce an updated document $y'$ that reflects new facts rather than a short answer string (Zeng et al., 26 May 2025).

The task is characterized by three central features. First, it has document-level inputs and outputs: both the prompt and the expected output are full documents. Second, it is extrapolative: the model receives supporting facts $F$ , but the target document $y'$ is not simply the concatenation or paraphrase of those facts. The model must understand the original document, integrate the new facts, preserve unchanged content, and maintain coherence and style. Third, it involves multiple facts within a single edit: multiple entities and facts are updated simultaneously within one document, so the model must determine which parts of the document are affected and how the updates should be inserted or rewritten (Zeng et al., 26 May 2025).

A fourth property is locality. DocMEdit is constructed so that updated parts and unchanged parts are both identifiable. This enables explicit evaluation of side effects: edit-related outputs should change, while unrelated outputs should remain as in the original model. A plausible implication is that success in this setting requires not only factual correction but also control over document-level preservation and unintended drift (Zeng et al., 26 May 2025).

3. Corpus construction and dataset statistics

DocMEdit is built entirely from Wikipedia, focusing on article INTRODUCTION sections. It leverages actual temporal updates between two Wikipedia dumps, 2023-11-01 and 2024-11-01, so that each instance reflects a real document-level change rather than a synthetic rewrite (Zeng et al., 26 May 2025).

Statistic	Value	Meaning
Data items	37,990 documents	Benchmark size
Average context length	1,535.5 tokens	Input + target per item
Expected output length	867.62 tokens	Average target length
Facts to edit	105,651 facts	Total edited facts
Facts per document	2.78	Average multi-fact load
Facts length	623.40 tokens	Sum over facts per item

For each article, the input document $y$ is the introduction from the 2023-11-01 dump, and the target document $y'$ is the introduction from the 2024-11-01 dump. Each data item includes $x$ , the updated document $y'$ , a fact set $F = \{f_1, \dots, f_m\}$ , and knowledge graph triples extracted from the input document, the target document, and the supporting facts (Zeng et al., 26 May 2025).

Construction proceeds in three stages. In Document Change Computation, introduction text is extracted at both timestamps and the pair $x$ 0 is retained only when at least one new entity is introduced, using entity additions as a proxy for meaningful factual updates. In Facts Collection, sentences in the updated document that mention newly introduced entities are treated as supporting facts $x$ 1. In Knowledge Graph Extraction, the open IE pipeline from Schmitz et al. (2012) is used to extract triples $x$ 2 from the source document, target document, and facts, and relations $x$ 3 are filtered to those present in Wikidata (Zeng et al., 26 May 2025).

The extracted knowledge graph contains 568,652 entities, 4,804 relations, and 1,411,057 triples. These structured artifacts support editing methods that operate over triples rather than raw text and therefore make the benchmark usable for both text-based and structure-aware model-editing pipelines (Zeng et al., 26 May 2025).

4. Formalization and benchmark properties

The paper formalizes document-level model editing with a LLM $x$ 4 with parameters $x$ 5, an input document $x$ 6, and an original output document $x$ 7. The document is written as

$x$ 8

where each $x$ 9 is a sentence. The supporting facts to be edited are written as $y'$ 0 (Zeng et al., 26 May 2025).

Editing produces an updated model $y'$ 1 with parameters $y'$ 2:

$y'$ 3

and the edited model generates

$y'$ 4

The updated document is then conceptualized as

$y'$ 5

This formulation makes the benchmark’s structural assumptions explicit: some sentences correspond to preserved document content, while others are newly inserted or modified under the influence of edited facts $y'$ 6 (Zeng et al., 26 May 2025).

The same formalization clarifies why DocMEdit is more difficult than short-answer editing. The edit operator must absorb $y'$ 7 into model parameters, but the resulting output is a long document that must preserve unaffected text, integrate multiple factual changes, and remain coherent. This suggests that methods optimized for single subject–relation–object updates or isolated QA-style corrections face an immediate scaling problem when the output space becomes a full document rather than a token-level answer span (Zeng et al., 26 May 2025).

5. Quality control, evaluation signals, and empirical findings

To ensure data quality, the authors manually remove unsupported updates, defined as cases where the automatically identified update is not actually supported by any fact. They also compare human annotations with automatic extractions using the same metrics later used for evaluation, treating these as internal consistency checks (Zeng et al., 26 May 2025).

The two named checks are Document-ROUGE (DR) and Document-Entity (DE). The reported alignment between human-labeled and automatically derived updates is DR: 81.17 and DE: 89.71. These values are presented as evidence that facts are well aligned with document changes and that the updates are well supported (Zeng et al., 26 May 2025).

The paper states that it proposes a series of evaluation metrics and experiments and reports a broad empirical conclusion rather than a narrow leaderboard claim: the difficulties in document-level model editing pose challenges for existing model editing methods. Within the benchmark design, those difficulties derive from long context, extrapolative generation, multiple facts in a single edit, and the requirement that unrelated parts of the document remain unchanged. A plausible implication is that document-level editing should be treated not merely as fact insertion but as a joint problem of factual updating, discourse integration, and locality control (Zeng et al., 26 May 2025).

6. Relation to adjacent benchmarks and later usage

DocMEdit occupies a specific position within the broader landscape of document-centered LLM evaluation. It is neither a raw-document reading benchmark nor a layout-grounded document-image editing benchmark. That distinction matters because several adjacent resources use similar terminology while targeting different computational problems (Zeng et al., 26 May 2025).

"DocBench: A Benchmark for Evaluating LLM-based Document Reading Systems" evaluates systems that take raw PDF files and questions as input and return textual answers. It contains 229 real documents and 1,102 questions across five domains, and it is explicitly not a document-editing benchmark; the name “DocEditBench” does not appear in that paper (Zou et al., 2024). By contrast, "DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding" uses the DocEdit-PDF dataset with 17,808 document image pairs for language-guided document structure editing, including command generation, region-of-interest localization, and end-to-end editing in HTML + CSS space (Suri et al., 2024).

A separate later usage appears in "DocRefine: An Intelligent Framework for Scientific Document Understanding and Content Optimization based on Multimodal Large Model Agents," where DocEditBench denotes a benchmark of scientific paper PDFs, paired with editing instructions and gold-standard output documents. That benchmark is organized around Text Refinement, Structural Editing, Summarization, and Multimodal Correction, and uses Semantic Consistency Score (SCS), Layout Fidelity Index (LFI), and Instruction Adherence Rate (IAR) as its principal metrics (Qian et al., 9 Aug 2025). This later usage is distinct from DocMEdit, even though both benchmarks evaluate document-scale modification.

A more distant but methodologically relevant comparison is "DTBench: A Synthetic Benchmark for Document-to-Table Extraction," which introduces a capability-aware taxonomy for document-to-structure transformation and separates direct from indirect extraction. Although DTBench targets extraction rather than editing, its emphasis on capability-specific evaluation, faithfulness, and conflict resolution suggests a possible direction for future document-editing benchmarks that need to distinguish simple local corrections from edits requiring reasoning, normalization, or conservative refusal (Guo et al., 14 Feb 2026).

In this broader context, DocMEdit’s main contribution is to anchor model editing in document-level outputs rather than short answers. Its benchmark design makes long-form updating, extrapolation, multi-fact integration, and side-effect control first-class evaluation targets, thereby shifting model-editing research toward settings that more closely resemble real document maintenance workflows (Zeng et al., 26 May 2025).