Wikipedia 2018 Corpus: Atomic Edits Dataset

• Wikipedia 2018 Corpus is a dataset of 43M atomic edits defined as single contiguous insertions or deletions extracted from Wikipedia revision histories across 8 languages.
• The corpus emphasizes atomic insertions to capture explicit semantic additions and discourse refinements for improved representation learning.
• Its extraction pipeline uses local window BLEU scoring and minimal byte-level diffs to ensure high precision in aligning sentence-level edits.

WikiAtomicEdits is a multilingual Wikipedia-derived corpus of 43 million atomic edits mined from Wikipedia edit history and released to support research in semantics, discourse, and representation learning (Faruqui et al., 2018). In this usage, the label “Wikipedia 2018 Corpus” refers not to a static snapshot of article text, but to a structured dataset of atomic insertion and deletion edits extracted from revision histories. The corpus spans 8 languages and is centered on edits in which a human editor has inserted or deleted a single contiguous phrase in an existing sentence, with the paper placing particular emphasis on atomic insertions because they provide a cleaner semantic signal than deletions (Faruqui et al., 2018).

1. Corpus definition and multilingual scope

An atomic edit is operationalized as an insertion or deletion of a single contiguous phrase, specifically a phrase inserted into or deleted from an existing sentence, such that the resulting sentence remains well-formed enough to support semantic analysis (Faruqui et al., 2018). The corpus totals 42.9M / 43M edits, consisting of 25.7M insertions and 17.2M deletions. Its language coverage is English, German, Spanish, French, Italian, Japanese, Russian, and Chinese (Faruqui et al., 2018).

The paper’s main analyses focus on insertions. The rationale is explicit: for an insertion, the original sentence did not communicate some information, a human editor believed it should be communicated, and the inserted phrase is the added information. Deletions are included in the release, but they are treated as methodologically noisier because deleted spans are more likely to be spam, false information, or malformed text, and because a deletion is harder to interpret semantically: the removed phrase may be false or irrelevant, or it may simply be redundant because the surrounding context already implies it (Faruqui et al., 2018).

Language	Insertions (M)	Deletions (M)
English	13.7	9.3
German	3.3	1.9
Spanish	1.4	0.9
French	2.0	2.0
Italian	1.0	0.6
Japanese	2.2	1.3
Russian	1.4	0.9
Chinese	0.7	0.4

English constitutes the largest share, with 23.0M total edits, but the corpus is explicitly multilingual rather than an English-only resource. This multilingual design makes it possible to compare editing behavior and edit-conditioned modeling across typologically different languages within a single extraction framework (Faruqui et al., 2018).

2. Extraction pipeline from revision history

The corpus is mined from Wikipedia edit history, where revisions are stored as page-level diffs. To recover sentence-level edits, the construction process compares historical snapshots of each article (Faruqui et al., 2018). The extraction pipeline is described as follows: take historical snapshots of pages, strip HTML and Wikipedia markup, sentence-split each snapshot, search for corresponding sentences across nearby regions in a later snapshot, and retain only pairs that differ by exactly one contiguous insertion or deletion (Faruqui et al., 2018).

A key implementation detail is the use of a precision-oriented approximation rather than full quadratic sentence alignment. For each sentence $s_i$ in a base snapshot, the method searches a local window of nearby sentences $t_j$ in a later snapshot, with $j = i-k \ldots i+k$ and $k=5$. It then uses BLEU score to identify the most similar candidate sentence within that window (Faruqui et al., 2018). This local-window restriction is intended to make extraction tractable while preserving precision.

The paper further notes that, for each article, the process considers the most recent 100,000 snapshots as of February 2018. After candidate sentence alignment, Python’s difflib is used to compute a minimal byte-level diff, and only cases where the difference is a single contiguous phrase are retained (Faruqui et al., 2018). This final step is the operational criterion that turns ordinary revision differences into “atomic” edits.

The resulting resource is not simply a collection of article versions. It is a corpus of aligned edit events, each linking an original sentence, an edited sentence, and the inserted or deleted phrase. This suggests a shift from document-centric corpus design toward edit-centric supervision, in which the signal comes from what human editors chose to change rather than from the final article state.

3. Quality control, annotation, and edit typology

To assess extraction quality and reproducibility, the authors crowdsourced annotations for 5,000 English edits, 1,000 Spanish edits, and 1,000 German edits, for a total of 7,000 edits (Faruqui et al., 2018). Annotators were shown a sentence $s$ and a phrase $p$ to insert, and were asked either to place $p$ where it naturally belongs or to label the item as an error if no coherent insertion point existed. The study used 5-way annotations for English and 3-way annotations for Spanish and German (Faruqui et al., 2018).

For edits judged meaningful by all annotators, the reported no-error rates are 78% for English, 55% for Spanish, and 85% for German (Faruqui et al., 2018). These figures indicate that most extracted examples are legitimate edits, although the corpus contains nontrivial noise. The paper also measures whether annotators place the inserted phrase in the same location as the original editor. Agreement with the original editor is 66% for English, 72% for Spanish, and 85% for German (Faruqui et al., 2018).

Disagreement is not treated as uniform failure. In a manual analysis of 100 English disagreements, 49% were classified as meaning equivalent, 22% as significant meaning differences, 13% as minor differences or ambiguities, and 16% as annotator error (Faruqui et al., 2018). The corpus therefore captures not only overt semantic additions but also subtler discourse-structural decisions about where an added phrase belongs.

The paper also proposes a manual typology of edits. In a 100-edit sample, the distribution was 43% Extension, 24% Refinement, 11% Referring Expressions, 9% Fluency, and 13% Error (Faruqui et al., 2018). Here, Extension adds genuinely new information not previously communicated; Refinement adds clarification, hedging, or scope narrowing; Fluency / Discourse includes grammatical fixes and discourse connectives; Referring Expressions modify the form of reference without changing the referent; and Error covers spam, vandalism, or malformed edits (Faruqui et al., 2018).

A common misconception is that disagreement over insertion location necessarily invalidates the example. The annotation results do not support that view: many disagreements were meaning equivalent or reflected minor ambiguities, which indicates that the resource contains discourse-sensitive variation rather than only noise (Faruqui et al., 2018).

4. Linguistic properties of edit language

A central empirical claim of the paper is that language observed in edits is not just ordinary Wikipedia text (Faruqui et al., 2018). The authors compare inserted phrases to general Wikipedia text and report clear distributional differences. In particular, adjectives and adverbs are much more common in inserted phrases than in ordinary Wikipedia text, and proper nouns are also inserted at higher rates than expected from baseline frequency (Faruqui et al., 2018). Some analyses are restricted to single-word insertions in order to make POS comparisons cleaner.

The lexical profile of inserted material is also distinctive. Words disproportionately inserted often reflect refinement or explicit clarification, including former, current, famous, and professional. By contrast, words underrepresented among insertions include first, only, all, and such, which the paper interprets as items that authors are unlikely to omit if they are relevant (Faruqui et al., 2018). This supports the view that edit histories expose information about what humans choose to make explicit in context.

These observations matter because they distinguish revision-based corpora from final-text corpora. A final article records what was eventually written; an edit corpus records what contributors felt was missing, misleading, underspecified, or pragmatically incomplete. The paper’s interpretation is that edits often make implicit information explicit or add discourse-level structure that is less visible in raw text (Faruqui et al., 2018).

5. Modeling tasks and empirical results

The paper evaluates whether models trained on WikiAtomicEdits learn something different from models trained on ordinary Wikipedia text (Faruqui et al., 2018). The first task is predict insertion location: given a sentence $s$ and a phrase $p$, predict the correct insertion point $i$. Two model families are compared. The General LM inserts $t_j$0 at every possible position and chooses the one with lowest perplexity, using a model trained on raw Wikipedia text. The Discriminative model is trained directly on insertion data and uses a biLSTM encoder over the sentence to compare candidate positions with the phrase (Faruqui et al., 2018).

The discriminative model outperforms the general LM on every language. Test accuracy is reported as follows: German 68.1 vs 72.9, English 58.7 vs 68.4, Spanish 67.0 vs 70.1, French 69.9 vs 73.4, Italian 69.0 vs 72.9, Japanese 73.0 vs 74.2, Russian 72.9 vs 74.3, and Chinese 65.5 vs 68.9; the average is 68.0 for the general LM and 71.8 for the discriminative model (Faruqui et al., 2018). The paper highlights that the edit-trained model performs better despite having fewer parameters and less text than the large general LLM.

The second task is generate insertion phrases: given a sentence $t_j$1 and an insertion point $t_j$2, generate a phrase $t_j$3 that would naturally belong there. The comparison is between an Edits model trained on WikiAtomicEdits and a General model trained on pseudo-edits sampled from raw Wikipedia text, where spans are removed under syntactic constraints (Faruqui et al., 2018). Evaluation uses Log perplexity, Exact Match, and Similarity@1 based on cosine similarity of phrase embeddings. The reported results are: Edits model — 8.32, 13.1%, 0.54; General model — 9.23, 8.0%, 0.48 (Faruqui et al., 2018).

The qualitative takeaway is that the Edits model better matches the discourse role of human insertions, whereas the General model tends to produce more generic, syntactically plausible phrases (Faruqui et al., 2018). Error analysis further shows that the general LM is especially good at fluency/discourse insertion locations but worse on refinement edits, while the discriminative edit-trained model does not show the same bias (Faruqui et al., 2018). This is presented as evidence that edit-conditioned training captures semantic information not easily recoverable from raw text alone.

6. Interpretation, limitations, and relation to other Wikipedia corpora

WikiAtomicEdits is best understood as an edit-history corpus, not a general-purpose snapshot of encyclopedic prose. Its release includes the full corpus, POS tags, dependency parses, LM scores for original and edited sentences, and the human annotation subset (Faruqui et al., 2018). Its main value lies in exposing how humans revise, refine, and extend language in context, rather than merely how encyclopedic text looks after revision is complete (Faruqui et al., 2018).

The corpus also has explicit limitations. Deletions are noisier than insertions; in a manual inspection of English examples, 16/100 deletions versus 7/100 insertions contained spam or malformed text (Faruqui et al., 2018). More broadly, the dataset includes Error cases, and the annotation study shows that reproducibility is imperfect. These are not incidental issues: they are part of the empirical profile of revision-history mining.

Within the larger ecosystem of Wikipedia-derived resources, WikiAtomicEdits occupies a distinct niche. DBpedia NIF represents the full content of Wikipedia articles for 128 Wikipedia languages in NIF, preserving article structure, links, and offsets for knowledge extraction (Dojchinovski et al., 2018). WikiConv reconstructs the complete conversational history of Wikipedia talk pages, including additions, modifications, deletions, and restorations, rather than article prose (Hua et al., 2018). Wikipedia Text Reuse: Within and Without compiles reuse cases by comparing all English Wikipedia articles against themselves and against a 10% sample of the Common Crawl, yielding about 100 million within-Wikipedia reuse cases and 1.6 million outside-Wikipedia cases (Alshomary et al., 2018). Harvesting Paragraph-Level Question-Answer Pairs from Wikipedia creates 1,259,691 QA pairs from the 10,000 top-ranking Wikipedia articles using answer extraction and paragraph-level question generation (Du et al., 2018). These resources all derive supervision from Wikipedia, but they formalize different objects: article text, talk-page actions, reuse alignments, QA pairs, and atomic edits.

A plausible implication is that “Wikipedia corpus” is not a single methodological category. In the 2018 research landscape, Wikipedia served as a substrate for several distinct forms of structured supervision. WikiAtomicEdits is the member of that family that turns revision history into a large-scale dataset for studying semantic addition, refinement, discourse behavior, and edit-conditioned representation learning (Faruqui et al., 2018).