MonkeyDoc v2: Document Pretraining Corpus

• MonkeyDoc v2 is a multilingual document-image pretraining corpus containing 113M visual–text samples across 17 languages, designed for document-native learning.
• It combines page-level images and cropped document elements from both real and synthetic sources to capture diverse layouts, fine-grained text, and multilingual scripts.
• Its dual-objective regime optimizes image-to-text generation and pixel-level document reconstruction, leading to notable improvements in OCR, detection, parsing, and document understanding.

Searching arXiv for the cited MonkeyOCRv2 paper and a few closely related document-AI baselines to ground the article. MonkeyDoc v2 is a multilingual, multi-type document-image pretraining corpus introduced as the sole pretraining corpus for the MonkeyOCRv2 visual encoder family in “MonkeyOCRv2: A Visual-Text Foundation Model for Document AI” (Liu et al., 13 Jul 2026). It contains 113 million visual–text samples spanning 17 languages, with each sample defined as either a full document page or a cropped document element paired with textual supervision. Its stated purpose is to teach a vision encoder to resolve dense text, fine-grained character strokes, and document layouts at character level rather than relying on visual representations inherited from natural-image pretraining. Within MonkeyOCRv2, MonkeyDoc v2 underwrites a dual-objective pretraining regime that jointly optimizes image-to-text generation and pixel-level document reconstruction, thereby supplying both semantic labels and document-native structural variation for downstream OCR, detection, parsing, and document understanding tasks (Liu et al., 13 Jul 2026).

1. Definition, scope, and role

MonkeyDoc v2 is defined as a multilingual, document-image pretraining corpus containing 113 million visual–text samples, where a sample is either a full page or a cropped document element such as a text line, table, or formula paired with supervision (Liu et al., 13 Jul 2026). The corpus is described as the sole pretraining corpus for MonkeyOCRv2 encoders, and its role is explicitly tied to a dual training objective: image-to-text generation for alignment between visual tokens and textual content, and pixel-level document reconstruction for preserving stroke-level and layout information (Liu et al., 13 Jul 2026).

This design positions MonkeyDoc v2 as a document-native alternative to mainstream visual pretraining regimes based on natural images. The motivating claim is that mainstream visual encoders are pretrained on natural images and “cannot be effectively applied to document images without document-oriented adaptation,” because documents require perception of dense text and fine character strokes rather than coarse object-level semantics (Liu et al., 13 Jul 2026). In that framing, MonkeyDoc v2 is not merely a benchmark collection but a foundation corpus for learning representations specialized for document intelligence.

A plausible implication is that MonkeyDoc v2 is intended to collapse several previously separated supervision regimes—OCR labels, layout annotations, table and formula content, multilingual script coverage, and synthetic edge-case generation—into a single pretraining substrate. The paper’s downstream results are presented as evidence that such unified, document-oriented pretraining can function “as a foundation for document intelligence in its own right” (Liu et al., 13 Jul 2026).

2. Scale, language coverage, and document composition

MonkeyDoc v2 comprises 113M total samples across 17 languages, counting Simplified and Traditional Chinese separately (Liu et al., 13 Jul 2026). Its granularity is explicitly split into 8M page-level images and 105M cropped document elements, with the term “sample” defined to include either kind of unit (Liu et al., 13 Jul 2026). This mixed granularity is central to the corpus design: page-level samples preserve global layout and reading order, whereas cropped elements concentrate supervision on local regions such as lines, words, table cells, and formulas.

The 17 languages are listed as Simplified Chinese, Traditional Chinese, English, Arabic, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Dutch, Portuguese, Russian, Thai, and Vietnamese (Liu et al., 13 Jul 2026). The language distribution is described as dominated by English (~19M samples) and Chinese (~13M Simplified + Traditional), followed in descending scale by Dutch, Portuguese, Russian, French, Indonesian, Spanish, Hindi, German, Japanese, Italian, Thai, Korean, Arabic, and Vietnamese (Liu et al., 13 Jul 2026). The corpus is therefore strongly English–Chinese in mass while still covering Latin, CJK, Arabic, Devanagari, Thai, and Vietnamese scripts.

MonkeyDoc v2 covers a broad spectrum of document types, including printed papers and scientific articles, scanned books, handwritten notes, newspapers and magazines, financial reports, rendered and digital-born documents such as PDF pages and slides, visually rich documents such as forms, tables, and infographics, scene or “in-the-wild” text, and both LaTeX-rendered and real-world formulas and tables (Liu et al., 13 Jul 2026). Representative examples are reported to include PDFs, invoices, forms, slides, manuals, receipts, web pages, and photographed documents across scripts (Liu et al., 13 Jul 2026).

The corpus deliberately mixes real and synthetic data. The stated split is 61M real samples (54%) and 52M synthetic samples (46%) (Liu et al., 13 Jul 2026). Real data is used to preserve realistic layouts, scanning and photographic noise, and document artifacts, while synthetic data is used to expand multilingual coverage, cover rare characters and low-frequency symbols, and generate diverse tables and formulas with controlled structures (Liu et al., 13 Jul 2026). This suggests a corpus design in which realism and coverage are treated as complementary rather than competing objectives.

3. Data sources and construction pipeline

MonkeyDoc v2 is built through what the paper terms a “data engine” with three core modules: Expert Model Labeling, Multilingual Corpus-Based Data Synthesis, and Data Filtering (Liu et al., 13 Jul 2026). Real-world documents are collected from the official training splits of many public datasets, including FinePDFs, MonkeyDoc (v1), Union14M, UniMER‑1M, CDLA, D4LA, DocGenome, DocLayNet, M6Doc, SVRD, TabRecSet, COCOTextV2, HierText, DSTextV2, LSVT, OpenImagesV5Text, ReCTS, TextOCR, and MTWI, along with large collections of newspapers, handwritten notes, and presentation slides gathered by the authors (Liu et al., 13 Jul 2026). Synthetic data is generated from multilingual LLM corpora, Unicode character sets per language, and formulas crawled from arXiv, producing approximately 0.8M formula image–LaTeX pairs (Liu et al., 13 Jul 2026).

For real documents, annotation is performed using a multi-expert agreement pipeline rather than a single OCR model. The process first applies a layout detection model to segment each page into text blocks, tables, formulas, and “other” regions; then runs several complementary OCR or expert recognizers on each cropped element; then computes pairwise similarity among predictions; and finally chooses the prediction with the highest average agreement across experts as the label (Liu et al., 13 Jul 2026). This produces bounding boxes and region types from layout detection, per-region text transcription from multi-expert OCR agreement, and reading order for page-level data (Liu et al., 13 Jul 2026).

Synthetic generation proceeds by sampling multilingual text from LLM corpora, rendering it with diverse fonts, styles, and resolutions, constructing full character sets per language to generate random strings that cover rare glyphs, generating table templates populated with multilingual text, and rendering formulas collected from arXiv (Liu et al., 13 Jul 2026). The paper emphasizes that synthetic data comes with perfect textual labels and controlled structural patterns, complementing the noise and variability of real documents.

Quality control is implemented through two LLM-based filters. The first is a layout completeness check: all detected regions are masked with white blocks, the masked page is fed into a strong document-oriented VLM, and if the model still reads residual text, the page is discarded because the detector likely missed regions (Liu et al., 13 Jul 2026). The second is a reading-order consistency check: recognized text is concatenated according to annotated reading order, an LLM evaluates whether the sequence is logically consistent and free of reading-order errors, and inconsistent samples are discarded (Liu et al., 13 Jul 2026). From an initial 1.2M pages drawn from particularly challenging sources, approximately 0.9M are retained after these filters (Liu et al., 13 Jul 2026).

The paper states that all validation and test splits of downstream benchmarks are excluded from pretraining, fine-tuning, and hyperparameter search to avoid contamination (Liu et al., 13 Jul 2026). It does not discuss deduplication explicitly, and it does not provide a detailed legal or ethical analysis, though it notes that the listed datasets are public research datasets, that only official training splits are used, and that synthetic data is generated from publicly available corpora and Unicode tables (Liu et al., 13 Jul 2026).

4. Pretraining objectives and representation learning function

MonkeyDoc v2 is the exclusive training corpus for MonkeyOCRv2 encoder variants including ViT-S, ViT-B, and ViTAEv2-S (Liu et al., 13 Jul 2026). The pretraining architecture contains a vision encoder $E_v$, a vision decoder $D_v$ used for reconstruction, and a text decoder $D_t$ used for generation. Given an input image $I \in \mathbb{R}^{H\times W \times 3}$, the encoder produces visual tokens $\mathbf{z} = E_v(I)$, the vision decoder reconstructs $\hat{I} = D_v(\mathbf{z})$, and the text decoder generates supervised text autoregressively from $\mathbf{z}$ plus task prompts (Liu et al., 13 Jul 2026).

The default pixel-level reconstruction loss is given as

$\mathcal{L}_{\mathrm{pix} = \frac{1}{3HW}\left\|\hat{\mathbf{I} - \mathbf{I}\right\|_2^2$

and is used to force the encoder to preserve enough information for the decoder to rebuild the input document image, including small glyphs and layout details (Liu et al., 13 Jul 2026). A structure-aware variant used in document understanding experiments augments this with edge- and distance-to-edge consistency through soft edge maps $\mathcal{E}(\mathbf{I})$, truncated distance-to-edge maps $\mathcal{D}(\mathbf{I})$, and a structure-matching loss

$D_v$0

with full reconstruction objective

$D_v$1

where $D_v$2 and $D_v$3 (Liu et al., 13 Jul 2026).

The text generation objective is standard autoregressive cross-entropy, denoted $D_v$4, over page-level reading order, line-level content, formulas, table cell text, and related supervised text targets (Liu et al., 13 Jul 2026). The overall pretraining objective is

$D_v$5

with $D_v$6 fixed (Liu et al., 13 Jul 2026). In the authors’ interpretation, $D_v$7 aligns visual tokens with semantic text content, while $D_v$8 ensures retention of fine-grained visual evidence such as strokes, punctuation, and layout structures that a purely semantic objective might discard (Liu et al., 13 Jul 2026).

To support character-level perception, the pretraining regime uses dynamic resolution training with variable-length tokenization, following the design pattern of Qwen2.5-VL, so that the number of visual tokens adapts to input resolution (Liu et al., 13 Jul 2026). Patch sizes are $D_v$9 for MonkeyOCRv2-S/B with maximum pixel budget approximately 1,003,520, and $D_t$0 for MonkeyOCRv2-AS with maximum pixel budget approximately 1,802,240 (Liu et al., 13 Jul 2026). The combination of 8M page-level samples and 105M crops is presented as a mechanism for simultaneously preserving global structure and exposing tiny text at high effective resolution (Liu et al., 13 Jul 2026).

5. Covered document phenomena and multilingual challenges

MonkeyDoc v2 is designed to cover the principal challenges of document vision: dense text and small fonts; complex layouts including multi-column text, sidebars, headers, footers, and mixed figure/table regions; multilingual and mixed-script documents; formulas and structured expressions; tables and spreadsheets with merged cells and complex borders; overlapping or occluded text; handwriting; and photographed pages with distortion, blur, and reflections (Liu et al., 13 Jul 2026).

The multilingual design is explicitly tied to script coverage and character inventory completeness. The dataset includes Latin-based languages, CJK scripts, Arabic, Devanagari for Hindi, Thai, and Vietnamese, and it deliberately extracts full character sets per language and generates random character strings to ensure coverage of rare characters and diacritics (Liu et al., 13 Jul 2026). The paper identifies Arabic contextual forms, Thai vowel signs, Vietnamese diacritics, and CJK ideographs as cases where such coverage is especially important (Liu et al., 13 Jul 2026).

The reported downstream multilingual parsing results on MDPBench are 84.2 for Latin languages and 82.1 for non-Latin languages using MonkeyOCRv2-B-Parsing, with the paper remarking that the gap is small despite heavy-tail languages (Liu et al., 13 Jul 2026). Qualitative examples are described as showing improved parsing of Arabic right-to-left multi-column documents and Chinese photographed manuals relative to PaddleOCR-VL-1.6 and dots.mocr (Liu et al., 13 Jul 2026). This suggests that MonkeyDoc v2’s multilingual balance, while uneven in sample counts, is sufficient to support broad cross-script transfer.

A common misconception would be to treat MonkeyDoc v2 simply as a larger OCR dataset. The corpus is described instead as multi-type and layout-rich, spanning page-level reading order, local element recognition, formulas, tables, photographed documents, handwriting, and scene text (Liu et al., 13 Jul 2026). Its function is therefore not restricted to transcription accuracy but extends to layout-sensitive and visually subtle tasks such as tampering detection and overlapping text segmentation.

6. Empirical impact and comparison with earlier corpora

The paper attributes a broad set of downstream gains to encoders pretrained solely on MonkeyDoc v2 (Liu et al., 13 Jul 2026). In text recognition, replacing standard backbones with MonkeyOCRv2-S raises CRNN overall accuracy across Union14M benchmark, a Chinese benchmark, and occluded scene text from 58.7% to 67.3%, and raises PARSeq overall accuracy from 82.2% to 84.3%; on Union14M-Benchmark, PARSeq improves from 86.4% to 87.6%, surpassing prior SOTA SVTRv2 at 86.1% (Liu et al., 13 Jul 2026). In formula recognition, UniMERNet-T with MonkeyOCRv2-S improves OmniDocBench 1.6 formula-subset CDM from 89.9 to 90.8 and ExpRate from 57.2 to 61.1, MathWriting CDM from 65.6 to 70.8, and MathWriting ExpRate from 12.9 to 16.2 (Liu et al., 13 Jul 2026).

In text detection, replacing ImageNet-pretrained backbones or oCLIP with MonkeyOCRv2-AS yields consistent F-measure gains across ICDAR 2015, ArT, Total-Text, and CTW1500; one cited example is DBNet on ICDAR2015 improving from 85.0 to 88.5 (Liu et al., 13 Jul 2026). In document tampering detection, swapping ConvNeXt-V2 or DeepSolo-backbone to MonkeyOCRv2-AS inside FFDN on DocTamper raises IoU from 69.4 to 87.4 and F1 from 82.0 to 93.3 on DocTamper-Test (Liu et al., 13 Jul 2026). In overlapping text segmentation on the MOT dataset, Mask2Former mIoU$D_t$1 increases from 70.3 to 76.6, and MOTS mIoU$D_t$2 from 72.6 to 76.9 (Liu et al., 13 Jul 2026).

The most prominent system-level result is in document parsing. MonkeyOCRv2-B-Parsing, a 0.7B model with 0.1B vision encoder, attains 83.3 on MDPBench, surpassing dots.mocr (80.5) and PaddleOCR-VL-1.6 (75.0) while using a vision encoder “about 11× smaller” than the previous best open-source dots.mocr (Liu et al., 13 Jul 2026). In controlled document understanding experiments, a frozen MonkeyOCRv2-B encoder paired with Qwen3-1.7B yields an average 57.2 across eight document VQA benchmarks, outperforming CLIP, SigLIP2, DINOv3, SAM/SAM2, oCLIP, DiT, OpenVision-B, and RADIOv2.5-B under identical training settings (Liu et al., 13 Jul 2026).

The paper compares MonkeyDoc v2 with earlier document-oriented corpora. It reports: oCLIP at 450K scene-text images in 2 languages; DiG at 35.6M scene-text images in 1 language; DiT (IIT-CDIP) at 42M scanned documents in 1 language with no fine-grained text annotations; Donut at 13M scanned or synthetic documents in 4 languages; Pix2Struct at 80M web screenshots in 1 language; and MonkeyDoc v2 at 113M multi-type documents in 17 languages with dense text annotations (Liu et al., 13 Jul 2026). On that basis, MonkeyDoc v2 is described as larger, more diverse in type and script, and more richly annotated than prior open document corpora known to the authors (Liu et al., 13 Jul 2026).

Relative to MonkeyDoc (v1), the paper states that MonkeyDoc v2 expands from a smaller, more limited corpus to 113M samples, broadens language coverage from primarily Chinese and English to 17 languages, expands document types to include scanned and digital documents, handwritten notes, newspapers, and slides, and shifts from mainly recognition and spotting data to a unified corpus supporting layout, recognition, tables, formulas, and document parsing (Liu et al., 13 Jul 2026). This suggests a transition from a task-oriented corpus toward a foundation-style pretraining resource.

7. Availability, reconstruction, and broader significance

The paper states that “Code and data will be released at https://github.com/Yuliang-Liu/MonkeyOCRv2” (Liu et al., 13 Jul 2026). It also notes that “data” in practice may mean training recipes for synthetic generation, lists of public datasets and usage, and possibly partial or derived annotations, since some underlying corpora have their own licenses and re-hosting the full aggregate may be constrained (Liu et al., 13 Jul 2026). The source enumeration and pipeline description are sufficiently explicit that practitioners can reconstruct a similar corpus by reusing the same public training splits, implementing a layout plus multi-expert OCR labeling pipeline, synthesizing multilingual text and tables from LLM corpora and Unicode sets, and applying layout-completeness and reading-order-consistency checks (Liu et al., 13 Jul 2026).

The paper advises respecting the licenses of underlying datasets, excluding validation and test splits of planned evaluation benchmarks, and being mindful of bias in OCR models and LLM-based filters if replicating the expert-labeling pipeline (Liu et al., 13 Jul 2026). In practical terms, it also suggests that the easiest route to benefit from MonkeyDoc v2 is to use MonkeyOCRv2’s pretrained encoders or the MonkeyOCRv2-Parsing model as drop-in backbones or frozen VLM vision modules (Liu et al., 13 Jul 2026).

In the broader document-AI landscape, MonkeyDoc v2 is presented as an answer to a longstanding dependency on natural-image encoders such as CLIP, DINO, SAM, and related families for document tasks (Liu et al., 13 Jul 2026). The central claim is that dense, multilingual, document-native supervision can produce a visual encoder specialized for reading documents rather than objects and scenes. The reported gains across recognition, formula understanding, detection, tampering detection, overlapping text segmentation, parsing, and document understanding are used to support that claim (Liu et al., 13 Jul 2026).

A plausible implication is that MonkeyDoc v2 marks a shift in document AI from downstream adaptation of generic vision backbones toward corpus-scale pretraining explicitly optimized for document statistics. On the evidence presented, its significance lies less in any single benchmark result than in establishing a corpus-and-objective template for document-native foundation models (Liu et al., 13 Jul 2026).