XDocParse: Multilingual Layout Benchmark

• XDocParse is a large-scale, multilingual benchmark built for end-to-end document layout parsing that rigorously evaluates detection, recognition, and relational reasoning tasks using diverse real-world documents.
• Its comprehensive annotation schema includes bounding boxes, category labels, recognized text, and reading-order indices, enabling joint geometric, semantic, and structural evaluations.
• Empirical results show that unified models like dots.ocr achieve state-of-the-art performance with significantly lower OverallEdit errors and improved metrics across text, formula, and table analyses.

XDocParse is a large-scale, multilingual benchmark specifically constructed for end-to-end document layout parsing, rigorously evaluating model performance across detection, recognition, and relational reasoning tasks within real-world documents spanning an unprecedented 126 languages. Originating alongside the introduction of dots.ocr—a vision-LLM demonstrating unified, end-to-end document parsing—XDocParse represents an exhaustive evaluation platform designed to measure a model’s capacity to detect layout elements, recognize content, and preserve document structure across highly varied scripts and domains (Li et al., 2 Dec 2025).

1. Benchmark Composition and Annotation Schema

XDocParse is constructed from thousands of real-world documents, including academic papers, business reports, scanned forms, magazines, and receipts. Its scope encompasses Latin, Cyrillic, CJK (Chinese, Japanese, Korean), Arabic, Devanagari, and multiple low-resource scripts.

Every document receives comprehensive block-level annotation, comprising:

• Bounding-box coordinates $\mathcal B_k=(x_{k,1},y_{k,1},x_{k,2},y_{k,2})$: defining the spatial extent of each segment.
• Category label $c_k$: among classes such as title, header, paragraph, table, figure, formula, caption, footer, and others.
• Recognized text $t_k$: plain UTF-8 for unstructured blocks, LaTeX markup for tables and equations.
• Reading-order index $r_k \in \{1, \ldots, K\}$: explicitly encoding sequence for human-like text flow.

Blocks are unified into an autoregressive sequence:

$$\mathbf S = \bigl[ (\mathcal B_{r_1}, c_{r_1}, t_{r_1}), \ldots, (\mathcal B_{r_K}, c_{r_K}, t_{r_K}) \bigr]$$

This format compels models to reason jointly about geometry, semantics, and logical order.

2. Formal Task Definitions

XDocParse evaluates three principal sub-tasks, each with precise mathematical and procedural definitions:

1. Layout Detection:

Models predict bounding boxes $\{\hat{\mathcal B}_k\}$. Training leverages an $\ell_1$ or Smooth-$\ell_1$ regression loss:

$$\mathcal L_{\text{det}} = \sum_{k=1}^K \text{SmoothL}_1(\hat{\mathcal B}_k - \mathcal B_k)$$

At evaluation, predicted and ground-truth boxes are paired via Hungarian matching based on IoU (intersection over union), followed by precision, recall, and $F_1$ score calculations. An IoU threshold $c_k$0 applies, with a two-stage matching protocol utilizing adjacency-based clustering to address block fragmentation.

2. Text Recognition:

Each block’s text is predicted as a sequence of tokens $c_k$1, with cross-entropy loss:

$c_k$2

Performance is assessed by normalized edit distance (TextEdit):

$c_k$3

3. Relational Understanding (Reading Order):

Models must generate blocks in the human reading order, learned through a joint generation loss over “flattened” tuples:

$c_k$4

Reading order quality is evaluated via normalized edit distance on the ordered block index sequence.

3. Multilingual Data Synthesis Pipeline

XDocParse’s diversity and annotation quality are underpinned by a scalable, three-stage data engine:

• Stage 1: Teacher-Student Synthesis:

A large vision-LLM (Qwen2.5-VL-72B) acts as “teacher,” ingesting English PDF + structure and generating semantically equivalent layouts in target languages, preserving layout. These are rendered as seed image + annotation pairs. A smaller “student” VLM (Qwen2.5-VL-7B) is fine-tuned on these seeds, supporting fast, layout-aware auto-labeling.

• Stage 2: Curated Pre-training Corpus:

The student auto-labels millions of PDFs, stratified by layout complexity (column count, table/formula density), domain rarity, and language. Layout and language outliers (complex tables, niche scientific domains) are intentionally over-sampled to enhance representational coverage.

• Stage 3: Human-in-the-Loop Correction:

The pre-trained dots.ocr model is run on a held-out set. Error cases (mislocalizations, hallucinations, recognition errors) are mined by an instruct-tuned oracle and corrected by annotators, yielding 15,000 high-signal samples targeting model failure modes.

4. Evaluation Protocol and Metrics

Assessment follows the OmniDocBench protocol for comprehensive end-to-end parsing, with the following key metrics:

Metric	Description
OverallEdit	Normalized edit distance across the entire $c_k$5 sequence (geometry, order, and semantics)
TextEdit	Normalized edit distance for plain text blocks
FormulaEdit	Normalized edit distance for LaTeX-encoded formulas/written math
TableEdit	Normalized edit distance for LaTeX-encoded tables
TableTEDS	Tree-edit distance between predicted and ground-truth table structures
ReadingOrderEdit	Edit distance for reading order (block index sequence)

Geometric scores employ the two-stage $c_k$6 metric at IoU $c_k$7, while text and structural metrics rely on standard Levenshtein normalization.

5. Empirical Results and Baseline Comparisons

Baseline performance on XDocParse demonstrates the benchmark’s difficulty and the advantage of unified modeling. The following table summarizes end-to-end and component metrics:

Method	OverallEdit ↓	TextEdit ↓	FormulaEdit ↓	TableTEDS ↑	TableEdit ↓	ReadingOrderEdit ↓
MonkeyOCR-3B	0.483	0.445	0.627	50.93	0.452	0.409
doubao-1-5-thinking-vision	0.291	0.226	0.440	71.20	0.260	0.238
doubao-1-6	0.299	0.270	0.417	71.00	0.258	0.253
Gemini-2.5-Pro	0.251	0.163	0.402	77.10	0.236	0.202
dots.ocr (ours)	0.177	0.075	0.297	79.20	0.186	0.152

dots.ocr attains an OverallEdit of 0.177, a 0.074 (≈7.4 point) absolute reduction versus Gemini-2.5-Pro, more than halving TextEdit (0.075 vs 0.163) and improving structure-sensitive metrics (e.g., TableTEDS 79.20 vs 77.10). This establishes a new state-of-the-art baseline (Li et al., 2 Dec 2025).

6. Challenges, Insights, and Representative Case Studies

Key challenges in XDocParse include extreme script diversity, layout variability, and the scarcity of ground-truth annotations for low-resource languages. The benchmark exposes the generalization limits of traditional pipeline approaches, as fragmentation of structural blocks and misalignment in reading order frequently occur in non-unified systems. Only a unified training paradigm, where detection, recognition, and relational reasoning are learned jointly, captures the hierarchy and flow inherent to real documents. This is evidenced by ablation studies: removing detection impairs reading order inference, while omitting recognition leads to severe structural fragmentation, despite nominal detection mAP gains.

Representative case studies illustrate the benchmark’s rigor:

• In a four-column Arabic (RTL) engineering document, baseline models misorder paragraphs and split formulas, while a unified model correctly localizes, recognizes, and generates right-to-left sequences.
• When confronted with multiple small tables per page, models lacking reading-order supervision collapse them into a single detection, but joint modeling preserves their independence and order.
• In chemical diagram layouts, omitting the recognition objective causes detectors to over-segment, whereas the unified approach maintains coherence at the figure level.

Together, these results and analyses demonstrate that XDocParse is an exceptionally challenging, diverse benchmark for multilingual document parsing, motivating the development of models capable of unified geometric, semantic, and logical reasoning (Li et al., 2 Dec 2025).