dots.ocr: Unified Document Layout Parsing

• dots.ocr is a unified vision-language model for document layout parsing that integrates layout detection, text recognition, and relational reading order analysis in an end-to-end framework.
• It utilizes a high-resolution Vision Transformer and language model decoder to fuse visual tokens and text features, achieving superior performance on benchmarks like OmniDocBench and XDocParse.
• Its multilingual data synthesis engine, combining teacher-student bootstrapping and human-in-loop corrections, enables robust parsing of over 10M pages across 126 languages.

dots.ocr is a unified vision-LLM for document layout parsing, architected to jointly solve layout detection, text recognition, and relational (reading order) understanding within a single, end-to-end framework. By integrating these tasks and leveraging a multilingual, large-scale data synthesis engine, dots.ocr establishes new standards of performance and generalization in multilingual document analysis, as validated on the comprehensive OmniDocBench and the newly introduced XDocParse benchmark spanning 126 languages (Li et al., 2 Dec 2025).

1. Unified Architecture and Task Synergy

dots.ocr employs a high-resolution Vision Transformer–LLM (ViT-LLM) backbone, consisting of a vision encoder (VE, 1.2B parameters, trained from scratch on document data) and a LLM decoder (LM, Qwen2.5-1.5B, 1.7B parameters, with tied embeddings). The vision encoder produces a sequence of fine-grained visual tokens, while the decoder autoregressively generates a structured output sequence $S = [ (B_1, c_1, t_1),\ldots,(B_K, c_K, t_K) ]$ where $B_k$ are bounding-box coordinates, $c_k$ is the region class (title, paragraph, table, figure, etc.), and $t_k$ is either plain text or LaTeX for structured regions. Cross-modal fusion is realized by concatenating the VE token embeddings to the LM input at each decoding step. The decoder’s self- and cross-attention layers enable joint reasoning over vision and language modalities.

The model is optimized by minimizing a weighted sum of three losses:

$$\mathcal{L}_{\text{total}} = \lambda_{\text{layout}} \mathcal{L}_{\text{layout}} + \lambda_{\text{recog}} \mathcal{L}_{\text{recog}} + \lambda_{\text{rel}} \mathcal{L}_{\text{rel}}$$

where $\mathcal{L}_{\text{layout}}$ is an $\ell_1+\mathrm{GIoU}$ loss over bounding boxes, $\mathcal{L}_{\text{recog}}$ is cross-entropy for text/LaTeX generation, and $\mathcal{L}_{\text{rel}}$ is a sequence order loss computed as the edit distance on reading order labels. Empirical results show that equal task weighting is robust, with modest performance variation (<0.3%) under changes (Li et al., 2 Dec 2025).

Ablation studies reveal strong synergy from joint training: removing detection increases reading order errors by 35%, and randomizing order destroys geometric understanding (detection F1 decreases from 0.822 to 0.738). Unified training alone outperforms single-task models, confirming that multi-task learning yields richer contextual representations.

2. Multilingual Data Engine and Synthesis

The training of dots.ocr relies on a three-stage scalable data engine that produces over 10 million labeled document pages across 126 languages. The workflow comprises:

1. Teacher-Student Bootstrapping: An LLM (Qwen2.5-VL-72B) translates and re-renders English documents into target languages, preserving structure. Its outputs seed the training of a student model (Qwen2.5-VL-7B), which becomes a fast auto-labeler.
2. Curated Large-Scale Pre-training: Documents are stratified by layout complexity, domain, and language rarity. The student model is used to auto-label approximately 8M pages, with oversampling of rare structural types and balanced language sampling $P(\ell)\propto N_{\text{pages}}(\ell)^{0.8}$ to avoid high-resource language bias.
3. Human-in-the-Loop Correction: A 30K-page audit set is analyzed by an oracle model, which flags high-confidence annotation errors. Human annotators correct these, yielding a set of 15K high-signal samples. The final supervised fine-tuning uses 300K combined samples from all stages.

Ablation results demonstrate that removing the multilingual pillar or human-in-the-loop correction causes marked drops in detection, reading order, and overall accuracy, underscoring the necessity of both scale and annotation quality in achieving robust cross-lingual performance (Li et al., 2 Dec 2025).

3. Benchmarks, Metrics, and Performance

dots.ocr is evaluated on OmniDocBench (English and Chinese) and the XDocParse benchmark (126 languages), employing metrics such as OverallEdit, TextEdit, TableTEDS, and ReadingOrder. It establishes state-of-the-art results on OmniDocBench:

Model Type	OverallEdit_EN	OverallEdit_ZH	TextEdit_EN	TextEdit_ZH	TableTEDS_EN	TableTEDS_ZH	ReadingOrder_EN	ReadingOrder_ZH
MinerU	0.150	0.357	0.061	0.215	78.6	62.1	0.079	0.292
MonkeyOCR-3B	0.138	0.206	0.067	0.107	81.5	87.5	0.100	0.185
Gemini2.5-Pro	0.148	0.212	0.055	0.168	85.8	86.4	0.049	0.121
dots.ocr	0.125	0.160	0.032	0.066	88.6	89.0	0.040	0.067

On XDocParse, dots.ocr registers an OverallEdit of 0.177, outperforming Gemini2.5-Pro by +0.074 (7.4 points), with consistent superiority in TextEdit, TableTEDS, and ReadingOrder across all 126 tested languages (Li et al., 2 Dec 2025).

4. Impacts of Corpus Size, Task Coupling, and Data Quality

Incremental data scaling shows logarithmic performance gains: corpus doubling from 2M to 4M pages yields ∼5% relative improvement in OverallEdit; further doubling to 8M adds ~3%, with diminishing returns beyond that point. Synergy ablations confirm that geometric and reading order reasoning are tightly coupled: omission of detection or correct ordering substantially degrades structural understanding.

Data ablation demonstrates that the absence of HITL correction substantially harms detection F1 (from 0.849 to 0.788), and omission of structured (tables/formulas) or multilingual data impairs both overall and language-specific metrics, particularly in low-resource scripts.

5. Practical Implications and Deployment Considerations

dots.ocr supports high-resolution inputs (≤11MP), mitigating failure on dense technical or scientific documents. Its single-pass, unified inference architecture yields lower latency compared to multi-stage pipelines and enables consistent operation across 126 languages, facilitating large-scale, global enterprise automation.

Nonetheless, challenges persist: rare scripts (complex Indic or African) present 10–15% higher error rates, indicating a need for further targeted synthesis. OCR on mixed-script regions requires improved aligning mechanisms, prompting investigation into script-aware adapter architectures (Li et al., 2 Dec 2025).

6. Future Directions and Research Opportunities

Several avenues are identified for advancing the unified VLM paradigm:

• Relational Understanding: Move beyond reading order to richer relations, such as table schema inference and cross-document entity linking.
• Vision-Language Data Generation: Exploit the data engine for producing diverse multimodal corpora (e.g., image-caption, text inpainting, next-page prediction) to pre-train generalist VLMs.
• Extending Modalities: Adapt the framework to multimodal documents (e.g., with embedded audio or video) for comprehensive, unified parsing.
• Low-Resource Script Specialization: Fine-grained data augmentation and domain adaptation to further reduce performance gaps in rare scripts.

In summary, dots.ocr demonstrates that a single, end-to-end vision-LLM—supported by a scalable, multilingual data engine—can supersede fragmented or single-task OCR pipelines, setting new empirical baselines for global document layout parsing and structured document intelligence (Li et al., 2 Dec 2025).