---
title: 'MonkeyOCR v1.5: Unified Document Parsing'
url: https://www.emergentmind.com/topics/monkeyocr-v1-5
type: topic
---

# MonkeyOCR v1.5: Unified Document Parsing

MonkeyOCR v1.5 is a unified vision-language framework for robust document parsing, designed to address intricate real-world document layouts with multi-level tables, embedded images or formulas, and cross-page structures. It employs a two-stage “divide-and-conquer” pipeline, leveraging a large multimodal model for global layout understanding and reading order prediction, followed by specialized region-level content recognition. Notable technical advances include a visual-consistency-based reinforcement learning (RL) scheme for complex table parsing, and two modules—Image-Decoupled Table Parsing (IDTP) and Type-Guided Table Merging (TGTM)—for handling embedded images and multi-segment tables. This framework achieves state-of-the-art performance on the OmniDocBench v1.5 benchmark and demonstrates exceptional robustness in visually complex scenarios [2511.10390].

## 1. Two-Stage Unified Parsing Pipeline

MonkeyOCR v1.5 utilizes a two-stage pipeline to decouple the global understanding of document structure from the fine-grained recognition of content within localized regions:

- **Stage I: Layout and Reading-Order Prediction**  
  A single large vision–language model (VLM) accepts the full-page image $I \in \mathbb R^{H\times W\times3}$ along with a layout prompt $p_\text{layout}$, and outputs a sequence $y = \{y_t\}_{t=1}^T$, where each token encodes:
  - Bounding box $(x_1, y_1, x_2, y_2)$
  - Reading order index $i$
  - Category $c \in \{\text{text}, \text{formula}, \text{table}\}$
  - Rotation angle $\alpha$

  The generation follows:
  $$
  p_\theta(y \mid I, p_\text{layout}) = \prod_{t=1}^T p_\theta(y_t \mid y_{<t}, I, p_\text{layout})
  $$
  With constrained decoding enforcing:
  $$
  y_t = \{\text{bbox}:(x_1, y_1, x_2, y_2),\, \text{index}:i,\, \text{label}:c,\, \text{rotation}:\alpha\}
  $$

- **Stage II: Region-Level Content Recognition**  
  For each detected region $y_i=(\mathrm{bbox}_i, \text{label}_i, \alpha_i)$:
  1. **Crop and rotate:**  
     $$
     I_i = \mathrm{Rotate}(\mathrm{Crop}(I, \mathrm{bbox}_i), \alpha_i)
     $$
  2. **Type-specific recognition:**  
     $$
     \hat c_i = 
     \begin{cases}
     \mathcal R_\text{text}(I_i), & \text{if label}_i=\text{text} \\
     \mathcal R_\text{formula}(I_i), & \text{if label}_i=\text{formula} \\
     \mathcal R_\text{table}(I_i), & \text{if label}_i=\text{table}
     \end{cases}
     $$
  3. **Merging for document output:**  
     $$
     \hat Y_\text{doc} = \mathrm{Merge}(\hat c_{\pi(1)}, \ldots, \hat c_{\pi(N)})
     $$
  where $\pi$ denotes ascending index order. This separation enables the VLM to perform both layout reasoning and robust, localized recognition with minimized error propagation.


## 2. Visual-Consistency-Based Reinforcement Learning for Table Parsing

Parsing complex table structures in real-world documents traditionally requires expensive manual annotation. MonkeyOCR v1.5 introduces a self-supervised RL scheme using “render-and-compare” visual consistency:

- **Reward Model Training:**
  1. Construct positive/negative pairs $(\mathrm{GT}, \hat y)$ via perturbation or error sampling.
  2. Render each candidate HTML $\hat y$ to image $I^\mathcal R$ and compare with the gold-crop $I^\mathcal O$.
  3. Train a VLM-based binary reward model:
     $$
     r_\phi(I^\mathcal O, \hat y, I^\mathcal R) \approx 
     \begin{cases}
     +1, & \text{visually consistent} \\
     -1, & \text{inconsistent}
     \end{cases}
     $$

- **Policy Optimization (GRPO):**
  The supervised fine-tuned policy $\pi_\theta(y \mid x)$ is improved over dataset $\mathcal D$ (labeled + unlabeled table crops), maximizing
  $$
  J(\theta) = \mathbb E_{x \sim \mathcal D} \mathbb E_{y \sim \pi_\theta(\cdot|x)} [r_\phi(x, y)]
  $$
  with policy gradient:
  $$
  \nabla_\theta J = \mathbb E_{x, y}\left[ \nabla_\theta\log\pi_\theta(y|x)\, r_\phi(x, y)\right]
  $$
  
This RL-based refinement steers the parser toward table structures where rendered HTML aligns visually with the observed tables, avoiding the need for additional annotation [2511.10390].

## 3. Specialized Modules: IDTP and TGTM

MonkeyOCR v1.5 addresses two longstanding challenges in document parsing via dedicated modules:

- **Image-Decoupled Table Parsing (IDTP):**  
  Many tables contain embedded images that disrupt text-centric parsers. IDTP operates by:
  1. Detecting embedded images in table regions (via YOLOv10).
  2. Replacing each figure with a precise placeholder $<\mathrm{IMG}_k>$, tracking the image–mask mapping.
  3. Running the masked table through the VLM table parser, yielding output HTML with `<img id="k"/>` tokens.
  4. Post-processing reinserts the original images at their positions.

  This decoupling ensures the integrity of textual recognition and layout structure, regardless of non-textual cell contents.

- **Type-Guided Table Merging (TGTM):**  
  For tables split across pages/columns, TGTM applies a pattern-aware merging scheme:
  - Pattern 1: Duplicate headers are dropped and bodies concatenated.
  - Patterns 2 and 3: A BERT-based row-boundary classifier determines if row splits warrant cell merging or straightforward concatenation.
  - Throughout, column indices are realigned, headings normalized, and span conflicts resolved.

  These modules enable parsing of tables that extend over multiple pages/columns and contain images, a limitation in previous OCR systems.

## 4. Empirical Results and Comparative Analysis

MonkeyOCR v1.5 is empirically validated on OmniDocBench v1.5 and public tables benchmarks, utilizing the following metrics:

- **OmniDocBench v1.5 metrics:**  
  - Overall accuracy (%)
  - Text edit distance (lower is better)
  - Formula correctness (Tree-Edit Distance on Math, CDM)
  - Table accuracy (TEDS/TEDS-s; higher is better)
  - Read-order edit score

- **Comparison Table (Table 4 from the report):**

| Model                 | Overall ↑ | Text⁽Edit⁾ ↓ | Formula⁽CDM⁾ ↑ | Table⁽TEDS⁾ ↑ | TEDS-s ↑ | ReadOrder⁽Edit⁾ ↓ |
|-----------------------|:---------:|:------------:|:--------------:|:-------------:|:--------:|:------------------:|
| PPOCR-VL              |   91.9    |    0.039     |     88.7       |     91.0      |   94.9   |      0.048         |
| MinerU 2.5            |   90.7    |    0.047     |     88.5       |     88.2      |   92.4   |      0.044         |
| MonkeyOCR v1.5 (Ours) | **92.9**  |    0.046     |   **91.2**     |   **92.0**    | **95.0** |      0.049         |

Further, on the OCRFlux-Complex subset, MonkeyOCR v1.5 attains 90.9% table recognition accuracy, outperforming both PPOCR-VL and MinerU 2.5 by ~9.2%.

Qualitatively, the system maintains lowest edit distances on dense, multi-column newspaper layouts, reconstructs tables with embedded images accurately, and preserves continuity in cross-page tables [2511.10390].

## 5. Technical and Conceptual Contributions

Key technical and conceptual advancements of MonkeyOCR v1.5 include:

- Streamlined, VLM-based two-stage pipeline for simultaneous layout and reading-order prediction with region-level recognition.
- Visual-consistency-based RL leveraging unlabeled data for table parsing, obviating the expense of new ground-truth annotation.
- Specialized modules (IDTP, TGTM) for reliable parsing of tables with embedded images and the reconstruction of cross-segment tables.
- Superior performance on challenging document benchmarks compared to existing OCR baselines, particularly in visually intricate scenarios.

These contributions mark a substantial advance in document parsing for complex, real-world layouts.

## 6. Limitations and Prospective Extensions

The technical report does not enumerate explicit limitations. A plausible implication is that latency and multimodality coverage could be practical concerns. Suggested future directions include:

- Extending the visual-consistency RL framework to formulas and mixed-content document regions.
- Reducing inference latency to enable real-time applications.
- Scaling the system to support additional languages and highly ornate layouts.
- Investigating deeper end-to-end integration of global and local parsing stages [2511.10390].

In summary, MonkeyOCR v1.5 advances the frontier in vision-language document parsing, particularly in handling complex tabular structures and visually multifaceted documents.

Source: https://www.emergentmind.com/topics/monkeyocr-v1-5