OCR-Agent: Iterative Self-Correction Framework

• OCR-Agent is a training-free iterative self-correction framework for vision-language models that improves OCR tasks by integrating structured self-reflection.
• Its Capability Reflection mechanism filters executable actions, preventing hallucinated corrections and ensuring only feasible steps are applied.
• Memory Reflection stores past reflections to avoid repetitive errors, leading to stable improvements in OCR benchmarks across English and Chinese.

OCR-Agent is a training-free iterative self-correction framework for large vision-LLMs (VLMs) designed to improve OCR-related understanding, localization, extraction, and reasoning without fine-tuning. It was introduced to address a recurrent failure pattern in multimodal OCR settings: during multi-turn revision, VLMs often lack effective self-correction mechanisms, propose actions beyond their actual abilities, and fall into repetitive correction loops rather than achieving stable improvement. OCR-Agent addresses these problems through two coupled mechanisms—Capability Reflection and Memory Reflection—within a structured reflection-refinement loop. On OCRBench v2, the framework is reported to outperform the open-source SOTA model InternVL3-8B by +2.0 on the English subset and +1.2 on the Chinese subset, while attaining state-of-the-art results in Understanding and Reasoning without additional training (Wen et al., 24 Feb 2026).

1. Conceptual setting and problem scope

OCR-Agent is motivated by the claim that OCR tasks are no longer limited to transcription. On OCRBench v2, many examples require structured extraction, parsing, visual question answering, counting, and long-form reasoning across Chinese and English. In that setting, standard prompting and generic self-reflection are reported to be insufficient, because the model may either hallucinate infeasible actions or repeatedly apply the same ineffective strategy across refinement rounds (Wen et al., 24 Feb 2026).

A central misconception addressed by the framework is that more iteration alone necessarily yields better OCR reasoning. The paper argues the opposite: unconstrained refinement can become unstable. In ordinary self-refinement, the model may suggest actions such as “zoom in,” “enhance the image,” or “ask a human to check,” even though it cannot actually execute those operations. The paper terms this failure mode capability hallucination. A second failure mode is refinement stagnation, in which the model keeps revisiting the same unsuccessful line of reasoning.

OCR-Agent is therefore organized around a narrower objective than generic “thinking harder.” It attempts to keep the model inside its executable reasoning boundary while still allowing iterative answer improvement. The method is described as making refinement both capability-aware and memory-aware, so that the model diagnoses its own mistakes, plans only feasible fixes, avoids repeating prior failures, and re-reasons toward a better answer.

2. Capability Reflection

Capability Reflection is the component that filters correction plans according to what the model can actually execute. The paper formalizes a proposed correction plan as

$\mathcal{P}=\{a_1,a_2,\ldots,a_n\},$

with a feasibility indicator over each action $a$:

$$\phi(a)= \begin{cases} 1 & \text{if } a \text{ can be executed by the model} \ 0 & \text{if } a \text{ is infeasible.} \end{cases}$$

The feasible plan is then defined as

$$\mathcal{P}_{\text{feas}}=\{a\in\mathcal{P}:\phi(a)=1\},$$

and the refined answer is generated by

$$y_{\text{new}} = R(x, y_{\text{prev}}, \mathcal{P}_{\text{feas}}).$$

The intended interpretation is operational rather than abstract. The model first produces a chain-of-thought-style correction plan, but only those steps that can be performed through reasoning or text-based manipulation are retained. Actions such as image enhancement or human proofreading are excluded because they are not executable by the model within the current inference loop (Wen et al., 24 Feb 2026).

This mechanism is not presented as external tool use. Rather, it is an internal constraint on self-reflection. Its function is to keep the plan grounded in the actual affordances of the base VLM. In the paper’s formulation, this reduces the tendency of a reflective model to generate attractive but unusable revisions.

3. Memory Reflection and iterative refinement

Memory Reflection addresses the second failure mode, namely repeated and ineffective correction. Instead of treating each refinement step independently, OCR-Agent stores prior reflections and reuses them in later iterations. The reflection memory at iteration $i$ is defined as

$M_i = \{R_1, R_2, \dots, R_{i-1}\},$

where $R_i$ is the reflection produced at iteration $i$.

The iterative process is written as

$R_i = \text{Reflect}(I, Q, A_{i-1}, M_i),$

$a$0

$a$1

In algorithmic form, the workflow starts from an initial answer $a$2, initializes $a$3, and then repeats the following sequence for a fixed number of rounds: generate a reflection, extract a correction plan, filter infeasible actions via $a$4, refine the answer using the feasible plan and accumulated memory, and update memory with the new reflection (Wen et al., 24 Feb 2026).

The paper emphasizes that this is a reflection-refinement pipeline rather than direct answer editing. The model does not simply overwrite its previous output; it first reasons about what was wrong, then uses that structured diagnosis to guide the rewrite. Memory Reflection ensures that the diagnosis is history-aware rather than myopic. Conceptually, the memory store records which reasoning paths were already tried and found unhelpful, thereby reducing repetitive loops and encouraging exploration of alternative solutions.

4. Benchmarking protocol and implementation

OCR-Agent is evaluated on OCRBench v2, described as an OCR benchmark with more than 10,000 manually verified question-answer pairs across English and Chinese. The benchmark includes difficult samples and six core evaluation categories: Parsing, Localization, Extraction, Long Reading, Counting, and Basic VQA. The reported evaluation follows the benchmark’s official metrics: TEDS for Parsing, IoU for Localization, F1 for Extraction, BLEU/METEOR/F1/Edit Distance for Long Reading, normalized L1 distance for Counting, and Exact String Matching, containment checks, and ANLS for Basic VQA depending on answer type (Wen et al., 24 Feb 2026).

The implementation uses three refinement rounds, denoted $a$5, after an initial baseline inference. Prompt templates are used both for reflection generation and for guided refinement. The full process is executed on four NVIDIA 3090 GPUs.

The benchmark framing is important because it situates OCR-Agent outside narrowly defined text recognition. The paper explicitly treats OCRBench v2 as a multimodal reasoning benchmark in which OCR is entangled with structured interpretation, counting, extraction, and cross-region inference. This explains why the framework focuses on self-correction robustness rather than on recognition architecture alone.

5. Reported results and ablation evidence

The main reported result is that OCR-Agent-7B reaches an average of 51.0 on the English subset, surpassing all open-source baselines and approaching Gemini-Pro’s 51.9. On the Chinese subset, OCR-Agent-7B reaches 54.7, which the paper states is second only to Qwen2.5-VL-7B at 55.6 among open-source models (Wen et al., 24 Feb 2026).

For the English subset, the paper reports the following scores for OCR-Agent-7B: Recognition 71.8, Referring 27.7, Spotting 0.7, Extraction 80.8, Parsing 39.4, Calculation 41.3, Understanding 79.9, Reasoning 66.5, and Average 51.0. The paper explicitly states that this outperforms InternVL3-8B by 2.0 points on English, since InternVL3-8B is listed at 49.0 average.

For the Chinese subset, the reported scores are: Recognition 77.0, Extraction 69.1, Parsing 22.9, Understanding 65.1, Reasoning 39.8, and Average 54.7. The paper highlights this as +1.2 over InternVL3-8B, which is reported at 53.5. It also states that OCR-Agent sets new open-source records in Chinese Recognition, Extraction, and Understanding, and that when applied on top of the RolmOCR-7B base model it improves Chinese performance by nearly 16 points over the base model’s 37.7 average.

The ablation studies are used to argue that Capability Reflection and Memory Reflection are complementary rather than redundant. On English, the progression is reported as: Naive 38.4 $a$6 CoT 42.0 $a$7 Self-Refine 43.4 $a$8 Capability Reflection 45.9 $a$9 Memory Reflection 48.4 $$\phi(a)= \begin{cases} 1 & \text{if } a \text{ can be executed by the model} \ 0 & \text{if } a \text{ is infeasible.} \end{cases}$$0 full OCR-Agent 51.0.

On Chinese, the progression is: Naive 37.7 $$\phi(a)= \begin{cases} 1 & \text{if } a \text{ can be executed by the model} \ 0 & \text{if } a \text{ is infeasible.} \end{cases}$$1 CoT 38.4 $$\phi(a)= \begin{cases} 1 & \text{if } a \text{ can be executed by the model} \ 0 & \text{if } a \text{ is infeasible.} \end{cases}$$2 Self-Refine 39.4 $$\phi(a)= \begin{cases} 1 & \text{if } a \text{ can be executed by the model} \ 0 & \text{if } a \text{ is infeasible.} \end{cases}$$3 Capability Reflection 44.6 $$\phi(a)= \begin{cases} 1 & \text{if } a \text{ can be executed by the model} \ 0 & \text{if } a \text{ is infeasible.} \end{cases}$$4 Memory Reflection 49.7 $$\phi(a)= \begin{cases} 1 & \text{if } a \text{ can be executed by the model} \ 0 & \text{if } a \text{ is infeasible.} \end{cases}$$5 full OCR-Agent 54.7.

These numbers are used in the paper to support two claims. First, both reflection mechanisms contribute positively. Second, Memory Reflection contributes a particularly large boost, while the combination of memory and capability constraints performs best. The iteration analysis further reports that CoT and Self-Refine tend to plateau or fluctuate after the first or second iteration, whereas OCR-Agent continues improving across three iterations, especially on English Reasoning and Chinese Understanding.

Qualitative examples are described as showing more stable and accurate final outputs than naive decoding or standard CoT, particularly when the task requires reading subtle text or reasoning across multiple image regions. The figures are said to show that OCR-Agent better re-focuses on relevant regions and avoids earlier errors.

6. Significance, limits, and position within agentic OCR research

OCR-Agent is best understood as a disciplined self-correction framework for multimodal OCR tasks. Its stated contribution is not additional OCR supervision or fine-tuning, but a structured method for constraining reflective reasoning. The paper’s broader claim is that structured, self-aware reflection can improve VLM reasoning robustness without additional training (Wen et al., 24 Feb 2026).

Its limitations are also explicit. First, the method adds computational overhead because multiple calls to the VLM are required for each input, increasing latency and cost. Second, its effectiveness is bounded by the base model: if the underlying VLM fundamentally misperceives an image or lacks relevant knowledge, reflection may not recover the correct context. The proposed future directions are dynamic iteration control, model distillation, external tools such as image super-resolution APIs, richer structured memory, and human-in-the-loop refinement for high-stakes settings.

Recent literature uses closely related terminology for substantially different systems. AgentOCR replaces long textual agent histories with rendered visual memory and introduces segment optical caching plus agentic self-compression (Feng et al., 8 Jan 2026). AgenticOCR turns OCR into a query-driven, on-demand extraction system for visual RAG, centered on selective region parsing rather than full-page OCR (Wang et al., 27 Feb 2026). DianJin-OCR-R1 trains a reasoning-and-tool interleaved VLM that first recognizes content, then queries external expert OCR tools, and finally rethinks before answering (Chen et al., 18 Aug 2025). MinerU-Popo, by contrast, is a lightweight OCR post-processing framework for reconstructing document-level structure from page-level OCR outputs (Xu et al., 24 May 2026). This suggests that “agentic OCR” has become an umbrella term for several distinct design patterns: self-correction, optical memory, selective retrieval-time parsing, tool-interleaved reasoning, and post-processing.

Within that landscape, OCR-Agent occupies the self-corrective branch. It neither removes OCR-like perception through an OCR-free formulation, nor primarily externalizes reasoning into tools, nor reframes OCR as retrieval middleware. Its central proposition is narrower and more internal: iterative improvement becomes more reliable when the model reflects on what it can actually do and remembers what it has already failed to do.