TextOCR: Real-Image Scene Text Benchmark

• TextOCR is a large-scale real-image dataset with polygon-based, word-level annotations designed for detecting and recognizing arbitrary-shaped text.
• It addresses challenges such as high text density, varied orientations, and cluttered backgrounds, significantly enhancing OCR model performance.
• TextOCR serves as a substrate for extended applications, including multimodal reasoning, motion-blur restoration, and text-aware image compression.

TextOCR is a large-scale real-image scene text dataset and benchmark for arbitrary-shaped text detection and recognition, collected on images from the TextVQA dataset and annotated at word level with polygons and transcriptions (Singh et al., 2021). It was introduced to support text detection, text recognition, end-to-end text spotting, and scene text based reasoning in tasks such as TextVQA and TextCaps, and it has subsequently been used not only as a benchmark for OCR systems but also as a training substrate for text spotting, motion-blur restoration, and text-aware image compression (Singh et al., 2021, Rashid et al., 8 Nov 2025, Wang et al., 4 Mar 2026).

1. Origin and motivation

TextOCR was introduced to address three gaps identified in prior scene text research: the lack of large real-world, text-dense OCR datasets; insufficient support for arbitrary-shaped, high-density text; and the absence of ground-truth scene text annotations for TextVQA and TextCaps images (Singh et al., 2021). Earlier resources such as Synth90k and SynthText were large but synthetic, while real datasets such as ICDAR variants, COCO-Text, and Total-Text were smaller, often collected for other purposes, or less suitable for arbitrary-shaped and text-dense scenes. TextOCR instead uses the images from TextVQA, which were selected because scene text is central to the semantics of the image, so that OCR can be evaluated in isolation and also integrated with downstream reasoning (Singh et al., 2021).

The benchmark is explicitly oriented toward arbitrary-shaped scene text detection and recognition on real images. It supports text detection, text recognition, end-to-end text spotting, and downstream reasoning in which recognized text is consumed by visual question answering and captioning systems (Singh et al., 2021). In later work, this role broadens further: GLASS treats TextOCR as a stress test for large-scale end-to-end scene text spotting under broad scale variation and arbitrary orientations, and DeepSolo uses TextOCR as a large-scale pretraining source for Transformer-based text spotting (Ronen et al., 2022, Ye et al., 2022).

2. Dataset composition and annotation design

TextOCR contains 28,134 images and 903,069 non-empty words, with 1,320,064 labeled instances in total (Singh et al., 2021). The split reported in the original benchmark is as follows.

Split	Images	Non-empty words
Train	21,749	714,186
Validation	3,153	108,386
Test	3,232	80,497

The dataset is unusually dense: it has 32.1 words per image on average, and more than 10% of images contain 100+ words (Singh et al., 2021). Text instances are distributed across documents, posters, product packaging, screens, charts, street scenes, and other real photographs inherited from TextVQA and OpenImages, rather than synthetic renderings (Singh et al., 2021).

TextOCR is explicitly designed for arbitrary-shaped text. It includes 150,197 rotated words with rotation angle greater than $20^\circ$, and 19,310 curved words where the polygon uses more than four points (Singh et al., 2021). Annotation is polygon-based rather than box-only: each word region is represented by an ordered polygon, and the edge from the first to second point indicates the top-left reading direction. Horizontal or near-horizontal words are often annotated with rectangles, while curved or irregular words use multi-point polygons (Singh et al., 2021).

For recognition, only Latin-script words are transcribed. Non-Latin or illegible words are labeled with “.”, treated as an empty word (Singh et al., 2021). This design makes the dataset predominantly English and Latin-script for recognition, while still preserving dense detection supervision over the whole image. The annotation interface supports arbitrary polygons, zoom and re-annotation, and case-sensitive transcription, and expert auditors reviewed annotations to enforce precise boundaries and minimal background inclusion (Singh et al., 2021).

3. Benchmark tasks and difficulty profile

TextOCR supports two principal OCR tasks. The first is word-level text recognition, where the input is a cropped word image and the metric is exact-match word accuracy. The second is end-to-end text spotting, where the model must both localize and transcribe text in the full image; evaluation follows ICDAR2015-style matching with polygonal or box overlap and reports Precision, Recall, and F-measure, with

$F = \frac{2 \cdot P \cdot R}{P + R}.$

The overlap term is expressed through IoU between predicted and ground-truth regions (Singh et al., 2021).

The dataset is difficult not merely because of scale, but because of scene structure. Heatmap analysis in the original benchmark shows that TextOCR text instances are more uniformly distributed across the image than COCO-Text, ICDAR15, and Total-Text (Singh et al., 2021). GLASS characterizes TextOCR as a large-scale, arbitrary-shaped scene-text benchmark with natural scenes, broad scale variation from very small to very large text, arbitrary orientations, cluttered backgrounds, and lexicon-free evaluation, making it a more representative test of unconstrained scene text spotting (Ronen et al., 2022).

This difficulty persists even for strong models. The original paper reports that current state-of-the-art OCR models trained only on synthetic or other real datasets perform poorly on TextOCR relative to more conventional benchmarks, and that even after TextOCR fine-tuning, TextOCR remains harder than ICDAR15 or Total-Text for the same architectures (Singh et al., 2021). A plausible implication is that TextOCR probes several failure modes simultaneously: dense instance layout, arbitrary geometry, clutter, and the need for lexicon-free recognition.

4. Effects on OCR and text spotting models

TextOCR was introduced not only as a benchmark but also as a training set that substantially improves OCR systems. In word-level recognition experiments, the TPS-ResNet-BiLSTM-Attn model trained on COCO-Text achieved 43.20% on the TextOCR test set, whereas the same architecture trained on TextOCR achieved 65.65%; the TextOCR-trained model also improved on COCO-Text and on irregular-text benchmarks such as ICDAR15, SVTP, and CUTE80 (Singh et al., 2021). When models pre-trained on Synth90k and SynthText were fine-tuned on TextOCR, the TextOCR test accuracy rose from 43.07% to 58.61% for CRNN, from 43.16% to 60.85% for Rosetta, from 48.23% to 66.84% for STAR-Net, and from 50.37% to 69.49% for TPS-ResNet-BiLSTM-Attn (Singh et al., 2021).

The same effect appears in end-to-end text spotting. Mask TextSpotter v3 trained on COCO-Text achieved 32.5 F-measure on TextOCR, while the same model trained on TextOCR achieved 45.8; fine-tuning the official Mask TextSpotter v3 on TextOCR increased TextOCR F-measure from 34.9 to 50.8 and also improved Total-Text and COCO-Text results (Singh et al., 2021). This supports the claim in the original benchmark that TextOCR helps achieve state-of-the-art performance on multiple OCR datasets as well as exposing weaknesses of prior models on real, text-heavy images.

Later spotters treat TextOCR as a decisive benchmark. GLASS reports end-to-end Hmean of 67.1 on the TextOCR test set, compared with 50.8 for Mask TextSpotter v3, and attributes the improvement to global-to-local attention and an orientation-aware rotated box loss that are especially useful under scale and angle extremities (Ronen et al., 2022). DeepSolo, a DETR-like text spotter with explicit point queries and a single decoder, uses TextOCR in extended pretraining and reports substantial gains on Total-Text, ICDAR 2015, and Inverse-Text when TextOCR is added to the training mixture (Ye et al., 2022). This suggests that TextOCR functions simultaneously as a benchmark of scene-text difficulty and as a source of supervision that transfers to other datasets.

5. Role in scene-text reasoning

A distinctive feature of TextOCR is its direct linkage to TextVQA and TextCaps. Because the images are drawn from TextVQA, the benchmark makes it possible to separate OCR errors from reasoning errors and to train end-to-end systems in which OCR is part of the scene-text reasoning pipeline rather than a fixed external component (Singh et al., 2021).

The original paper uses a TextOCR-trained OCR model to build PixelM4C, an end-to-end model for scene text based reasoning. Relative to the earlier M4C setup using Rosetta OCR, PixelM4C with TextOCR-trained Mask TextSpotter v3 improves TextVQA validation accuracy, reaching 41.23% with the full TextOCR OCR configuration and 42.12% when ST-VQA is added as additional training data; the ground-truth TextOCR upper-bound setting reaches 48.04% with ST-VQA, while human accuracy is 85.01% (Singh et al., 2021). For TextCaps, PixelM4C-Captioner with TextOCR-trained OCR improves BLEU-4 from 23.30 to 24.10 and CIDEr from 89.60 to 91.99 relative to the Rosetta-based baseline (Singh et al., 2021).

These results clarify TextOCR’s dual status. It is an OCR benchmark in the conventional sense, but it is also an enabling layer for multimodal reasoning systems that attend to recognized tokens, OCR features, and object features jointly. The benchmark therefore supports analysis at two levels: recognition quality itself, and the effect of recognition quality on downstream question answering and caption generation (Singh et al., 2021).

6. TextOCR as substrate for restoration and compression

TextOCR has also been repurposed beyond recognition. The motion-blur restoration work "Hybrid CNN-ViT Framework for Motion-Blurred Scene Text Restoration" constructs a TextOCR-derived benchmark by taking a subset of 5,000 sharp TextOCR images, generating synthetically blurred counterparts with realistic motion-blur kernels from Shen et al., and assembling 6,000 strictly aligned blurred-sharp pairs split into 5,000 training, 500 validation, and 500 test samples (Rashid et al., 8 Nov 2025). Images are resized to $256 \times 256 \times 3$, normalized to $[0,1]$, and paired one-to-one with blurred versions; TextOCR’s annotations are not used directly for restoration training, but restored outputs remain compatible with the original text labels for downstream OCR or detection (Rashid et al., 8 Nov 2025). On this TextOCR-derived blur benchmark, the proposed model reports 32.20 dB PSNR, 0.934 SSIM, 2.83 million parameters, and an average inference time of 61 ms (Rashid et al., 8 Nov 2025).

TextOCR also appears in ultra-low bitrate compression research. TextBoost fine-tunes its text-aware stage on the TextOCR training set and evaluates on the TextOCR validation split, using text detection F1, end-to-end recognition F1, PSNR, MS-SSIM, LPIPS, and bpp as metrics (Wang et al., 4 Mar 2026). It exploits TextOCR annotations in two ways: binary text masks built from word-level boxes for the guidance-consistent loss

$$\mathcal{L}_{\text{gc}} = \mathrm{MSE}\big(m \odot x,\; m \odot \hat{x}\big),$$

and size-based filtering of small text through average character area thresholds $T_{\text{train}} = 150$ on $256 \times 256$ crops and $T_{\text{test}} = 800$ on full-resolution images (Wang et al., 4 Mar 2026). On TextOCR at approximately 0.033 bpp, TextBoost reports DET F1 $= 0.404$, while the best baseline ELIC reports DET F1 $= 0.2515$, corresponding to the paper’s stated relative improvement of up to 60.6% higher text-recognition F1 at comparable PSNR and bpp (Wang et al., 4 Mar 2026).

These later uses show that TextOCR has become a general-purpose substrate for scene-text fidelity research. This suggests that its value lies not only in transcription supervision, but also in the combination of dense real-image text, arbitrary geometry, and word-level spatial annotations.

7. Terminological extensions and adjacent systems

Although TextOCR originally denotes the 2021 dataset and benchmark, later literature sometimes uses the term more broadly. DTrOCR explicitly describes “TextOCR” as the general problem of recognizing text from images—scene text on signs and storefronts, printed text in documents and receipts, and handwriting—and proposes a decoder-only Transformer that consumes image patches as tokens and autoregressively generates text (Fujitake, 2023). In that usage, TextOCR names a task family rather than a particular benchmark.

A more explicit terminological shift appears in "Text images processing system using artificial intelligence models", where “TextOCR” is used to denote an end-to-end processing system rather than the benchmark itself (Bahjat, 12 Dec 2025). That system converts images to grayscale, applies RealESRGAN_x2 with tiled inference and CLAHE, detects text with DBNet++ through differentiable binarization and an FPN over ResNet-50, and classifies extracted content into Invoice, Form, Letter, or Report with facebook/bart-large-mnli, reporting about 94.62% text recognition accuracy on Total-Text over approximately 10 hours of testing (Bahjat, 12 Dec 2025). This broader usage reflects a pipeline interpretation of TextOCR.

A separate but similarly named line of work is TexOCR, which concerns page-level reconstruction of scientific PDFs into compilable LaTeX rather than scene text spotting (Wang et al., 24 Apr 2026). TexOCR-Bench contains 2,135 expertly annotated documents, TexOCR-Train contains 404K page-level pairs derived from 57K arXiv papers, and the 2B-parameter TexOCR model is trained with supervised fine-tuning and reinforcement learning with verifiable rewards for compilability and referential integrity (Wang et al., 24 Apr 2026). Despite the lexical resemblance, TexOCR addresses document-level page-to-LaTeX reconstruction, not the scene-text dataset introduced in 2021.

Accordingly, the most precise usage distinguishes three layers: TextOCR as the 2021 arbitrary-shaped scene text benchmark (Singh et al., 2021); TextOCR as a broader shorthand for scene text OCR pipelines in some later papers (Bahjat, 12 Dec 2025, Fujitake, 2023); and TexOCR as a separate document OCR program centered on compilable LaTeX reconstruction (Wang et al., 24 Apr 2026).