TextAnchor-Bench (TABench)

• The paper introduces TABench, a benchmark that fills the evaluation gap in spatially grounding text in images using bidirectional tasks.
• TABench leverages datasets from HierText, SVRD, ICDAR2015, and CDLA and employs rigorous metrics such as exact-match accuracy and IoU-based F1 scoring.
• Evaluations reveal significant disparities in VLMs between reading and localization, underlining the need for architectural innovations in text anchoring.

TextAnchor-Bench (TABench) is a benchmark for fine-grained text-region grounding that systematically evaluates the ability of vision-LLMs (VLMs) to localize and anchor textual elements—such as words or phrases—within images. While contemporary VLMs have demonstrated advances in optical character recognition (OCR) and downstream reasoning tasks, their capacity to ground recognized text tokens spatially remains limited. TABench is designed to fill this evaluation gap by providing an objective, bidirectional suite for measuring both reading and spatial localization performance in diverse real-world visual contexts (Xu et al., 31 Mar 2026).

1. Motivation and Objectives

TABench addresses a critical shortcoming in OCR-oriented vision-language systems—the lack of reliable “text anchoring.” Conventional OCR tasks and associated benchmarks have primarily emphasized either end-to-end text recognition or coarse text detection. They seldom quantify a model’s ability to map each decoded text token precisely to its spatial support, which is crucial for practical applications including smart glasses, document understanding systems, and interactive assistants. TABench’s core objective is to diagnose this capability using a unified framework that equally balances the dual axes of region-sensitive reading and fine-grained spatial localization.

In this construction, TABench aims to reveal whether models possess merely the ability to read, or a genuinely robust mechanism for stable bidirectional text-region correspondences under constrained querying scenarios.

2. Dataset Composition and Annotation

TABench is compiled using four public datasets: HierText, SVRD, ICDAR2015, and CDLA. This heterogeneous pool is curated to encompass a broad spectrum of document and scene text, including unconstrained environments, receipts, tickets, warehouse slips, reports, Chinese documents, books, posters, notices, price tags, invoices, and certificates. The benchmark comprises 5,450 queries defined over approximately 970 images, distributed across 12 categories. Each category contributes an equal number of region-to-text and text-to-region queries (2,725 of each).

The initial annotation sources provide line-level bounding boxes and text transcripts. TABench subsequently applies a deterministic post-processing pipeline: for Region-to-Text (R2T), the prompt specifies the bounding box and the ground truth is the associated transcript; for Text-to-Region (T2R), the text query is canonicalized (removing whitespace, punctuation, and normalizing Unicode via NFKC), and all occurrences are grouped into a multi-box query. This ensures consistency and rigorous definition of evaluation instances.

3. Task Structure

TABench consists of two symmetric tasks designed to rigorously evaluate fine-grained spatial-language grounding:

• Region-to-Text (R2T): The model is presented with a prompt of the form “What is the text at location [xₘᵢₙ,yₘᵢₙ,xₘₐₓ,yₘₐₓ]?” It must output the exact substring located within these coordinates.
• Text-to-Region (T2R): The prompt “Where is ‘<Text>’ located in the image?” requires the model to return a list of bounding boxes in JSON format corresponding to all spatial occurrences of the queried text.

During inference, models are expected to output normalized absolute or relative coordinates as applicable. Outputs that are unparsable (e.g., fail to yield valid bounding boxes or text substrings given the input format) are assigned empty predictions. This strict format enforces uniform evaluation and discourages ad-hoc heuristics.

4. Evaluation Metrics

TABench employs rigorous and interpretable metrics to assess both directional tasks:

• R2T – Exact-Match Accuracy:

$$\mathrm{Acc_{R2T}} = \frac{1}{N} \sum_i \mathbb{I}(\hat{y}_i = y_i)$$

where $\hat{y}_i$ is the normalized predicted string and $y_i$ is the ground truth.

• T2R – Detection-Style F1 Score:
  • Intersection-over-Union (IoU) threshold: $$\mathrm{IoU}(B_{\mathrm{pred}}, B_{\mathrm{gt}}) = \frac{\mathrm{area}(B_{\mathrm{pred}} \cap B_{\mathrm{gt}})}{\mathrm{area}(B_{\mathrm{pred}} \cup B_{\mathrm{gt}})}$$, with IoU ≥ 0.5 considered a match.
  • Precision: TP/(TP+FP), Recall: TP/(TP+FN), and $$F1_{\mathrm{T2R}} = \frac{2 \, \mathrm{Precision} \cdot \mathrm{Recall}}{\mathrm{Precision} + \mathrm{Recall}}$$
• Overall Score: The average of the two primary metrics,

$$\mathrm{Overall} = \frac{\mathrm{Acc_{R2T}} + F1_{\mathrm{T2R}}}{2}$$

with unsupported directions scored as zero. This balanced aggregation penalizes models that excel at only one of the subtasks, providing a composite diagnostic of joint grounding ability.

5. Baseline Model Performance

Empirical results indicate that most state-of-the-art VLMs exhibit significant disparities between reading and localization performance. Table 1 summarizes representative benchmark outcomes:

Model	Acc₍R2T₎	F1₍T2R₎	Overall
Gemini 3.0 Pro (closed)	25.9%	62.6%	44.2%
GPT 5.2 (closed)	10.6%	0.6%	5.6%
Kimi K2.5	49.5%	57.7%	53.6%
Qwen 3.5	61.1%	72.8%	66.9%
DeepSeek-OCR 2	—	11.7%	5.8%
Q-Mask (3B)	50.6%	40.4%	45.5%

The benchmark exposes common failure modes such as moderate-to-high T2R (grounding) performance co-occurring with near-random R2T (reading) ability, and vice versa, as well as the complete inability to support bidirectional querying for some systems. This suggests current model architectures and training protocols do not inherently enforce robust coupling between recognition and localization.

6. Insights and Research Directions

TABench’s bidirectional evaluation paradigm demonstrates that end-to-end OCR and vision-language pretraining do not suffice to guarantee reliable text anchoring. Most leading systems display substantial performance imbalances across directions and lack the capacity for stable, fine-grained correspondences. The diagnostic power of TABench lies in its granular prescription: architectures must separate the “where” from the “what” in visual reasoning.

The Q-Mask framework, featuring a causal query-driven mask decoder, exemplifies one such approach by implementing a “visual chain-of-thought” paradigm—first sequentially generating query-conditioned visual masks (acquiring grounded spatial evidence) prior to textual recognition. This decoupling enforces grounded evidence collection and explicit text anchor extraction, thereby integrating spatial priors directly into model reasoning and output formation (Xu et al., 31 Mar 2026).

A plausible implication is that future advances in VLM design—especially for OCR-intensive applications—will require architectural and training innovations that prioritize explicit spatial-textual disentanglement. TABench is positioned to remain a central benchmark for such developments due to its reproducibility, granularity, and coverage across challenging real-world visual genres.