Valid Region Focus Module (VRFM)

• VRFM is an architectural module focused on document parsing that identifies and isolates semantically important regions while filtering out redundant background content.
• It employs a two-stage process with an RT-DETR-based layout detector and a pointer network to efficiently generate region proposals and predict the correct reading order.
• Empirical results show that VRFM reduces vision tokens by up to 56% and boosts recognition accuracy, underscoring its efficiency in coarse-to-fine document understanding pipelines.

The Valid Region Focus Module (VRFM) is an architectural component designed to improve the computational efficiency and accuracy of document image parsing by identifying and focusing processing on only semantically relevant regions within high-resolution pages. The module represents the "coarse" stage in coarse-to-fine visual document understanding systems, enabling downstream vision-LLMs to avoid direct encoding of redundant background, margins, and other non-informative areas. The VRFM plays a pivotal role in minimizing the number of vision tokens passed to large models, reducing both FLOPs and memory requirements, while simultaneously improving end-to-end recognition accuracy by preserving the relevant structural and reading order information of valid layout elements (Cui et al., 25 Mar 2026).

1. Motivation: Visual Redundancy in Document Parsing

Document parsing tasks often require high-resolution page inputs to accurately capture small-font text, mathematical equations, tables, and charts. However, naïve attention over all image patches in such high-resolution inputs leads to quadratic scaling of token numbers and computational burden inherent to transformer-based vision encoders. Extensive analysis shows that, for benchmarks like OmniDocBench v1.5, only 39–60% of a standard page image's area contains semantically important layout elements, with the remainder consisting of background or decorative content. Thus, the majority of visual tokens produced by monolithic VLMs offer little benefit for recognition or structural understanding (Cui et al., 25 Mar 2026). By introducing a targeted region selection stage upstream of the main recognition pipeline, VRFM aims to eliminate this inefficiency—filtering out redundant areas and focusing resources on only valid regions.

2. Architectural Composition and Data Flow

The VRFM comprises two tightly-coupled submodules:

1. RT-DETR-based Layout Detector: This stage uses a compact convolutional backbone followed by a transformer decoder, adopting the Real-Time DEtection TRansformer (RT-DETR) architecture. The detector outputs $N$ region proposals $\{B_i, c_i\}$ for each page, where $B_i \in \mathbb{R}^4$ are normalized bounding boxes and $c_i$ represents classification logits across element classes (e.g., text, table, formula, chart). Decoder embeddings $f_i \in \mathbb{R}^d$ are retained for downstream order modeling.
2. Pointer Network for Sequencing: Utilizing the intermediate features $\{f_i\}$, a pointer-network models the probable pairwise reading order among regions. Each $f_i$ is projected via learned weights $W_q$ and $W_k$ into queries $q_i$ and keys $k_i$, respectively. A raw affinity matrix $A \in \mathbb{R}^{N \times N}$, $A_{ij} = q_i^\top k_j$, is row-normalized to yield a probability matrix $P_{ij}$ indicating the likelihood that region $j$ follows $i$ in logical reading sequence.

A region is accepted as valid if its classification score $s_i = \text{softmax}(c_i)_{\text{valid-class}}$ exceeds a defined threshold $\tau$, or it ranks within the top-$K$ by confidence. This yields a filtered set of $M \ll N$ "valid" regions, each with assigned position in sequential order.

3. Optimization Objectives and Learning Algorithm

VRFM optimization proceeds in two stages:

1. Layout Detection Training: The RT-DETR detector is trained end-to-end using composite detection losses post-Hungarian matching:

$$L_{\text{det}} = L_{\text{cls}} + \lambda_1 L_{L1}(B_i, B_i^*) + \lambda_2 L_{GIoU}(B_i, B_i^*)$$

where $B_i^*$ are ground-truth region boxes, $L_{\text{cls}}$ is cross-entropy over classes, $L_{L1}$ is $L_1$ localization error, and $L_{GIoU}$ ensures precise spatial coverage.

2. Reading Order Supervision: After detection training, the backbone and decoder are frozen. The pointer network is trained to predict the binary relation matrix $Y \in \{0,1\}^{M \times M}$, with $Y_{ij}=1$ iff region $j$ follows $i$. The Generalized Cross-Entropy (GCE) loss is used to enhance robustness against annotation noise:

$$L_{\text{order}} = \frac{1}{M(M-1)} \sum_{i \neq j} \frac{1 - P_{ij}^q}{q}$$

where $q$ is a robustness tuning parameter (typ. $q=0.7$). The total loss is $L_{\text{VRFM}} = L_{\text{det}}$ (+ $L_{\text{order}}$ during pointer network training).

4. Integration into Coarse-to-Fine Document Understanding Pipelines

Once trained, VRFM infers on a full-page image $I$ as follows:

1. Region Proposal and Filtering: The network infers $\{B_i, s_i, f_i\}$ and $P_{ij}$. Post-thresholding, only regions satisfying $s_i \geq \tau$ or top-$K$ ranking are retained.
2. Region Cropping: Each valid region is cropped at its original resolution for downstream analysis.
3. Ordered Sequencing: A greedy argmax traversal over the $P_{ij}$ matrix yields the final reading order, $R = (r_1,\dots,r_M)$.
4. Fine-Grained Recognition: Each cropped region $I_{r_k}$ is processed independently by a fine-tuned vision-LLM (e.g., PaddleOCR-VL-0.9B), leveraging dynamic visual patch allocation (e.g., NaViT).

Finally, structured outputs from all regions are reassembled in the predicted reading order. Diagrams in the source illustrate this pipeline, with VRFM producing both spatial and sequential structure for subsequent text extraction and layout understanding (Cui et al., 25 Mar 2026).

5. Empirical Evaluation: Efficiency and Accuracy Gains

Ablation and comparative experiments demonstrate the operational impact of VRFM within PaddleOCR-VL and related pipelines:

Configuration	Vision Tokens	Accuracy (%)	Throughput (pages/s)
Monolithic VLM (Qwen3-VL-8B)	5,818	89.60	1.06
Crop-only baseline (DocKylin+Qwen3-VL-8B)	4,316 (-25%)	87.75	-
Heuristic pipeline (PP-DocLayout+Qwen3-VL-8B)	3,876	89.30	-
VRFM + Qwen3-VL-8B	4,262 (-27%)	90.37 (+0.77)	-
VRFM + PaddleOCR-VL-0.9B	2,561 (-56%)	92.62 (+3.02)	1.62 (+53%)

VRFM achieves a 25–56% reduction in vision token count and a 50–53% increase in page and token throughput. Notably, this is accomplished with no loss—and frequently, a measurable gain—in end-to-end accuracy, attributable to precise focus on valid elements and robust order modeling. GPU memory consumption is also reduced (78.5 GB → 42.1 GB) (Cui et al., 25 Mar 2026).

6. Pseudocode and Inference Workflow

A representative pseudocode (verbatim from the source) illustrates the operational loop:

det_out = RTDETR(I) boxes, cls_logits, feats = det_out.boxes, det_out.logits, det_out.region_features scores = softmax(cls_logits)[:, valid_class] keep = topK_or_threshold(scores, K, tau) Q = Wq(feats[keep]); K = Wk(feats[keep]) A = Q @ K.T # pairwise affinity P = softmax_rowwise(A) order = greedy_argmax(P) # reading-order permutation crops = [crop(I, boxes[i]) for i in keep] for idx in order: result[idx] = PaddleOCR_VL_0.9B(crops[idx]) final_doc = assemble(result, order)

This explicit staged looping—layout detection, ranking, region cropping, and ordered recognition—fundamentally differentiates VRFM from heuristic token-pruning, enabling data-driven structural preservation (Cui et al., 25 Mar 2026).

7. Distinction from Interactive Region Policies in Multimodal Models

Mechanisms superficially similar to VRFM's role appear in reinforcement-based region selection strategies for multimodal chain-of-thought models, such as the Region-Conditioned Reinforcement Policy Optimization (R-GRPO) component within VLM-R³ (Jiang et al., 22 May 2025). There, region focus occurs via an autoregressive policy $\pi_\theta(a_t | s_t)$ emitting either text tokens or bounding box commands in an interactive inference loop; the system rewards syntactically valid, information-seeking crops and incorporates corresponding visual patches back into an interleaved reasoning trajectory. However, no module equivalent to VRFM—explicitly targeting spatial redundancy in static document images, coupled with layout structure and reading order prediction—exists in VLM-R³. Instead, the core aim there is to ground dynamic textual reasoning steps in iteratively identified regions, not the structured elimination of background prior to recognition. This underscores the domain-specific nature of VRFM as a pre-recognition filter for document parsing, distinct from multimodal iterative attention or RL-based focus used in visual reasoning models (Jiang et al., 22 May 2025, Cui et al., 25 Mar 2026).