EviGRPO for Multi-Page Document Reasoning

• The paper introduces EviGRPO, a reinforcement-learning framework that integrates explicit evidence-page rewards to improve multi-page document reasoning.
• It employs a structured output format with binary T/F evidence-page judgments and a two-stage curriculum learning strategy for robust evidence localization.
• Empirical results show significant gains, including a +38.77 point improvement in evidence recall, outperforming baseline models on multi-page tasks.

Searching arXiv for the specified paper and closely related context papers to ground the article. Evidence Page-Guided GRPO (EviGRPO) is a reinforcement-learning framework introduced in "DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding" for multimodal LLMs (MLLMs) operating on multi-page documents (Xiong et al., 10 Aug 2025). It extends GRPO with an explicit, verifiable reward for page retrieval, with the stated goal of making the model follow a coarse-to-fine reasoning strategy: first retrieve relevant pages, then generate an answer. In DocR1, this framework is combined with structured outputs, a two-stage annotation pipeline, and a curriculum learning strategy, and the resulting model is reported to achieve state-of-the-art performance on multi-page tasks while consistently maintaining strong results on single-page benchmarks (Xiong et al., 10 Aug 2025).

1. Problem setting and motivation

EviGRPO is designed for multi-page document understanding, where the core difficulty is not only reading text and figures on individual pages but also finding the right evidence pages and then reasoning across them (Xiong et al., 10 Aug 2025). The motivating document types explicitly include scientific papers, contracts, reports, and slide decks. The paper characterizes the task as requiring page-level retrieval of sparse evidence, multi-hop reasoning across pages, and robustness to irrelevant pages and long contexts.

The central motivation is that standard supervised fine-tuning (SFT), or standard GRPO using answer correctness alone, does not sufficiently teach the model to localize evidence. In sparse multi-page settings, a model may sometimes guess the answer without truly identifying supporting pages, which the paper associates with weaker reliability and generalization. EviGRPO is introduced to explicitly reward three factors: correct output format, correct final answer, and correct evidence-page selection (Xiong et al., 10 Aug 2025).

This framing places evidence localization inside the learning objective rather than treating it as a downstream diagnostic. A plausible implication is that the method targets a failure mode common in long-context MLLMs: superficially correct answering without verifiable grounding in the document.

2. Objective, outputs, and optimization

DocR1 is trained to produce structured outputs containing three fields:

• > ... for the reasoning trace
• <evidence_page> ... </evidence_page> for page-level evidence judgment
• <answer> ... </answer> for the final answer

The evidence-page format is either a list of page indices or a binary T/F label per page. The main method uses the per-page T/F format with page count not explicitly given in the prompt, which, according to the paper, forces the model to infer the number of pages and reason more carefully (Xiong et al., 10 Aug 2025).

For each sample with question $q$ and $N$ document images, the current policy generates $G$ candidate responses $\mathcal{O} = \{o_1,\dots,o_G\}$. Each response receives a verifiable reward

$$r_i = r_i^{\text{format}} + r_i^{\text{acc}} + r_i^{\text{evi}},$$

where $r_i^{\text{format}} \in \{0,1\}$ indicates whether the output strictly follows the required format, $r_i^{\text{acc}}$ is answer accuracy measured by ANLS, and $r_i^{\text{evi}}$ is the evidence-page reward (Xiong et al., 10 Aug 2025).

The evidence reward is defined as an F1-style overlap between predicted evidence pages $P_i$ and ground-truth evidence pages $G_i$:

$N$0

Here $N$1 is the number of predicted evidence-page judgments. The paper explicitly motivates the F1 design by noting that simple accuracy would be misleading in sparse-evidence settings, since predicting all pages as irrelevant could appear artificially good. The formulation is intended to reward both precision and recall of evidence-page prediction, while setting the evidence reward to zero if the model predicts the wrong number of page judgments (Xiong et al., 10 Aug 2025).

As in GRPO, rewards are normalized within the sampled group:

$N$2

The policy update then uses the standard GRPO clipped objective with a KL penalty to keep the policy near the reference model:

$N$3

The stated role of these terms is standard: $N$4 is the current policy, $N$5 is the previous policy used to generate rollouts, $N$6 is the clipping threshold, $N$7 is the group-normalized advantage, $N$8 is the KL penalty weight, and $N$9 is a frozen reference policy. The paper states that this preserves training stability and prevents the policy from drifting too far from the base instruction-tuned model (Xiong et al., 10 Aug 2025).

The resulting reward structure is explicitly interpreted as encouraging a hierarchical decision process. The coarse stage identifies which pages contain relevant evidence, and the fine stage reasons over the selected pages and answers. In the paper’s terminology, the evidence reward directly supervises the coarse stage, while answer ANLS supervises the fine stage.

3. Evidence localization as policy behavior

A defining feature of EviGRPO is that evidence-page retrieval is learned implicitly through RL rather than delegated to an external retriever. The mechanism described in the paper has three components: the model must output evidence-page judgments before the answer, correct page selection is explicitly rewarded, and the prompt structure forces per-page judgments. Retrieval is therefore part of the policy output rather than a separate module (Xiong et al., 10 Aug 2025).

This is also the paper’s main clarification against a common misunderstanding. Evidence retrieval in DocR1 is not a separate page-ranking subsystem. Instead, the model is trained to “self-retrieve” the relevant pages as an intermediate step. The novelty is therefore concentrated in the training framework and output specification, not in a new visual encoder or transformer backbone.

The ablation on page-selection formats is used to support this claim. The paper compares three formats:

• PSF-1: output page indices directly
• PSF-2: binary T/F labels with explicit page count
• Ours: binary T/F labels without explicitly giving the count

The reported finding is that the main method performs best. The paper attributes weaker performance in PSF-1 to the fact that direct index output does not force exhaustive per-page decisions, and it argues that PSF-2 can be partially gamed because the page count is known. By contrast, the chosen format requires the model to infer the page count and make per-page judgments more genuinely (Xiong et al., 10 Aug 2025).

This suggests that EviGRPO is not only a reward modification but also a protocol for structuring the action space. The page-localization subproblem is made explicit in the output, then coupled to a verifiable reward, so that retrieval competence emerges as a directly optimized behavior.

4. Annotation pipeline and data resources

To support training with limited supervision, the paper describes a two-stage annotation procedure. In the generation stage, Gemini 2.5 Flash generates initial annotations from a task-specific prompt. In the verification stage, the same MLLM checks the generated annotation again, and only samples whose predicted output remains consistent with ground truth are retained. The paper presents this as a self-checking pipeline for filtering noisy labels and obtaining higher-quality data (Xiong et al., 10 Aug 2025).

The framework is trained with EviBench and evaluated in part with ArxivFullQA. Their construction is summarized below.

Resource	Composition	Purpose
EviBench	4.8k examples: 1.3k single-page samples and 3.5k multi-page samples	High-quality training set
ArxivFullQA	8.6k QA samples based on DocMatrix arXiv scientific papers	Evaluation benchmark; also includes a training subset

The single-page portion of EviBench contains 13 datasets, with 100 samples each: DocVQA, InfographicVQA, ChartQA, DeepForm, DVQA, FigureQA, KleisterCharity, OCRVQA, TabFact, TextCaps, TextVQA, VisualMRC, and WikiTableQuestions. These were chosen to cover diverse layouts, charts, tables, forms, and text-rich images (Xiong et al., 10 Aug 2025).

The multi-page portion contains five main multi-page datasets plus a paper-reading subset: DUDE with 1000 samples, MP-DocVQA with 500, TATDoc with 500, SlideVQA with 500, MultiHiertt with 500, and $G$0 with 500, yielding 3.5k multi-page samples in total.

ArxivFullQA is presented as both a training subset and a new evaluation benchmark. Its questions cover seven categories: factual, reasoning, comparison, summary, procedural, motivation, and result. The paper highlights two annotation differences relative to EviBench. First, in stage 1, the input is LaTeX-formatted text rather than images, with the stated effect of improving question generation and answer quality. Second, in stage 2, the model is given the visual full paper and must answer the annotated question; only consistent outputs are retained. The paper therefore characterizes ArxivFullQA as a benchmark for full-paper understanding rather than only page-level question answering (Xiong et al., 10 Aug 2025).

5. Model instantiation and empirical performance

DocR1 does not introduce a new backbone architecture from scratch. It is initialized from Qwen2.5-VL-Instruct (7B). The paper gives three reasons for this choice: the base model already has instruction-tuned multimodal reasoning ability, it avoids a “cold-start” RL problem, and large-scale chain-of-thought labels for document reasoning are scarce. The novelty is thus stated to lie primarily in the training framework, not in a new visual encoder or transformer architecture (Xiong et al., 10 Aug 2025).

The reported training setup uses 8 NVIDIA A100 GPUs, a batch size of 16, $G$1 candidates per sample, $G$2 for the KL penalty, and image resolution capped at $G$3. Evaluation covers six single-page benchmarks—DocVQA, InfographicVQA, WikiTableQuestions, TabFact, TextVQA, and VisualMRC—and seven multi-page benchmarks—DUDE, MP-DocVQA, TATDoc, MultiChartQA, MultiHiertt, SlideVQA, and ArxivFullQA (Xiong et al., 10 Aug 2025).

On single-page benchmarks, DocR1 is reported as competitive with Qwen2.5-VL-Instruct and slightly better on some tasks, with the following scores: DocVQA 95.1, InfoVQA 82.6, WTQ 63.1, TabFact 79.6, TextVQA 81.0, and VisualMRC 251.6. The paper states that it is especially strong on table-related tasks, specifically WTQ and TabFact, while maintaining comparable performance elsewhere.

On multi-page benchmarks, the reported scores are:

Benchmark	Score
MP-DocVQA	87.45
DUDE	54.39
SlideVQA	71.96
MultiChartQA	62.41
MultiHiertt	33.88
TATDoc	64.80
ArxivFullQA	40.65

The corresponding average is reported as 59.36, which the paper states is 6.93 points higher than the strongest baseline average reported there. The gains are described as especially large on MultiChartQA, MultiHiertt, TATDoc, and ArxivFullQA, and the paper further states that the model surpasses larger 32B and 38B baselines, suggesting that the training strategy matters more than scale alone in this setting (Xiong et al., 10 Aug 2025).

6. Ablations and diagnostic analyses

The paper presents multiple ablations to isolate the contribution of evidence-page guidance. Using the same mixed training data, SFT is reported to yield limited gains, GRPO improves many tasks substantially, and EviGRPO is generally better than standard GRPO (Xiong et al., 10 Aug 2025). The intended interpretation is that RL is useful for document reasoning, but explicit evidence-page supervision further improves the reasoning process.

A separate analysis examines data composition. Training on single-page only is reported to harm multi-page reasoning significantly. Training on multi-page only helps more, but less than combining both. The best overall results come from combining single-page and multi-page data. The paper interprets this as indicating that single-page data helps the model learn output format and a stable reasoning style, while multi-page data teaches evidence localization and long-context reasoning.

The curriculum-learning ablation compares mixed training from the start against a two-stage curriculum consisting of single-page warm-up followed by multi-page training. The two-stage curriculum is reported to outperform mixed training from the start. The paper argues that this reduces cold-start difficulty and helps the model internalize the desired structured output before harder multi-page reasoning (Xiong et al., 10 Aug 2025).

The most direct diagnostic concerns evidence-page retrieval itself. The paper reports evidence-page recall of 52.70 for the baseline Qwen2.5-VL-Instruct and 91.47 for DocR1, for an improvement of +38.77 points. This is presented as direct evidence that EviGRPO teaches the model to retrieve relevant pages reliably (Xiong et al., 10 Aug 2025).

Qualitative case studies are also summarized. According to the paper, DocR1 correctly identifies sparse evidence pages, handles very long documents, produces coherent reasoning traces, and avoids answer hallucination by grounding in selected pages. A plausible implication is that the evidence-aware reward affects not only final accuracy but also the internal structure of the model’s solution behavior.

7. Limitations and interpretive significance

The paper also implies several limitations. It relies on high-quality page-level evidence annotations, which are described as expensive to obtain. The annotated training set remains relatively small, with 4.8k EviBench examples, so performance depends on careful data construction. The method does not provide a separate external retriever or explicit page-ranking module; page selection is learned implicitly and may still fail on highly ambiguous or very noisy documents. Evaluation is conducted mainly on English document benchmarks and scientific papers, so generalization to other languages or formats is not established. The paper further suggests that failure cases likely remain when evidence is distributed across many pages or when answer cues are highly implicit (Xiong et al., 10 Aug 2025).

Within these limits, EviGRPO can be understood as a GRPO-based RL framework that adds evidence-page supervision to multimodal document reasoning. Its defining contribution is to make evidence localization a rewarded intermediate behavior rather than an unobserved byproduct of answer generation. This suggests a broader methodological point for multi-page MLLMs: in sparse-evidence settings, answer supervision alone may be insufficient, and explicit optimization of coarse retrieval decisions can materially improve grounding, retrieval recall, and downstream reasoning performance.