---
title: EviGRPO for Multi-Page Document Reasoning
url: https://www.emergentmind.com/topics/evidence-page-guided-grpo-evigrpo
type: topic
---
# EviGRPO for Multi-Page Document Reasoning
Searching arXiv for the specified paper and closely related context papers to ground the article.
Evidence Page-Guided GRPO (EviGRPO) is a reinforcement-learning framework introduced in "DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding" for multimodal large language models (MLLMs) operating on multi-page documents [2508.07313]. It extends GRPO with an explicit, verifiable reward for page retrieval, with the stated goal of making the model follow a coarse-to-fine reasoning strategy: first retrieve relevant pages, then generate an answer. In DocR1, this framework is combined with structured outputs, a two-stage annotation pipeline, and a curriculum learning strategy, and the resulting model is reported to achieve state-of-the-art performance on multi-page tasks while consistently maintaining strong results on single-page benchmarks [2508.07313].
## 1. Problem setting and motivation
EviGRPO is designed for **multi-page document understanding**, where the core difficulty is not only reading text and figures on individual pages but also **finding the right evidence pages** and then reasoning across them [2508.07313]. The motivating document types explicitly include scientific papers, contracts, reports, and slide decks. The paper characterizes the task as requiring **page-level retrieval** of sparse evidence, **multi-hop reasoning** across pages, and robustness to irrelevant pages and long contexts.
The central motivation is that standard supervised fine-tuning (SFT), or standard GRPO using answer correctness alone, does not sufficiently teach the model to **localize evidence**. In sparse multi-page settings, a model may sometimes guess the answer without truly identifying supporting pages, which the paper associates with weaker reliability and generalization. EviGRPO is introduced to explicitly reward three factors: correct output format, correct final answer, and correct evidence-page selection [2508.07313].
This framing places evidence localization inside the learning objective rather than treating it as a downstream diagnostic. A plausible implication is that the method targets a failure mode common in long-context MLLMs: superficially correct answering without verifiable grounding in the document.
## 2. Objective, outputs, and optimization
DocR1 is trained to produce structured outputs containing three fields:
- ` ... ` for the reasoning trace
- ` ... ` for page-level evidence judgment
- ` ... ` for the final answer
The evidence-page format is either a list of page indices or a binary T/F label per page. The main method uses the per-page T/F format **with page count not explicitly given in the prompt**, which, according to the paper, forces the model to infer the number of pages and reason more carefully [2508.07313].
For each sample with question $q$ and $N$ document images, the current policy generates $G$ candidate responses $\mathcal{O} = \{o_1,\dots,o_G\}$. Each response receives a verifiable reward
$$
r_i = r_i^{\text{format}} + r_i^{\text{acc}} + r_i^{\text{evi}},
$$
where $r_i^{\text{format}} \in \{0,1\}$ indicates whether the output strictly follows the required format, $r_i^{\text{acc}}$ is answer accuracy measured by **ANLS**, and $r_i^{\text{evi}}$ is the evidence-page reward [2508.07313].
The evidence reward is defined as an F1-style overlap between predicted evidence pages $P_i$ and ground-truth evidence pages $G_i$:
$$
r_i^{\text{evi}}=
\begin{cases}
\frac{2 \cdot |P_i \cap G_i|}{|P_i| + |G_i|}, & \text{if } |P_i| + |G_i| > 0 \text{ and } N = N_i' \\
0, & \text{otherwise.}
\end{cases}
$$
Here $N_i'$ is the number of predicted evidence-page judgments. The paper explicitly motivates the F1 design by noting that simple accuracy would be misleading in sparse-evidence settings, since predicting all pages as irrelevant could appear artificially good. The formulation is intended to reward both precision and recall of evidence-page prediction, while setting the evidence reward to zero if the model predicts the wrong number of page judgments [2508.07313].
As in GRPO, rewards are normalized within the sampled group:
$$
A_i = \frac{r_i - \mathrm{mean}(\{r_i\}_{i=1}^{G})}{\mathrm{std}(\{r_i\}_{i=1}^{G})}.
$$
The policy update then uses the standard GRPO clipped objective with a KL penalty to keep the policy near the reference model:
$$
\max_{\pi_\theta}\; \mathbb{E}_{\mathcal{O} \sim \pi_{\theta_{\text{old}}}}
\left[
\min\left(
\operatorname{clip}\left(
\frac{\pi_\theta(o_i)}{\pi_{\theta_{\text{old}}}(o_i)},
1-\epsilon, 1+\epsilon
\right)A_i,\;
\frac{\pi_\theta(o_i)}{\pi_{\theta_{\text{old}}}(o_i)}A_i
\right)
- \beta\, D_{\mathrm{KL}}(\pi_\theta \,\|\, \pi_{\text{ref}})
\right].
$$
The stated role of these terms is standard: $\pi_\theta$ is the current policy, $\pi_{\theta_{\text{old}}}$ is the previous policy used to generate rollouts, $\epsilon$ is the clipping threshold, $A_i$ is the group-normalized advantage, $\beta$ is the KL penalty weight, and $\pi_{\text{ref}}$ is a frozen reference policy. The paper states that this preserves training stability and prevents the policy from drifting too far from the base instruction-tuned model [2508.07313].
The resulting reward structure is explicitly interpreted as encouraging a hierarchical decision process. The **coarse stage** identifies which pages contain relevant evidence, and the **fine stage** reasons over the selected pages and answers. In the paper’s terminology, the evidence reward directly supervises the coarse stage, while answer ANLS supervises the fine stage.
## 3. Evidence localization as policy behavior
A defining feature of EviGRPO is that evidence-page retrieval is learned **implicitly through RL** rather than delegated to an external retriever. The mechanism described in the paper has three components: the model must output evidence-page judgments before the answer, correct page selection is explicitly rewarded, and the prompt structure forces per-page judgments. Retrieval is therefore part of the policy output rather than a separate module [2508.07313].
This is also the paper’s main clarification against a common misunderstanding. Evidence retrieval in DocR1 is **not** a separate page-ranking subsystem. Instead, the model is trained to “self-retrieve” the relevant pages as an intermediate step. The novelty is therefore concentrated in the training framework and output specification, not in a new visual encoder or transformer backbone.
The ablation on page-selection formats is used to support this claim. The paper compares three formats:
- **PSF-1**: output page indices directly
- **PSF-2**: binary T/F labels with explicit page count
- **Ours**: binary T/F labels without explicitly giving the count
The reported finding is that the main method performs best. The paper attributes weaker performance in PSF-1 to the fact that direct index output does not force exhaustive per-page decisions, and it argues that PSF-2 can be partially gamed because the page count is known. By contrast, the chosen format requires the model to infer the page count and make per-page judgments more genuinely [2508.07313].
This suggests that EviGRPO is not only a reward modification but also a protocol for structuring the action space. The page-localization subproblem is made explicit in the output, then coupled to a verifiable reward, so that retrieval competence emerges as a directly optimized behavior.
## 4. Annotation pipeline and data resources
To support training with limited supervision, the paper describes a two-stage annotation procedure. In the **generation stage**, Gemini 2.5 Flash generates initial annotations from a task-specific prompt. In the **verification stage**, the same MLLM checks the generated annotation again, and only samples whose predicted output remains consistent with ground truth are retained. The paper presents this as a self-checking pipeline for filtering noisy labels and obtaining higher-quality data [2508.07313].
The framework is trained with **EviBench** and evaluated in part with **ArxivFullQA**. Their construction is summarized below.
| Resource | Composition | Purpose |
|---|---|---|
| **EviBench** | **4.8k examples**: **1.3k single-page samples** and **3.5k multi-page samples** | High-quality training set |
| **ArxivFullQA** | **8.6k QA samples** based on DocMatrix arXiv scientific papers | Evaluation benchmark; also includes a training subset |
The **single-page** portion of EviBench contains 13 datasets, with 100 samples each: DocVQA, InfographicVQA, ChartQA, DeepForm, DVQA, FigureQA, KleisterCharity, OCRVQA, TabFact, TextCaps, TextVQA, VisualMRC, and WikiTableQuestions. These were chosen to cover diverse layouts, charts, tables, forms, and text-rich images [2508.07313].
The **multi-page** portion contains five main multi-page datasets plus a paper-reading subset: DUDE with 1000 samples, MP-DocVQA with 500, TATDoc with 500, SlideVQA with 500, MultiHiertt with 500, and $\text{ArxivFullQA}_{\text{train}}$ with 500, yielding 3.5k multi-page samples in total.
ArxivFullQA is presented as both a training subset and a new evaluation benchmark. Its questions cover seven categories: factual, reasoning, comparison, summary, procedural, motivation, and result. The paper highlights two annotation differences relative to EviBench. First, in stage 1, the input is **LaTeX-formatted text** rather than images, with the stated effect of improving question generation and answer quality. Second, in stage 2, the model is given the **visual full paper** and must answer the annotated question; only consistent outputs are retained. The paper therefore characterizes ArxivFullQA as a benchmark for full-paper understanding rather than only page-level question answering [2508.07313].
## 5. Model instantiation and empirical performance
DocR1 does not introduce a new backbone architecture from scratch. It is initialized from **Qwen2.5-VL-Instruct (7B)**. The paper gives three reasons for this choice: the base model already has instruction-tuned multimodal reasoning ability, it avoids a “cold-start” RL problem, and large-scale chain-of-thought labels for document reasoning are scarce. The novelty is thus stated to lie primarily in the **training framework**, not in a new visual encoder or transformer architecture [2508.07313].
The reported training setup uses **8 NVIDIA A100 GPUs**, a **batch size of 16**, **$G=8$** candidates per sample, **$\beta=0.04$** for the KL penalty, and image resolution capped at **$1024 \times 28 \times 28$**. Evaluation covers six single-page benchmarks—DocVQA, InfographicVQA, WikiTableQuestions, TabFact, TextVQA, and VisualMRC—and seven multi-page benchmarks—DUDE, MP-DocVQA, TATDoc, MultiChartQA, MultiHiertt, SlideVQA, and ArxivFullQA [2508.07313].
On **single-page** benchmarks, DocR1 is reported as competitive with Qwen2.5-VL-Instruct and slightly better on some tasks, with the following scores: DocVQA **95.1**, InfoVQA **82.6**, WTQ **63.1**, TabFact **79.6**, TextVQA **81.0**, and VisualMRC **251.6**. The paper states that it is especially strong on table-related tasks, specifically WTQ and TabFact, while maintaining comparable performance elsewhere.
On **multi-page** benchmarks, the reported scores are:
| Benchmark | Score |
|---|---:|
| MP-DocVQA | 87.45 |
| DUDE | 54.39 |
| SlideVQA | 71.96 |
| MultiChartQA | 62.41 |
| MultiHiertt | 33.88 |
| TATDoc | 64.80 |
| ArxivFullQA | 40.65 |
The corresponding average is reported as **59.36**, which the paper states is **6.93 points** higher than the strongest baseline average reported there. The gains are described as especially large on MultiChartQA, MultiHiertt, TATDoc, and ArxivFullQA, and the paper further states that the model surpasses larger 32B and 38B baselines, suggesting that the training strategy matters more than scale alone in this setting [2508.07313].
## 6. Ablations and diagnostic analyses
The paper presents multiple ablations to isolate the contribution of evidence-page guidance. Using the same mixed training data, **SFT** is reported to yield limited gains, **GRPO** improves many tasks substantially, and **EviGRPO** is generally better than standard GRPO [2508.07313]. The intended interpretation is that RL is useful for document reasoning, but explicit evidence-page supervision further improves the reasoning process.
A separate analysis examines data composition. Training on **single-page only** is reported to harm multi-page reasoning significantly. Training on **multi-page only** helps more, but less than combining both. The best overall results come from combining single-page and multi-page data. The paper interprets this as indicating that single-page data helps the model learn output format and a stable reasoning style, while multi-page data teaches evidence localization and long-context reasoning.
The curriculum-learning ablation compares mixed training from the start against a two-stage curriculum consisting of **single-page warm-up** followed by **multi-page training**. The two-stage curriculum is reported to outperform mixed training from the start. The paper argues that this reduces cold-start difficulty and helps the model internalize the desired structured output before harder multi-page reasoning [2508.07313].
The most direct diagnostic concerns evidence-page retrieval itself. The paper reports evidence-page recall of **52.70** for the baseline Qwen2.5-VL-Instruct and **91.47** for DocR1, for an improvement of **+38.77 points**. This is presented as direct evidence that EviGRPO teaches the model to retrieve relevant pages reliably [2508.07313].
Qualitative case studies are also summarized. According to the paper, DocR1 correctly identifies sparse evidence pages, handles very long documents, produces coherent reasoning traces, and avoids answer hallucination by grounding in selected pages. A plausible implication is that the evidence-aware reward affects not only final accuracy but also the internal structure of the model’s solution behavior.
## 7. Limitations and interpretive significance
The paper also implies several limitations. It relies on **high-quality page-level evidence annotations**, which are described as expensive to obtain. The annotated training set remains relatively small, with **4.8k EviBench examples**, so performance depends on careful data construction. The method does not provide a separate external retriever or explicit page-ranking module; page selection is learned implicitly and may still fail on highly ambiguous or very noisy documents. Evaluation is conducted mainly on English document benchmarks and scientific papers, so generalization to other languages or formats is not established. The paper further suggests that failure cases likely remain when evidence is distributed across many pages or when answer cues are highly implicit [2508.07313].
Within these limits, EviGRPO can be understood as a GRPO-based RL framework that adds **evidence-page supervision** to multimodal document reasoning. Its defining contribution is to make evidence localization a rewarded intermediate behavior rather than an unobserved byproduct of answer generation. This suggests a broader methodological point for multi-page MLLMs: in sparse-evidence settings, answer supervision alone may be insufficient, and explicit optimization of coarse retrieval decisions can materially improve grounding, retrieval recall, and downstream reasoning performance.
Source: https://www.emergentmind.com/topics/evidence-page-guided-grpo-evigrpo