Arithmetic RLVR Benchmarks

• Arithmetic RLVR Benchmarks are a suite of datasets and protocols designed to evaluate and improve mathematical reasoning through verifiable, rule-based reward signals.
• They encompass contest-style, multilingual, and visual arithmetic challenges, enabling robust assessment of LLMs and LVLMs across diverse problem types.
• Benchmarks leverage automatic numerical verification and advanced RL techniques like GRPO and PPO to enhance model accuracy and policy diversity.

Arithmetic RLVR Benchmarks constitute a diverse and rapidly evolving suite of datasets and protocols underpinning Reinforcement Learning with Verifiable Rewards (RLVR) for arithmetic and mathematical reasoning tasks. These benchmarks are critical for evaluating and advancing the reasoning capabilities of LLMs and, more recently, large vision-LLMs (LVLMs), under post-training fine-tuning regimes that rely on unambiguous, automatically evaluable reward signals.

1. Scope and Rationale of Arithmetic RLVR Benchmarks

Arithmetic RLVR benchmarks focus on mathematical problem-solving tasks where model outputs can be automatically, reliably scored via rule-based verification. The RLVR paradigm mandates that every problem–solution pair admits a verifiable, typically numeric, answer—criteria met by most arithmetic, algebraic, and contest-style benchmarks such as MATH500, AMC, and AIME (Li et al., 2 Sep 2025, Dobler et al., 11 Mar 2026, Yuan et al., 1 May 2026). By guaranteeing unambiguous reward signals (correct = +1, incorrect = 0), RLVR benchmarks enable stable, high-signal policy optimization protocols (e.g., GRPO, PPO) that drive reasoning improvements unobtainable with ambiguous or fuzzy human-labeled datasets.

These suites serve multiple roles: (1) diagnosing reasoning efficacy and generalization in LLMs; (2) serving as curriculum for RLVR; (3) enabling large-scale, multi-lingual or multi-modal evaluation; and (4) providing stringent testbeds resistant to shortcut exploitation.

2. Canonical Benchmarks, Datasets, and Task Taxonomy

The RLVR arithmetic benchmark landscape comprises both "contest-style" datasets with integer/numeric short answers and synthetic or MCQ-style datasets designed for automated verification.

a. Classic Numeric-Answer Contest Benchmarks

Benchmark	Problem Type	Answer Format	Notable Features
AIME (2024/25)	Multi-step, mixed-topic	Integer (0–999)	15 problems/test; increasing difficulty
AMC (23)	Medium-difficulty, mixed	MCQ, short answer	Broadest school-level coverage
MATH500	Algebra, geom., comb., NT	Numeric/boxed	Moderate difficulty, 500 Qs
OlympiadBench	Math+Science Olympiad	Mixed freeform	Bilingual, multimodal
Minerva	STEM word problems	Free/numeric	College-level, diverse domains

b. Multilingual and Synthetic Arithmetic Datasets

• mAceReason-Math provides 143,045 rigorously cleaned and translated problems in 14 languages, with a parallel split for direct cross-lingual RLVR evaluation. Arithmetic problem types are manually categorized: ~28 % addition/subtraction, ~18 % multiplication/division, ~22 % multi-step numeric expressions, ~20 % fractions/decimals/percentages, and ~12 % integer counting/combinatorial (Dobler et al., 11 Mar 2026). All prompts and answers are text+LaTeX; no diagrams.
• Golden Goose / GooseReason synthesizes over 600,000 MCQ-style fill-in-the-middle math reasoning tasks from AoPS-Instruct, featuring masked key solution spans and eight GPT-5-generated distractors, enabling unlimited-scale RLVR augmentation (Lu et al., 30 Jan 2026).
• IRPD (Image-Relation-Pair Dataset) in the visual semantic arithmetic domain constitutes ~1,500 text-subject/object pairs per 18 ConceptNet relations, with both image and text modalities available and task variations for two- and three-term operations (Xu et al., 21 Apr 2026).

3. RLVR Task Formulation and Evaluation Protocols

A defining trait of RLVR arithmetic benchmarks is compatibility with automatic, unambiguous reward verification. Most datasets restrict themselves to tasks where a verifiable mapping $$\text{reward } : (\text{problem}, \text{answer}) \mapsto \{0,1\}$$ is computable via:

• String or numeric equality (e.g., Math-Verify, custom Extract and comparison functions for contest integer answers (Liang et al., 19 Aug 2025, Dobler et al., 11 Mar 2026)),
• Option selection for MCQ/fill-in-the-middle tasks.

Evaluation Metrics are tailored to stochastic policy evaluation:

• Pass@k (probability at least one out of $k$ sampled attempts is correct), formalized as:

$$\mathrm{Pass}@k = 1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}$$

where $n$ samples, $c$ correct (Li et al., 2 Sep 2025, Liang et al., 19 Aug 2025).

• average@k (fraction of questions for which at least one of $k$ samples is correct), used for small contest-style benchmarks (Yuan et al., 1 May 2026).
• Policy entropy to monitor generation diversity and prevent mode collapse during RL (Liang et al., 19 Aug 2025).

Typical RLVR protocol (e.g., GRPO, PPO variants) involves grouped stochastic rollouts per problem, reward normalization, and clipped policy gradient updates.

4. Data Generation, Cleaning, and Scalability

Benchmarks with robust RLVR design enforce strict pre-processing:

• Removal of ambiguous, leaky, or diagram-dependent items.
• LLM-based translation and LaTeX-preserving refinement for multilingual datasets, followed by native speaker validation (e.g., mAceReason-Math’s iterative LLM–human validation pipeline, up to 5 refinement rounds) (Dobler et al., 11 Mar 2026).
• Automated MCQ synthesis with plausible distractors for maximal reward reliability and diversity (e.g., Golden Goose’s prompt-driven span masking and distractor generation; ~70 % of MCQs deemed “effective” for RL) (Lu et al., 30 Jan 2026).
• For visual arithmetic benchmarks, image synthesis via conditional image models and rigorous CLIP-score/manual vetting to guarantee semantic alignment with relation triplets (Xu et al., 21 Apr 2026).

Reward verification is locale/format-sensitive, with number format normalization and strict output tagging (e.g., <answer>…</answer>) for streamlined parsing and policy update gating (Dobler et al., 11 Mar 2026).

5. Experimental Use and Empirical Performance

Arithmetic benchmarks are the primary metric for progress in RLVR-based LLM post-training:

• Contest-style tasks: RLVR methods (e.g., GRPO, PACS, DoTS) demonstrate substantial gains in pass@k/average@k over both SFT and standard RL approaches, especially on high-difficulty testbeds such as AIME2024/2025 and MATH500. For instance, on AIME 2024, PACS achieves 59.78 % pass@256 vs. 46.46 % (PPO) and 45.42 % (GRPO), and DoTS attains 32.9 % average@32, outperforming LUFFY (+5.8 pts absolute) (Li et al., 2 Sep 2025, Yuan et al., 1 May 2026). On mAceReason-Math’s arithmetic subset, top models reach ~90 % accuracy, but RLVR is required to close the gap further (Dobler et al., 11 Mar 2026).
• Multilingual benchmarks: mAceReason-Math supports RLVR in 14 languages, retaining a held-out human-validated test set and parallel translation splits to enable cross-lingual policy transfer studies.
• Policy entropy: RLVR tends to collapse policy diversity, harming high-pass@k; the SvS protocol (self-play with variational problem synthesis) restores entropy, realizing +18–22 pp absolute pass@32 gains on AIME24/25 (Liang et al., 19 Aug 2025).
• Synthetic MCQ tasks: Golden Goose’s MCQ format yields reliable binary rewards and measurable gains where open-ended arithmetic fail to provide trainable signals (>83 % zero accuracy for open-ended fill-in) (Lu et al., 30 Jan 2026).
• Vision-language benchmarks: SAri-RFT with IRPD achieves state-of-the-art on cross-modal relational arithmetic, validating that structured, verifiable rewards and group-based optimization extend to LVLM settings (Xu et al., 21 Apr 2026).

6. Practical Guidelines and Benchmark Integration

For RLVR on arithmetic datasets, protocols typically recommend:

• Grouped rollouts ($k=8$ or higher),
• Curriculum or difficulty-stratified sampling (for curriculum RLVR),
• Explicit output tagging and locale-matched formatting,
• Regular pass@1 or average@k monitoring on held-out test sets (e.g., mAceReason-Math’s 190-item human-validated set),
• Early stopping when validation accuracy plateaus,
• Mixed-precision decoding and controlled exploration (temperature annealing during training) (Dobler et al., 11 Mar 2026).

Many setups incorporate synthetic augmentation (e.g., SvS for problem diversity, MCQ syntheses in GooseReason) to combat reward saturation. New strategies, such as DoTS, combine SFT and RLVR capabilities post hoc via task vector arithmetic, addressing integration challenges (e.g., 30× scale disparity, 45 % sign interference between SFT and RLVR task vectors) and achieving up to +7 pp gains over on-policy RL on AIME24 (Yuan et al., 1 May 2026).

7. Significance, Limitations, and Extensions

Arithmetic RLVR benchmarks have set a new standard for evaluating LLM mathematical reasoning, supporting fine-grained, scalable, and cross-lingual/multimodal post-training evaluation. They have catalyzed:

• The development of new optimization algorithms (e.g., PACS’s implicit actor–critic coupling (Li et al., 2 Sep 2025)),
• Empirical stratification of model strengths and weaknesses (e.g., specific subtype performance, pass@k/entropy tradeoffs),
• Scalable data construction (e.g., Golden Goose, mAceReason-Math (Lu et al., 30 Jan 2026, Dobler et al., 11 Mar 2026)),
• And generalized testbeds for integration schemes (e.g., DoTS, SAri-RFT (Yuan et al., 1 May 2026, Xu et al., 21 Apr 2026)).

Current limitations include incomplete public difficulty stratification and subtype breakdowns (noted in Golden Goose), occasional cue leakage in MCQ distractors, and saturation of gains without continual data augmentation.

Emerging directions include visually grounded arithmetic reasoning, fine-grained curriculum RLVR, and more robust integration of SFT with RLVR via architectural or test-time synthesis approaches.

---

Key References: (Dobler et al., 11 Mar 2026, Li et al., 2 Sep 2025, Liang et al., 19 Aug 2025, Lu et al., 30 Jan 2026, Xu et al., 21 Apr 2026, Yuan et al., 1 May 2026)