---
title: Arithmetic RLVR Benchmarks
url: https://www.emergentmind.com/topics/arithmetic-rlvr-benchmarks
type: topic
---

# Arithmetic RLVR Benchmarks

Arithmetic RLVR Benchmarks constitute a diverse and rapidly evolving suite of datasets and protocols underpinning Reinforcement Learning with Verifiable Rewards (RLVR) for arithmetic and mathematical reasoning tasks. These benchmarks are critical for evaluating and advancing the reasoning capabilities of large language models (LLMs) and, more recently, large vision-language models (LVLMs), under post-training fine-tuning regimes that rely on unambiguous, automatically evaluable reward signals.

## 1. Scope and Rationale of Arithmetic RLVR Benchmarks

Arithmetic RLVR benchmarks focus on mathematical problem-solving tasks where model outputs can be automatically, reliably scored via rule-based verification. The RLVR paradigm mandates that every problem–solution pair admits a verifiable, typically numeric, answer—criteria met by most arithmetic, algebraic, and contest-style benchmarks such as MATH500, AMC, and AIME [2509.02522, 2603.10767, 2605.00610]. By guaranteeing unambiguous reward signals (correct = +1, incorrect = 0), RLVR benchmarks enable stable, high-signal policy optimization protocols (e.g., GRPO, PPO) that drive reasoning improvements unobtainable with ambiguous or fuzzy human-labeled datasets.

These suites serve multiple roles: (1) diagnosing reasoning efficacy and generalization in LLMs; (2) serving as curriculum for RLVR; (3) enabling large-scale, multi-lingual or multi-modal evaluation; and (4) providing stringent testbeds resistant to shortcut exploitation.

## 2. Canonical Benchmarks, Datasets, and Task Taxonomy

The RLVR arithmetic benchmark landscape comprises both "contest-style" datasets with integer/numeric short answers and synthetic or MCQ-style datasets designed for automated verification.

**a. Classic Numeric-Answer Contest Benchmarks**

| Benchmark   | Problem Type                | Answer Format     | Notable Features              |
|-------------|----------------------------|-------------------|-------------------------------|
| AIME (2024/25) | Multi-step, mixed-topic    | Integer (0–999)   | 15 problems/test; increasing difficulty |
| AMC (23)    | Medium-difficulty, mixed    | MCQ, short answer | Broadest school-level coverage|
| MATH500     | Algebra, geom., comb., NT   | Numeric/boxed     | Moderate difficulty, 500 Qs   |
| OlympiadBench | Math+Science Olympiad     | Mixed freeform    | Bilingual, multimodal         |
| Minerva     | STEM word problems          | Free/numeric      | College-level, diverse domains|

**b. Multilingual and Synthetic Arithmetic Datasets**

- **mAceReason-Math** provides 143,045 rigorously cleaned and translated problems in 14 languages, with a parallel split for direct cross-lingual RLVR evaluation. Arithmetic problem types are manually categorized: ~28 % addition/subtraction, ~18 % multiplication/division, ~22 % multi-step numeric expressions, ~20 % fractions/decimals/percentages, and ~12 % integer counting/combinatorial [2603.10767]. All prompts and answers are text+LaTeX; no diagrams.

- **Golden Goose / GooseReason** synthesizes over 600,000 MCQ-style fill-in-the-middle math reasoning tasks from AoPS-Instruct, featuring masked key solution spans and eight GPT-5-generated distractors, enabling unlimited-scale RLVR augmentation [2601.22975].

- **IRPD (Image-Relation-Pair Dataset)** in the visual semantic arithmetic domain constitutes ~1,500 text-subject/object pairs per 18 ConceptNet relations, with both image and text modalities available and task variations for two- and three-term operations [2604.19567].

## 3. RLVR Task Formulation and Evaluation Protocols

A defining trait of RLVR arithmetic benchmarks is compatibility with automatic, unambiguous reward verification. Most datasets restrict themselves to tasks where a verifiable mapping $\text{reward } : (\text{problem}, \text{answer}) \mapsto \{0,1\}$ is computable via:
- String or numeric equality (e.g., Math-Verify, custom `Extract` and comparison functions for contest integer answers [2508.14029, 2603.10767]),
- Option selection for MCQ/fill-in-the-middle tasks.

**Evaluation Metrics** are tailored to stochastic policy evaluation:
- **Pass@k** (probability at least one out of $k$ sampled attempts is correct), formalized as:
  $$
  \mathrm{Pass}@k = 1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}
  $$
  where $n$ samples, $c$ correct [2509.02522, 2508.14029].
- **average@k** (fraction of questions for which at least one of $k$ samples is correct), used for small contest-style benchmarks [2605.00610].
- **Policy entropy** to monitor generation diversity and prevent mode collapse during RL [2508.14029].

Typical RLVR protocol (e.g., GRPO, PPO variants) involves grouped stochastic rollouts per problem, reward normalization, and clipped policy gradient updates.

## 4. Data Generation, Cleaning, and Scalability

Benchmarks with robust RLVR design enforce strict pre-processing:
- Removal of ambiguous, leaky, or diagram-dependent items.
- LLM-based translation and LaTeX-preserving refinement for multilingual datasets, followed by native speaker validation (e.g., mAceReason-Math’s iterative LLM–human validation pipeline, up to 5 refinement rounds) [2603.10767].
- Automated MCQ synthesis with plausible distractors for maximal reward reliability and diversity (e.g., Golden Goose’s prompt-driven span masking and distractor generation; ~70 % of MCQs deemed “effective” for RL) [2601.22975].
- For visual arithmetic benchmarks, image synthesis via conditional image models and rigorous CLIP-score/manual vetting to guarantee semantic alignment with relation triplets [2604.19567].

Reward verification is locale/format-sensitive, with number format normalization and strict output tagging (e.g., `<answer>…</answer>`) for streamlined parsing and policy update gating [2603.10767].

## 5. Experimental Use and Empirical Performance

Arithmetic benchmarks are the primary metric for progress in RLVR-based LLM post-training:

- **Contest-style tasks:** RLVR methods (e.g., GRPO, PACS, DoTS) demonstrate substantial gains in pass@k/average@k over both SFT and standard RL approaches, especially on high-difficulty testbeds such as AIME2024/2025 and MATH500. For instance, on AIME 2024, PACS achieves 59.78 % pass@256 vs. 46.46 % (PPO) and 45.42 % (GRPO), and DoTS attains 32.9 % average@32, outperforming LUFFY (+5.8 pts absolute) [2509.02522, 2605.00610]. On mAceReason-Math’s arithmetic subset, top models reach ~90 % accuracy, but RLVR is required to close the gap further [2603.10767].

- **Multilingual benchmarks:** mAceReason-Math supports RLVR in 14 languages, retaining a held-out human-validated test set and parallel translation splits to enable cross-lingual policy transfer studies.

- **Policy entropy:** RLVR tends to collapse policy diversity, harming high-pass@k; the SvS protocol (self-play with variational problem synthesis) restores entropy, realizing +18–22 pp absolute pass@32 gains on AIME24/25 [2508.14029].

- **Synthetic MCQ tasks:** Golden Goose’s MCQ format yields reliable binary rewards and measurable gains where open-ended arithmetic fail to provide trainable signals (>83 % zero accuracy for open-ended fill-in) [2601.22975].

- **Vision-language benchmarks:** SAri-RFT with IRPD achieves state-of-the-art on cross-modal relational arithmetic, validating that structured, verifiable rewards and group-based optimization extend to LVLM settings [2604.19567].

## 6. Practical Guidelines and Benchmark Integration

For RLVR on arithmetic datasets, protocols typically recommend:
- Grouped rollouts ($k=8$ or higher),
- Curriculum or difficulty-stratified sampling (for curriculum RLVR),
- Explicit output tagging and locale-matched formatting,
- Regular pass@1 or average@k monitoring on held-out test sets (e.g., mAceReason-Math’s 190-item human-validated set),
- Early stopping when validation accuracy plateaus,
- Mixed-precision decoding and controlled exploration (temperature annealing during training) [2603.10767].

Many setups incorporate synthetic augmentation (e.g., SvS for problem diversity, MCQ syntheses in GooseReason) to combat reward saturation. New strategies, such as DoTS, combine SFT and RLVR capabilities post hoc via task vector arithmetic, addressing integration challenges (e.g., 30× scale disparity, 45 % sign interference between SFT and RLVR task vectors) and achieving up to +7 pp gains over on-policy RL on AIME24 [2605.00610].

## 7. Significance, Limitations, and Extensions

Arithmetic RLVR benchmarks have set a new standard for evaluating LLM mathematical reasoning, supporting fine-grained, scalable, and cross-lingual/multimodal post-training evaluation. They have catalyzed:
- The development of new optimization algorithms (e.g., PACS’s implicit actor–critic coupling [2509.02522]),
- Empirical stratification of model strengths and weaknesses (e.g., specific subtype performance, pass@k/entropy tradeoffs),
- Scalable data construction (e.g., Golden Goose, mAceReason-Math [2601.22975, 2603.10767]),
- And generalized testbeds for integration schemes (e.g., DoTS, SAri-RFT [2605.00610, 2604.19567]).

Current limitations include incomplete public difficulty stratification and subtype breakdowns (noted in Golden Goose), occasional cue leakage in MCQ distractors, and saturation of gains without continual data augmentation.

Emerging directions include visually grounded arithmetic reasoning, fine-grained curriculum RLVR, and more robust integration of SFT with RLVR via architectural or test-time synthesis approaches.

---

**Key References**: [2603.10767], [2509.02522], [2508.14029], [2601.22975], [2604.19567], [2605.00610]

Source: https://www.emergentmind.com/topics/arithmetic-rlvr-benchmarks