MathSearch-200K for Process Reward Models

• The paper introduces MathSearch-200K, a dataset of 200K annotated reasoning trajectories that enhances PRM training with soft, continuous supervision.
• It leverages Adaptive Monte Carlo Search to dynamically assign step-level targets using uncertainty measures like Wilson confidence intervals and K-means clustering.
• Empirical results demonstrate that PRMs trained on MathSearch-200K outperform larger models and other baselines across benchmarks such as MATH, AIME, and Olympiad-Bench.

MathSearch-200K is a large-scale process supervision dataset for training Process Reward Models (PRMs) in mathematical reasoning. It is introduced together with Adaptive Monte Carlo Search (AMCS), a data-generation framework that assigns soft step-level targets to partial reasoning trajectories. The dataset consists of about 200,000 annotated reasoning trajectories / process supervision examples, each stored in the form $(p, s_{1:t}, \hat{\mu}_t)$, where $p$ is a math problem, $s_{1:t}$ is a partial reasoning trajectory up to step $t$, and $\hat{\mu}_t$ is a step/node quality score estimating how likely that partial trajectory is to eventually reach the correct final answer (Ma et al., 29 Sep 2025).

1. Definition, scope, and annotation schema

MathSearch-200K was created to provide higher-fidelity process labels for PRMs. The motivating claim is that PRMs need high-quality step-level supervision, but such supervision is difficult and expensive to obtain manually. The dataset therefore uses automatic annotation rather than human step annotation, and its labels are derived from Monte Carlo search over continuations of intermediate reasoning states (Ma et al., 29 Sep 2025).

The formal supervision target is the empirical success estimate for a partial reasoning sequence $s_{1:t}$. Given $N$ expansions and a final-answer check against the correct answer $a$, the paper defines

$$\hat{\mu}_t = \frac{1}{N} \sum \mathbb{I}(\text{expand}(s_{1:t})) = a).$$

This quantity is interpreted as the empirical probability that continuing from the current partial trajectory leads to the correct answer. In the released training format, each example is

$(p, s_{1:t}, \hat{\mu}(s_t)),$

with $p$0. The labels are explicitly described as soft, continuous supervision rather than binary right/wrong targets (Ma et al., 29 Sep 2025).

The training dataset $p$1 comprises approximately 200,000 reasoning trajectories generated by applying AMCS to problems from MATH500 and GSM8K. The paper reports mean reasoning steps of 11 steps and mean token density of 65 tokens per step. It contrasts these figures with Math-Shepherd and PRM800K, which have 6–7 steps on average and 32–46 tokens per step. This suggests that MathSearch-200K emphasizes longer intermediate trajectories and denser step annotations (Ma et al., 29 Sep 2025).

2. Dataset construction through Adaptive Monte Carlo Search

AMCS is the framework used to generate MathSearch-200K. Its stated contribution is to transform data generation from fixed, static to adaptive, dynamic search at the level of node value estimation and path expansion. The two major components are adaptive node value estimation and temporally adaptive Monte Carlo path expansion (Ma et al., 29 Sep 2025).

For each math problem $p$2, AMCS generates a reasoning tree of candidate partial trajectories. For a node $p$3, it begins with an initial sample count $p$4 using non-greedy decoding. Each rollout $p$5 is represented by a two-dimensional feature vector

$p$6

where confidence is measured by average token-level negative log-likelihood,

$p$7

and complexity is measured by

$p$8

with $p$9. In the appendix this feature is also written as

$s_{1:t}$0

Because the two features are on different scales, z-score standardization is applied: $s_{1:t}$1 where

$s_{1:t}$2

and $s_{1:t}$3. K-Means is then used to partition the rollouts into $s_{1:t}$4 clusters, denoted

$s_{1:t}$5

At the cluster level, the framework maintains the number of successful rollouts $s_{1:t}$6, the total rollouts $s_{1:t}$7, and the estimated success probability

$s_{1:t}$8

After adaptive refinement and path expansion, AMCS extracts supervision tuples of the form

$s_{1:t}$9

and aggregates them across source problems into MathSearch-200K (Ma et al., 29 Sep 2025).

3. Adaptive node value estimation and temporally adaptive search policy

The first algorithmic pillar of AMCS is adaptive node value estimation. Instead of assigning a fixed budget to every intermediate node, it quantifies uncertainty at the cluster level and reallocates computation to the most uncertain regions of the search tree (Ma et al., 29 Sep 2025).

Cluster-level uncertainty $t$0 is defined as half the width of the Wilson confidence interval: $t$1 where $t$2. The paper explicitly prefers Wilson over Wald/normal approximation because it behaves better for small $t$3 and for probabilities near 0 or 1. Node-level uncertainty is then aggregated as

$t$4

Sampling continues only on active clusters: $t$5 The next cluster selected for refinement is

$t$6

and the number of new rollouts is

$t$7

New rollouts are assigned to the nearest cluster centroid in standardized feature space: $t$8

Adaptive refinement terminates when any of the following conditions holds: $t$9 The paper reports $\hat{\mu}_t$0 and $\hat{\mu}_t$1. The final node value is the weighted average of cluster estimates,

$\hat{\mu}_t$2

The second pillar is temporally adaptive Monte Carlo path expansion. The exploitation score is

$\hat{\mu}_t$3

where $\hat{\mu}_t$4 is the normalized length of the problem statement and $\hat{\mu}_t$5. The exploration bonus is

$\hat{\mu}_t$6

These are combined through a time-varying policy

$\hat{\mu}_t$7

Early iterations therefore emphasize exploration, whereas later iterations shift toward exploitation. The paper presents this temporal adaptation as the second main novelty of AMCS (Ma et al., 29 Sep 2025).

4. Use in PRM training and inference

MathSearch-200K is used to train a PRM initialized from Qwen2.5-Math-7B-Instruct, yielding the model called Qwen2.5-Math-7B-PRM-AMCS. The targets are not binarized. Instead, the PRM is trained directly on the soft supervision values $\hat{\mu}_t$8 (Ma et al., 29 Sep 2025).

The loss is binary cross-entropy with soft labels: $\hat{\mu}_t$9 where $s_{1:t}$0 is the PRM prediction. The interpretation given in the paper is direct: values near 1 provide strong positive supervision, values near 0 provide strong negative supervision, and values near 0.5 preserve uncertainty information from AMCS.

At inference time, the trained PRM is used as a verifier to guide actor models under Beam Search, Best-of-N, and MCTS. The paper also evaluates PRM-based PPO fine-tuning, where the PRM supplies the reward signal. This positions MathSearch-200K not merely as a static corpus of annotated trajectories, but as training data for verifiers and reward models that affect both search-time selection and reinforcement learning (Ma et al., 29 Sep 2025).

5. Reported empirical properties and downstream results

The paper reports several dataset-generation statistics that characterize how AMCS allocates computation. Uncertain nodes with $s_{1:t}$1 receive 20.0 rollouts on average, whereas confident low-value nodes with $s_{1:t}$2 receive 6.9 rollouts and confident high-value nodes with $s_{1:t}$3 receive 7.1 rollouts. The paper describes this as roughly 3× higher sampling intensity on uncertain nodes. Search depth is also higher in the intermediate-value regime, with depth 5.0–5.1 compared with depth 3.1 for low-valued nodes; nodes explored are 14–16 for intermediate values and 8–10 for extremes (Ma et al., 29 Sep 2025).

The central empirical claim is that PRMs trained on MathSearch-200K outperform prior PRMs. With GLM-4-9B + MCTS + Qwen2.5-Math-7B-PRM-AMCS, the paper reports 15.0 on AIME, 76.2 on MATH, 22.1 on Olympiad-Bench, 19.0 on OmniMATH, and 33.1 average. In the same GLM-4-9B + MCTS setting, the reported MATH scores are 70.4 for the base Qwen2.5-Math-7B-Instruct verifier, 69.0 for Deepseek PRM, 76.0 for PRM800K, 74.2 for Math-Shepherd, and 76.2 for PRM-AMCS.

The paper further highlights a model-scale comparison: a 7B model supervised by Qwen2.5-Math-7B-PRM-AMCS surpasses a 72B model with weaker supervision on MATH500, with 70.6% for the 7B model and 65.0% for the 72B model. In PPO experiments on Hungarian Math, the AMCS-trained PRM reaches 53.1 pass@1 and 75.0 pass@5, compared with 43.8 / 62.5 for PRM800K, 46.9 / 71.9 for Skywork-o1-Open-PRM, and 46.9 / 65.6 for Qwen2.5-Math-PRM-7B. These results are presented as evidence of consistent advantages on out-of-distribution problems and stronger reward quality for reinforcement learning (Ma et al., 29 Sep 2025).

6. Limitations, assumptions, and name disambiguation

The paper does not present MathSearch-200K as a complete answer to process supervision. Several assumptions are explicit or implicit: final-answer correctness is treated as the signal for estimating step quality; rollout clustering by confidence and complexity is assumed to capture meaningful strategy differences; Wilson-interval uncertainty is assumed to be an appropriate basis for adaptive allocation; and search quality depends on the actor model generating the rollouts (Ma et al., 29 Sep 2025).

Practical limitations are also stated. Data generation used four Tesla A800 GPUs for about one week, and PRM training used four A800s for about three days. Exact train/validation/test splits, class balances, and finer metadata are not given in the paper text. The source data is limited to MATH500 and GSM8K, and Monte Carlo supervision remains approximate rather than equivalent to expert proofs of step correctness. The paper’s outlook is therefore oriented toward extending adaptive search-based process supervision beyond the current setting (Ma et al., 29 Sep 2025).

The name can overlap with older work on mathematical information retrieval, but those systems refer to a different object. “Making Math Searchable in Wikipedia” presents MathSearch as an open-source MediaWiki site search extension that combines MathWebSearch for formulas and Lucene for text, rather than a process supervision dataset (Schubotz, 2013). Likewise, “SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics” introduces ActiveViewPose-200K and explicitly states that “MathSearch-200K” does not appear in that paper (Liu et al., 12 Mar 2026). A plausible implication is that MathSearch-200K should be identified specifically with AMCS-based process supervision for PRMs, not with Wikipedia formula retrieval or with unrelated “200K” datasets in other domains.