---
title: 'WarriorMath: Defect-Aware Math Training'
url: https://www.emergentmind.com/topics/warriormath
type: topic
---

# WarriorMath: Defect-Aware Math Training

WarriorMath is a defect-aware mathematical data synthesis and training framework for improving the mathematical reasoning ability of large language models. Its central premise is that many augmentation pipelines generate problems that are merely harder or more numerous, but not necessarily diagnostic of a model’s actual failure modes. WarriorMath instead synthesizes math problems from scratch with a committee of expert LLMs, retains only problems that the current model fails to solve, selects high-quality expert solutions through multi-agent comparison, and then trains the model progressively by combining supervised fine-tuning with iterative alignment on its own mistakes. In the reported experiments, the framework is evaluated on six mathematical benchmarks and is stated to outperform strong baselines by 12.57% on average [2508.01245].

## 1. Conceptual basis and problem formulation

WarriorMath is explicitly framed as a response to three limitations in prior math-data augmentation: increasing difficulty does not guarantee that an example targets the model’s actual failure patterns; many synthesized questions are already solvable by the base model and therefore contribute little marginal training signal; and standard pipelines are not interactive with the target model’s evolving capabilities. The framework therefore treats defects—concrete failure cases of the current model—as the primary unit of data selection and curriculum construction rather than treating difficulty alone as sufficient [2508.01245].

This design leads to a specific pedagogical logic. The system first asks what the model cannot do, then preferentially trains on those gaps, and then re-evaluates the improved model to discover the next layer of weaknesses. The paper describes this as “teaching according to aptitude.” In operational terms, WarriorMath is not a rephrasing or back-translation pipeline, and it is not simply a difficulty progression scheme. It is a model-adaptive loop in which synthesis, filtering, answer selection, and training are all conditioned on the learner’s current defects [2508.01245].

The same paper also emphasizes that the synthesized data are generated from scratch rather than expanded from an existing corpus. This matters because the resulting training set is intended to be novel, diverse, and adversarial in a way that directly probes the weaknesses of the target model rather than merely re-expressing known problem families.

## 2. Defect-aware synthesis and failure-based filtering

The synthesis stage uses a committee of five strong open-source math LLMs: **DeepSeek-R1-Distill-Llama-70B**, **Qwen2.5-Math-72B-Instruct**, **QwQ-32B**, **AceMath-72B-Instruct**, and **Phi-4-reasoning**. In each round, one model acts as the examiner and generates problems, while the others act as judges. The examiner is prompted to create problems satisfying four requirements: **Quality**, **Difficulty**, **Diversity**, and **Challenge**. The appendix prompt is described as requiring the model to act as “a world-class expert in designing extremely challenging and diverse math problems,” and final answers are required to be enclosed in `\boxed{}` [2508.01245].

Generated candidates are then passed through a quality filter. Problems that are repetitive, ambiguous, excessively difficult to learn from, poorly formatted, unverifiable, or incomplete are removed. The appendix includes discarded examples such as a Python-code prompt about `log(base, number)`, malformed HTML math expressions, and incomplete story problems with no clear task. This filtering stage is not merely cosmetic; it constrains the downstream training set to items that are both usable and checkable [2508.01245].

The defect-aware step is the decisive filter. For each candidate problem, the current model is sampled for **\(N = 16\)** rollouts. If all sampled solutions are correct, the problem is discarded. Only problems that the model answers incorrectly are retained. This converts “hardness” into an operational criterion tied to empirical failure rather than nominal problem difficulty. To preserve semantic diversity after this failure-based selection, the framework applies **KCenterGreedy** using **all-roberta-large-v1 embeddings** [2508.01245].

A common misunderstanding is to treat WarriorMath as a generic hard-example miner. The paper’s procedure is narrower and more structured: it mines failed examples only after expert generation and quality control, and it preserves diversity after filtering rather than allowing the curriculum to collapse onto a narrow cluster of adversarial variants.

## 3. Multi-expert answer generation, judging, and ranking

For each retained problem, the examiner produces a solution, the other committee members also answer it, and judges compare the responses. The paper uses an **Elo rating system** to reduce judge bias and to combine local and global evidence. The stated goal is to integrate **local contingency**—who wins the current comparison—with **global consistency**—who has been stronger across many battles [2508.01245].

The local pairwise voting scores are defined as
\[
x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad
x^i_{B>A} = \frac{t_B}{t_A+t_B},
\]
where \(t_A\) and \(t_B\) are the number of votes won by responses \(A\) and \(B\). The Elo expected win probabilities are
\[
X^{Elo}_{A>B} = \frac{1}{1+10^{(R_B-R_A)/400}}, \qquad
X^{Elo}_{B>A} = \frac{1}{1+10^{(R_A-R_B)/400}},
\]
with updates
\[
R_A \gets R_A + K \times (s^i_{A>B}-X^{Elo}_{A>B}), \qquad
R_B \gets R_B + K \times (s^i_{B>A}-X^{Elo}_{B>A}).
\]
The final score for response \(A\) is
\[
e^{i}_{A} = \sum_{B \in Com \setminus A} \alpha X^{Elo}_{A>B} + (1-\alpha) x^i_{A>B}.
\]
This scoring mechanism is used to select the best answer among committee responses [2508.01245].

The significance of this stage is that WarriorMath does not equate problem difficulty with supervision quality. It separately controls the quality of solutions through inter-model comparison. The resulting dataset is therefore a paired artifact: a set of problems that the current model fails on and a set of expert-selected responses intended to represent stronger supervisory targets.

## 4. Progressive training and iterative defect correction

WarriorMath uses a two-stage progressive learning framework. The first stage is **supervised fine-tuning (SFT)** on expert-selected outputs. The paper defines
\[
\mathcal{D} = \{(x_i, Y_i, \{r_i^j\}_{j=1}^N)\}_{i=1}^N,
\]
where \(x_i\) is the instruction, \(Y_i = \{y_i^j\}_{j=1}^N\) are responses from different expert models, and \(r_i^j\) is the score assigned to each response. The gold response is selected by
\[
y_i^{\text{gold}} = \arg\max_{y_i^j \in Y_i} r_i^j.
\]
These \((x_i, y_i^{\text{gold}})\) pairs initialize model \(M_0\) via maximum-likelihood SFT [2508.01245].

The second stage is **iterative alignment** on the model’s own failures. At iteration \(t\), for each input \(x_i\), the model samples
\[
G_i = \{(c_i^n, y_i^n)\}_{n=1}^{N_i} \sim M_t(x_i).
\]
Each response receives reward
\[
r_i^n =
\begin{cases}
1, & \text{if } y_i^n = \hat{y}_i \\
0, & \text{otherwise}.
\end{cases}
\]
Incorrect responses are collected as
\[
G_i^{\text{neg}} = \{(c_i^n, y_i^n) \mid r_i^n = 0\}.
\]
Preference pairs are then formed between the gold response and each incorrect response, and optimization uses the hybrid objective
\[
\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{DPO}} + \alpha \mathcal{L}_{\text{NLL}},
\]
where \(\alpha\) and \(\beta\) are balancing hyperparameters in the paper’s DPO-style formulation. After optimization, \(M_{t+1} = M_\theta\), and the loop repeats [2508.01245].

The training configuration reported for synthesis and SFT includes **9 generation configs** with temperature \(t \in \{0.60,0.65,0.70\}\) and top-p \(p \in \{0.85,0.90,0.95\}\), a **global batch size of 512**, **448 total training steps**, **learning rate \(1\times 10^{-5}\)**, **weight decay \(3\times 10^{-7}\)**, and **WarmupLR** with warmup ratio **0.2**. For iterative alignment, the seed model is **WarriorMath-Qwen-7b-SFT**; per problem, **\(N = 32\)** solutions are sampled with **temperature 0.7** and **top-p 0.8**; **\(K = 10\)** training pairs are used per problem; the process runs for **3** iterations; and the final tuned values are **\(\alpha = 1\)** and **\(\beta = 0.1\)** [2508.01245].

In methodological terms, the framework is progressive because it separates broad capability acquisition from targeted defect repair. The first stage supplies a strong supervised prior; the second stage repeatedly concentrates optimization on the model’s residual error surface.

## 5. Empirical performance, data properties, and ablations

The paper reports results for two initializations: **WarriorMath-Qwen**, initialized from **Qwen2.5-Math-7B**, and **WarriorMath-DS**, initialized from **DeepSeek-R1-Distill-Qwen-7B**. Evaluation is conducted on **AIME 2024**, **AIME 2025**, **AMC 2023**, **MATH-500**, **Minerva**, and **OlympiadBench** with **Pass@1** as the metric; results are reported as **mean ± std**; **AIME** and **AMC** are averaged over **10 seeds**; the others are averaged over **3 seeds**; and the tooling uses **lighteval** with a **vLLM backend** [2508.01245].

For **WarriorMath-Qwen-7B**, the reported scores are **48.3 ± 2.6** on AIME’24, **36.5 ± 5.0** on AIME’25, **83.0 ± 2.5** on AMC’23, **88.3 ± 1.4** on MATH500, **41.2 ± 2.8** on Minerva, and **52.1 ± 0.8** on Olympiad. For **WarriorMath-DS-7B**, the corresponding scores are **60.0 ± 9.1**, **50.7 ± 9.1**, **93.2 ± 4.9**, **95.0 ± 1.0**, **43.2 ± 1.9**, and **69.6 ± 1.8**. The paper contrasts these with **Qwen2.5-Math-7B-Instruct** and **DeepSeek-R1-Distill-Qwen-7B**, and states an average improvement of **12.57%** over strong baselines [2508.01245].

The paper also reports a data-quality comparison on **GSM8K**, **MATH-500**, and **AIME2024**. **WarriorMath-Qwen-7b-SFT** reaches **95.7** on GSM8K, **83.8** on MATH-500, and **36.7** on AIME2024, outperforming synthesis methods including **OpenMathInstruct**, **NuminaMath**, **Evol-Instruct**, **KPDDS**, and **PromptCoT**. On AIME2024 in particular, the paper highlights an increase from **26.7** in PromptCoT to **36.7** for WarriorMath [2508.01245].

The iterative alignment study shows steady gains across three rounds: **36.7 ± 4.5** for the SFT baseline, **42.5 ± 5.5** after iteration 1, **44.7 ± 7.2** after iteration 2, and **48.3 ± 2.6** after iteration 3. The committee-size ablation further reports that increasing the number of expert models improves downstream performance: with **1** expert, AIME’24/AIME’25/AMC’23 are **19.7 ± 2.9 / 15.7 ± 2.7 / 59.5 ± 4.5**; with **2** experts, they are **29.4 ± 3.2 / 23.5 ± 4.2 / 69.3 ± 3.6**; and with **5** experts, **48.3 ± 2.6 / 36.5 ± 5.0 / 83.0 ± 2.5** [2508.01245].

The paper also analyzes novelty and diversity. ROUGE overlap is computed with **DeepScaleR** and **Omni-MATH**; most mined instructions have **ROUGE < 0.3**, and none exceed **0.6**. Reported domain proportions include **Algebra 30.3%**, **Applied Mathematics 11.4%**, **Discrete Mathematics 12.9%**, **Geometry 14.7%**, **Number Theory 13.1%**, **Precalculus 1.2%**, **Calculus 1.8%**, and **Differential Equations 0.5%** [2508.01245].

## 6. Scope, limitations, and relation to adjacent work

WarriorMath is specifically a defect-aware, multi-expert framework for mathematical problem synthesis and progressive training. This scope distinguishes it from adjacent lines of work that target different bottlenecks. In nearby literature, the phrase **“WarriorMath-style”** is also used for a resource-constrained training program in which a **1.5B Qwen2.5-Math model** is fine-tuned with **GRPO** on **GSM8K** using **LoRA** and **Flash Attention 2** on a single **RTX 3080 Ti 16GB**, reaching **73.77** on GSM8K and **46.20** on MMLU-STEM; that work emphasizes hardware accessibility rather than defect-aware synthesis and committee-based judging [2506.08935].

Another adjacent but distinct line is **WARM**, a weakly supervised math word problem framework that learns latent equations from problem text and final answers, then uses the generated equations to train downstream models such as **GTS** and **Graph2Tree**. WARM addresses the cost of equation annotation and reports gains of **4.5%** on **Math23K** and **32%** on **AllArith** over the prior weakly supervised approach, but its central mechanism is answer-only supervision and equation generation rather than failure-driven problem synthesis and iterative preference alignment [2104.06722].

Within WarriorMath itself, the stated limitation is operational rather than conceptual: as the number of expert models increases, evaluation and multi-agent coordination become **time-consuming**. The paper leaves more scalable multi-agent collaboration mechanisms to future work. A plausible implication is that WarriorMath’s main trade-off is not between supervision quality and defect-awareness, but between these benefits and the coordination cost of repeated expert generation, judging, and rollout-based failure detection [2508.01245].

Source: https://www.emergentmind.com/topics/warriormath