WarriorMath: Defect-Aware Math Training

• WarriorMath is a defect-aware math data synthesis framework that generates original problems specifically targeting LLMs' failure modes.
• It employs a committee of expert LLMs to synthesize, judge, and filter problems, ensuring only those the model struggles with are retained.
• The framework progressively trains models by combining supervised fine-tuning with iterative alignment focused on correcting observed defects.

WarriorMath is a defect-aware mathematical data synthesis and training framework for improving the mathematical reasoning ability of LLMs. Its central premise is that many augmentation pipelines generate problems that are merely harder or more numerous, but not necessarily diagnostic of a model’s actual failure modes. WarriorMath instead synthesizes math problems from scratch with a committee of expert LLMs, retains only problems that the current model fails to solve, selects high-quality expert solutions through multi-agent comparison, and then trains the model progressively by combining supervised fine-tuning with iterative alignment on its own mistakes. In the reported experiments, the framework is evaluated on six mathematical benchmarks and is stated to outperform strong baselines by 12.57% on average (Chen et al., 2 Aug 2025).

1. Conceptual basis and problem formulation

WarriorMath is explicitly framed as a response to three limitations in prior math-data augmentation: increasing difficulty does not guarantee that an example targets the model’s actual failure patterns; many synthesized questions are already solvable by the base model and therefore contribute little marginal training signal; and standard pipelines are not interactive with the target model’s evolving capabilities. The framework therefore treats defects—concrete failure cases of the current model—as the primary unit of data selection and curriculum construction rather than treating difficulty alone as sufficient (Chen et al., 2 Aug 2025).

This design leads to a specific pedagogical logic. The system first asks what the model cannot do, then preferentially trains on those gaps, and then re-evaluates the improved model to discover the next layer of weaknesses. The paper describes this as “teaching according to aptitude.” In operational terms, WarriorMath is not a rephrasing or back-translation pipeline, and it is not simply a difficulty progression scheme. It is a model-adaptive loop in which synthesis, filtering, answer selection, and training are all conditioned on the learner’s current defects (Chen et al., 2 Aug 2025).

The same paper also emphasizes that the synthesized data are generated from scratch rather than expanded from an existing corpus. This matters because the resulting training set is intended to be novel, diverse, and adversarial in a way that directly probes the weaknesses of the target model rather than merely re-expressing known problem families.

2. Defect-aware synthesis and failure-based filtering

The synthesis stage uses a committee of five strong open-source math LLMs: DeepSeek-R1-Distill-Llama-70B, Qwen2.5-Math-72B-Instruct, QwQ-32B, AceMath-72B-Instruct, and Phi-4-reasoning. In each round, one model acts as the examiner and generates problems, while the others act as judges. The examiner is prompted to create problems satisfying four requirements: Quality, Difficulty, Diversity, and Challenge. The appendix prompt is described as requiring the model to act as “a world-class expert in designing extremely challenging and diverse math problems,” and final answers are required to be enclosed in \boxed{} (Chen et al., 2 Aug 2025).

Generated candidates are then passed through a quality filter. Problems that are repetitive, ambiguous, excessively difficult to learn from, poorly formatted, unverifiable, or incomplete are removed. The appendix includes discarded examples such as a Python-code prompt about log(base, number), malformed HTML math expressions, and incomplete story problems with no clear task. This filtering stage is not merely cosmetic; it constrains the downstream training set to items that are both usable and checkable (Chen et al., 2 Aug 2025).

The defect-aware step is the decisive filter. For each candidate problem, the current model is sampled for $N = 16$ rollouts. If all sampled solutions are correct, the problem is discarded. Only problems that the model answers incorrectly are retained. This converts “hardness” into an operational criterion tied to empirical failure rather than nominal problem difficulty. To preserve semantic diversity after this failure-based selection, the framework applies KCenterGreedy using all-roberta-large-v1 embeddings (Chen et al., 2 Aug 2025).

A common misunderstanding is to treat WarriorMath as a generic hard-example miner. The paper’s procedure is narrower and more structured: it mines failed examples only after expert generation and quality control, and it preserves diversity after filtering rather than allowing the curriculum to collapse onto a narrow cluster of adversarial variants.

3. Multi-expert answer generation, judging, and ranking

For each retained problem, the examiner produces a solution, the other committee members also answer it, and judges compare the responses. The paper uses an Elo rating system to reduce judge bias and to combine local and global evidence. The stated goal is to integrate local contingency—who wins the current comparison—with global consistency—who has been stronger across many battles (Chen et al., 2 Aug 2025).

The local pairwise voting scores are defined as

$$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$

where $t_A$ and $t_B$ are the number of votes won by responses $A$ and $B$. The Elo expected win probabilities are

$$X^{Elo}_{A>B} = \frac{1}{1+10^{(R_B-R_A)/400}}, \qquad X^{Elo}_{B>A} = \frac{1}{1+10^{(R_A-R_B)/400}},$$

with updates

$$R_A \gets R_A + K \times (s^i_{A>B}-X^{Elo}_{A>B}), \qquad R_B \gets R_B + K \times (s^i_{B>A}-X^{Elo}_{B>A}).$$

The final score for response $A$ is

$$e^{i}_{A} = \sum_{B \in Com \setminus A} \alpha X^{Elo}_{A>B} + (1-\alpha) x^i_{A>B}.$$

This scoring mechanism is used to select the best answer among committee responses (Chen et al., 2 Aug 2025).

The significance of this stage is that WarriorMath does not equate problem difficulty with supervision quality. It separately controls the quality of solutions through inter-model comparison. The resulting dataset is therefore a paired artifact: a set of problems that the current model fails on and a set of expert-selected responses intended to represent stronger supervisory targets.

4. Progressive training and iterative defect correction

WarriorMath uses a two-stage progressive learning framework. The first stage is supervised fine-tuning (SFT) on expert-selected outputs. The paper defines

$$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$0

where $$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$1 is the instruction, $$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$2 are responses from different expert models, and $$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$3 is the score assigned to each response. The gold response is selected by

$$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$4

These $$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$5 pairs initialize model $$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$6 via maximum-likelihood SFT (Chen et al., 2 Aug 2025).

The second stage is iterative alignment on the model’s own failures. At iteration $$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$7, for each input $$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$8, the model samples

$$x^i_{A>B} = \frac{t_A}{t_A+t_B}, \qquad x^i_{B>A} = \frac{t_B}{t_A+t_B},$$9

Each response receives reward

$t_A$0

Incorrect responses are collected as

$t_A$1

Preference pairs are then formed between the gold response and each incorrect response, and optimization uses the hybrid objective

$t_A$2

where $t_A$3 and $t_A$4 are balancing hyperparameters in the paper’s DPO-style formulation. After optimization, $t_A$5, and the loop repeats (Chen et al., 2 Aug 2025).

The training configuration reported for synthesis and SFT includes 9 generation configs with temperature $t_A$6 and top-p $t_A$7, a global batch size of 512, 448 total training steps, learning rate $t_A$8, weight decay $t_A$9, and WarmupLR with warmup ratio 0.2. For iterative alignment, the seed model is WarriorMath-Qwen-7b-SFT; per problem, $t_B$0 solutions are sampled with temperature 0.7 and top-p 0.8; $t_B$1 training pairs are used per problem; the process runs for 3 iterations; and the final tuned values are $t_B$2 and $t_B$3 (Chen et al., 2 Aug 2025).

In methodological terms, the framework is progressive because it separates broad capability acquisition from targeted defect repair. The first stage supplies a strong supervised prior; the second stage repeatedly concentrates optimization on the model’s residual error surface.

5. Empirical performance, data properties, and ablations

The paper reports results for two initializations: WarriorMath-Qwen, initialized from Qwen2.5-Math-7B, and WarriorMath-DS, initialized from DeepSeek-R1-Distill-Qwen-7B. Evaluation is conducted on AIME 2024, AIME 2025, AMC 2023, MATH-500, Minerva, and OlympiadBench with Pass@1 as the metric; results are reported as mean ± std; AIME and AMC are averaged over 10 seeds; the others are averaged over 3 seeds; and the tooling uses lighteval with a vLLM backend (Chen et al., 2 Aug 2025).

For WarriorMath-Qwen-7B, the reported scores are 48.3 ± 2.6 on AIME’24, 36.5 ± 5.0 on AIME’25, 83.0 ± 2.5 on AMC’23, 88.3 ± 1.4 on MATH500, 41.2 ± 2.8 on Minerva, and 52.1 ± 0.8 on Olympiad. For WarriorMath-DS-7B, the corresponding scores are 60.0 ± 9.1, 50.7 ± 9.1, 93.2 ± 4.9, 95.0 ± 1.0, 43.2 ± 1.9, and 69.6 ± 1.8. The paper contrasts these with Qwen2.5-Math-7B-Instruct and DeepSeek-R1-Distill-Qwen-7B, and states an average improvement of 12.57% over strong baselines (Chen et al., 2 Aug 2025).

The paper also reports a data-quality comparison on GSM8K, MATH-500, and AIME2024. WarriorMath-Qwen-7b-SFT reaches 95.7 on GSM8K, 83.8 on MATH-500, and 36.7 on AIME2024, outperforming synthesis methods including OpenMathInstruct, NuminaMath, Evol-Instruct, KPDDS, and PromptCoT. On AIME2024 in particular, the paper highlights an increase from 26.7 in PromptCoT to 36.7 for WarriorMath (Chen et al., 2 Aug 2025).

The iterative alignment study shows steady gains across three rounds: 36.7 ± 4.5 for the SFT baseline, 42.5 ± 5.5 after iteration 1, 44.7 ± 7.2 after iteration 2, and 48.3 ± 2.6 after iteration 3. The committee-size ablation further reports that increasing the number of expert models improves downstream performance: with 1 expert, AIME’24/AIME’25/AMC’23 are 19.7 ± 2.9 / 15.7 ± 2.7 / 59.5 ± 4.5; with 2 experts, they are 29.4 ± 3.2 / 23.5 ± 4.2 / 69.3 ± 3.6; and with 5 experts, 48.3 ± 2.6 / 36.5 ± 5.0 / 83.0 ± 2.5 (Chen et al., 2 Aug 2025).

The paper also analyzes novelty and diversity. ROUGE overlap is computed with DeepScaleR and Omni-MATH; most mined instructions have ROUGE < 0.3, and none exceed 0.6. Reported domain proportions include Algebra 30.3%, Applied Mathematics 11.4%, Discrete Mathematics 12.9%, Geometry 14.7%, Number Theory 13.1%, Precalculus 1.2%, Calculus 1.8%, and Differential Equations 0.5% (Chen et al., 2 Aug 2025).

6. Scope, limitations, and relation to adjacent work

WarriorMath is specifically a defect-aware, multi-expert framework for mathematical problem synthesis and progressive training. This scope distinguishes it from adjacent lines of work that target different bottlenecks. In nearby literature, the phrase “WarriorMath-style” is also used for a resource-constrained training program in which a 1.5B Qwen2.5-Math model is fine-tuned with GRPO on GSM8K using LoRA and Flash Attention 2 on a single RTX 3080 Ti 16GB, reaching 73.77 on GSM8K and 46.20 on MMLU-STEM; that work emphasizes hardware accessibility rather than defect-aware synthesis and committee-based judging (Shin, 10 Jun 2025).

Another adjacent but distinct line is WARM, a weakly supervised math word problem framework that learns latent equations from problem text and final answers, then uses the generated equations to train downstream models such as GTS and Graph2Tree. WARM addresses the cost of equation annotation and reports gains of 4.5% on Math23K and 32% on AllArith over the prior weakly supervised approach, but its central mechanism is answer-only supervision and equation generation rather than failure-driven problem synthesis and iterative preference alignment (Chatterjee et al., 2021).

Within WarriorMath itself, the stated limitation is operational rather than conceptual: as the number of expert models increases, evaluation and multi-agent coordination become time-consuming. The paper leaves more scalable multi-agent collaboration mechanisms to future work. A plausible implication is that WarriorMath’s main trade-off is not between supervision quality and defect-awareness, but between these benefits and the coordination cost of repeated expert generation, judging, and rollout-based failure detection (Chen et al., 2 Aug 2025).