MaxProof: Scalable Proof Generation for Competitions

• MaxProof is a population-level framework for generating, verifying, and refining competition-level mathematical proofs.
• It leverages integrated generative-verifier reinforcement learning and population-based search to tackle advanced problems like IMO and USAMO.
• The system unifies Proof, Verifier, and Fixer Experts within the M3 model, enabling tournament-style selection and robust error correction.

MaxProof is a population-level test-time scaling framework for competition-level mathematical proof generation, verification, and refinement, developed in the context of the MiniMax-M3 series of LLMs. It operationalizes tightly integrated generative-verifier reinforcement learning (RL) and population-based search to address the challenge of reliably producing top-quality long-form solutions to advanced mathematical problems, such as those on the International Mathematical Olympiad (IMO) and the United States of America Mathematical Olympiad (USAMO). MaxProof treats the underlying M3 model as a composite executor—generator, verifier, refiner, and ranker—and systematically orchestrates population-based proof search, error critique-and-fix, and final tournament-style selection to robustly output competition-caliber proofs (Chen et al., 11 Jun 2026).

1. Atomic Capabilities of the M3 Model

The M3 model underlying MaxProof is trained to unify three atomic proof-oriented capabilities, often referred to as "experts": Proof Expert (generation), Verifier Expert (critique and scoring), and Fixer Expert (critique-conditioned repair). Each is initially trained separately via RL and distillation, then merged into a unified, prompt-driven decoder-only LLM.

• Proof Expert: Generates long-form candidate proofs $y = (y_1, \ldots, y_T)$ conditioned on a given problem statement $p$. Trained via RL with a frozen, defense-in-depth generative verifier $\phi$ as the reward, using standard decoder-only transformer architecture and prompt engineering.
• Verifier Expert: Given $(p, y)$, produces a step-by-step assessment, a numbered list of local errors (or “none”), and a verdict in $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$. Trained via distillation from the multi-judge verifier $\phi$, using a critique-rich and rebalance dataset.
• Fixer Expert: Given $$(p, \texttt{flawed\_proof}, \langle \texttt{errors} \rangle)$$, outputs corrected proofs explicitly addressing those errors. Fine-tuned by rejection sampling: candidate fixes are rescored by $\phi$, and only those judged “no_errors” are retained for further supervision.

This architecture allows M3 to function flexibly in multiple roles at inference, supporting population search, multi-perspective critique, and interactive refinement.

2. Defense-in-Depth Generative-Verifier Reinforcement Learning

The foundation of M3's capabilities is RL training under a robust generative verifier pipeline designed for minimal false positives, implemented as $\phi$.

Verifier $\phi$ Pipeline

$p$0 comprises four sequential filtering and aggregation layers:

1. Bad-Case Filter: Discards proofs that are empty, have unclosed blocks, trivial loops, or are anomalously short/long; assigns score 0.
2. Solution Normalizer: Strips headers and footers to canonicalize content.
3. Multi-Judge Scoring: Invokes three parallel LLM “judges”: two rubric-guided, one freeform, each returning a score in $p$1 and a structured critique.
4. Pessimistic Min Aggregation: Final score is $p$2, imposing a conservative standard that minimizes false positives at the expense of increased false negatives.

This design explicitly targets reward hacking paths observed in earlier versions: length bias, prompt format exploits, vacuous semantic “shortcuts,” and judge-specific artifacts.

CISPO RL Objective

Training leverages the clipped-ratio policy gradient (CISPO) objective, using population-based group statistics to normalize advantage:

• Given $p$3 samples $p$4 per problem, compute $p$5, $p$6, $p$7.
• Normalized advantage:

$p$8

• Per-token policy ratio (clipped):

$p$9

• CISPO objective:

$\phi$0

• Groups with $\phi$1 are filtered out to ensure updates are driven by a meaningful quality gradient.

This configuration induces robust optimization against adversarial reward hacking and is essential for scaling to high-stakes proof assessment.

3. Population-Level Test-Time Scaling via MaxProof

MaxProof augments the model's test-time reliability and selectivity by orchestrating a structured population search that interleaves generation, multi-criteria verification, iterative refinement, and robust selection.

Algorithmic Overview

The process unfolds through the following steps:

1. Initialization: $\phi$2 candidate proofs $\phi$3 are sampled from Generator $\phi$4.
2. Verification: Each $\phi$5 receives $\phi$6 assessments by Verifier $\phi$7 ($\phi$8 instance); its fitness $\phi$9 is the minimum $(p, y)$0-score, with critiques $(p, y)$1 and approach summaries $(p, y)$2 retained.
3. Archival: All $(p, y)$3 tuples are archived.
4. Iterative Refinement (for $(p, y)$4 rounds):
   • If at least 2 candidates achieve fitness $(p, y)$5, exit early.
   • Select $(p, y)$6 high-fitness, diverse parents (excluding those already $(p, y)$7).
   • For each parent, apply:
     • PATCH: Error-targeted fix via Fixer $(p, y)$8
     • REWRITE: Alternate, less constrained rephrase via $(p, y)$9
   • Fresh candidates are verified and archived with their scores and critiques.
5. Final Selection: Top-4 by fitness undergo round-robin pairwise tournament; Ranker $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$0 adjudicates $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$1 votes per match. The candidate prevailing in most pairwise contests is self-picked as the output champ $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$2.

Key Design Choices

• Conservative fitness: Multiple $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$3 calls per candidate + min aggregation.
• Diversity filter: Bans near-duplicate candidate parents.
• Dual refinement: PATCH/REWRITE facilitates both local error correction and global exploration.
• Early stopping: Halts when ≥2 $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$4 proofs are found.
• Tournament selection: Reduces selection noise from $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$5 tie-breaks.

Role	Prompted Function	Module Used
Generator G	Proof generation	Proof Expert
Verifier V	Scoring/critique	Verifier Expert
Refiner F	PATCH/REWRITE	Fixer Expert
Ranker Q	Pairwise ranking	Verifier Expert

4. Empirical Performance and Benchmarks

Standalone Model Evaluations

On established mathematical proof benchmarks:

• IMOProofBench (7-point scale): M3 achieves a score of 67.40, compared with Opus 4.7 (65.85), Gemini 3.1 (75.71), and GPT-5.5 (90.85).
• IMOAnswerBench (0–7 scale): M3 delivers 81.56 versus Opus (79.90), Gemini (90.00), and GPT-5.5 (90.60).

Contest Results with MaxProof

Ablation against one-shot performance demonstrates substantial gains:

System	IMO 2025 (/42)	USAMO 2026 (/42)
M3 one-shot	27	26
M3 + MaxProof	35 (+8)	36 (+10)

The gold-medal threshold for both contests is 35/42; M3+MaxProof surpasses it in both cases.

Oracle vs. Self-Pick Analysis

• Oracle-best (highest $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$6 score in archive) obtains $$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$7 on 9/12 problems by round 4; plateaus on the hardest instances.
• Self-pick matches oracle except for one USAMO P2 task (4-point difference due to ranker loss).
• This suggests that population search with robust ranking achieves near-oracle selection with minimal loss on competitive tasks.

5. Core Equations and Algorithmic Details

CISPO Group Advantage

$$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$8

$$\{\texttt{no\_errors}, \texttt{minor\_gaps}, \texttt{has\_errors}, \texttt{fundamentally\_wrong}\}$$9

$\phi$0

CISPO Objective

$\phi$1

Population Search Key Steps

• Early stop on ≥2 perfect $\phi$2 proofs.
• PATCH and REWRITE refinements per parent.
• Min aggregation for fitness.
• Final top-4 round-robin, majority-vote for self-champion.

6. Reproducibility and Workflow

The MaxProof methodology can be instantiated by:

1. Training a base LLM under $\phi$3-supervised RL to reach M3 proficiency.
2. Distilling $\phi$4 into an accessible Verifier Expert module.
3. Fine-tuning a Fixer Expert via rejection sampling on critique-labeled data.
4. Deploying MaxProof at inference under the described population search framework, with fixed hyperparameters: $\phi$5, $\phi$6, $\phi$7, $\phi$8, $\phi$9, and early stop threshold of 2.

This systematic framework enables effective scaling of mathematical proof generation, curation, and repair for competition-level benchmarks, and is positioned as a robust test-time scaling approach for complex, multi-step factual reasoning tasks (Chen et al., 11 Jun 2026).