Mean-Sensitivity Frontier

• Mean-Sensitivity Frontier is a framework that assesses configurations by jointly measuring mean performance (e.g., balanced accuracy) and sensitivity (e.g., self-consistency and prompt sensitivity).
• The approach uses a Pareto-based trade-off in a two-dimensional space to distinguish models that achieve high verification accuracy with minimal instability.
• Empirical studies show that prompt ensembling can shift smaller models toward frontier performance and that the concept applies across disciplines like finance and robust statistics.

Searching arXiv for the cited paper and closely related work to ground the article in current literature. arXiv search: (Naik et al., 2 Apr 2026) "Do We Need Frontier Models to Verify Mathematical Proofs?" Mean-Sensitivity Frontier denotes a class of frontier constructions in which a system is evaluated jointly by a mean-performance coordinate and a sensitivity coordinate, and admissible configurations are ordered by domination in that two-dimensional space. In the most explicit recent formulation, for LLM-based mathematical proof verification, the frontier is defined in the plane $(\bar{A}, S)$, where $\bar{A}$ is mean balanced accuracy and $S$ is a sensitivity measure such as inconsistency $1-C_{\text{unanimous}}$ or prompt sensitivity $S_{\text{prompt}}$; the frontier is the set of undominated model–prompt–ensemble configurations (Naik et al., 2 Apr 2026). The same label also appears in topological dynamics, portfolio theory, stochastic frontier analysis, and robust statistics, but with domain-specific meanings of “mean” and “sensitivity,” so the term is best understood as a family of related trade-off formalisms rather than a single invariant construction (Hauser et al., 16 Jan 2026, Sayit, 2022, Acerenza et al., 28 Apr 2026, Iverson et al., 21 May 2026).

1. Frontier concept in LLM proof verification

In mathematical proof verification, the Mean-Sensitivity Frontier formalizes the trade-off between expected verification quality and instability of judgments. The underlying setting is natural-language verification of Olympiad-level proofs against human labels. Mean performance refers to expected verification quality under a given evaluation protocol, measured primarily by balanced accuracy and averaged across datasets, prompts, and repeated runs. Sensitivity refers to instability from two sources: self-inconsistency across repeated judgments on the same proof under the same prompt, and prompt sensitivity, meaning variability in accuracy across judging prompts with different rubric styles or strategies (Naik et al., 2 Apr 2026).

This formulation is motivated by the observation that verification can be easier than generation because a verifier evaluates an existing proof rather than producing a new one. The same work also emphasizes that reliable verification still requires substantial capability: a verifier must detect subtle reasoning gaps, wrong directions of implications, missing cases, unjustified theorem applications, and failures in global proof structure. In that sense, the frontier is not merely a benchmarking device; it is a way to separate raw mathematical competence from instability induced by elicitation and inference-time variation (Naik et al., 2 Apr 2026).

The operational interpretation is Pareto-theoretic. A configuration lies on the frontier when no evaluated alternative achieves both higher mean performance and lower sensitivity. In the proof-verification setting, this makes it possible to distinguish models that are close in average balanced accuracy but differ sharply in reproducibility and robustness to prompt choice. The paper argues that this distinction is central for trust in proof-checking systems because a verifier that is accurate on average but unstable across repeated calls or rubric variants is difficult to deploy as a dependable grading or auditing component (Naik et al., 2 Apr 2026).

2. Metric structure and dominance relations

The proof-verification formulation uses a small set of explicit metrics.

Quantity	Definition	Role
$A$	Simple accuracy over binary labels	Raw verifier accuracy
$\mathrm{BA}$	$\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$	Primary accuracy metric
$C$	Agreement across repeated judgments	Self-consistency
$S$	$\bar{A}$0 or $\bar{A}$1	Inconsistency
$\bar{A}$2	Std. dev. of accuracy across prompts	Prompt sensitivity
$\bar{A}$3	$\bar{A}$4	Accuracy spread across prompts

For $\bar{A}$5 proofs with binary ground truth $\bar{A}$6 and model judgments $\bar{A}$7, simple accuracy is

$\bar{A}$8

Balanced accuracy is

$\bar{A}$9

with

$S$0

The related error rates are

$S$1

These definitions matter because the paper’s later error-profile analysis is expressed in terms of differences in FNR and FPR rather than balanced accuracy alone (Naik et al., 2 Apr 2026).

Self-consistency is defined over $S$2 repeated judgments per proof. One formulation uses pairwise agreement,

$S$3

A second formulation uses vote concentration for binary outputs,

$S$4

The paper’s operational measure is unanimity consistency with $S$5,

$S$6

and the associated inconsistency is

$S$7

Prompt sensitivity over $S$8 prompts is defined from accuracies $S$9 by

$1-C_{\text{unanimous}}$0

with an auxiliary spread statistic

$1-C_{\text{unanimous}}$1

The Mean-Sensitivity Frontier is then defined in the plane $1-C_{\text{unanimous}}$2. A point $1-C_{\text{unanimous}}$3 dominates a point $1-C_{\text{unanimous}}$4 if $1-C_{\text{unanimous}}$5 and $1-C_{\text{unanimous}}$6, with at least one strict inequality. The frontier is the set of undominated points. In the same work, separate frontiers are proposed for $1-C_{\text{unanimous}}$7 and $1-C_{\text{unanimous}}$8, making self-inconsistency and prompt sensitivity distinct axes of robustness (Naik et al., 2 Apr 2026).

3. Empirical frontier for competition-level proof verification

The empirical study evaluates LLMs as natural-language proof verifiers on three datasets: IMO-GradingBench, ProofArena, and ProofBench. Each dataset contains Olympiad-level problems and multiple LLM-generated natural-language proofs with human grades. The paper binarizes correctness as grade $1-C_{\text{unanimous}}$9 for IMO-GradingBench and ProofBench, while ProofArena is already binary; an alternative threshold treating $S_{\text{prompt}}$0 or $S_{\text{prompt}}$1 as correct is analyzed in the appendix and yields similar conclusions. The evaluated models comprise two frontier models, OpenAI GPT-5.2 and Google Gemini 3.1 Pro, each in low- and high-reasoning settings, together with four open-source models, OSS (20B, 120B) and Qwen (35B, 122B) (Naik et al., 2 Apr 2026).

The central empirical result is that smaller open-source models are much closer to frontier models in mean balanced accuracy than in sensitivity. Across all datasets and prompts, Qwen3.5-35B achieves $S_{\text{prompt}}$2 balanced accuracy, while Gemini 3.1 Pro in low reasoning achieves $S_{\text{prompt}}$3 balanced accuracy, so the mean-performance gap is only about $S_{\text{prompt}}$4. By contrast, open-source models can be up to about $S_{\text{prompt}}$5 more inconsistent. Even the least consistent frontier model, Gemini 3.1 Pro in low reasoning, leads the most consistent open-source model, Qwen 122B, by about $S_{\text{prompt}}$6 in self-consistency. The paper therefore presents sensitivity, rather than average proof-verification competence, as the main differentiator between frontier and smaller models (Naik et al., 2 Apr 2026).

Prompt sensitivity is reported as strong for all models. Accuracy varies significantly across prompts such as IMOBench, GIMO, and ProofBench7pt. The largest reported drop is for the GIMO prompt, which decreases balanced accuracy by $S_{\text{prompt}}$7 for GPT-5.2 high reasoning and biases models toward rejecting correct proofs by increasing FNR. This result is methodologically important because it shows that even strong verifiers can move substantially in the sensitivity coordinate without any change in base model weights (Naik et al., 2 Apr 2026).

The error-profile analysis is asymmetric. The paper reports a mean FNR difference of $S_{\text{prompt}}$8 percentage points between smaller and frontier models, while the mean FPR difference is only $S_{\text{prompt}}$9 percentage points. It also reports that smaller models tend to accept flawed proofs more often. Taken together, these results indicate that frontier placement is not determined only by generic reasoning level; it is also shaped by the ease or difficulty of eliciting a reliable verification strategy from the model (Naik et al., 2 Apr 2026).

A concrete frontier movement is given for Qwen3.5-35B on ProofBench. Under a base prompt, the model is reported at approximately

$A$0

After ensembling, it moves to

$A$1

The paper characterizes this as an outward move: higher mean performance and lower sensitivity. Before prompt optimization, many open-source points lie below and to the right of frontier points; after prompt ensembling, smaller models such as Qwen3.5-35B move onto or near the frontier previously occupied by frontier models (Naik et al., 2 Apr 2026).

4. Frontier shifts via LLM-guided prompt search and ensemble design

The frontier shift is produced by an explicit inference-time methodology. The first stage is failure-mode diagnosis on a balanced subset of IMO-GradingBench called TrainProofs, consisting of 200 examples, 140 incorrect and 60 correct. Errors in incorrect proofs were annotated by GPT-5 Mini using the rubric and reference solution, and manual spot-checking of 15 samples validated annotation quality. This stage is intended to expose recurrent failure modes such as reproducing flawed logic and fabricating missing justifications (Naik et al., 2 Apr 2026).

The second stage is targeted prompt synthesis for error detection. Claude Code iteratively refined a set of general, non-instance-specific prompts to mitigate common failure modes. Each candidate prompt was validated by checking OSS-120B’s detected error against the annotated error under high reasoning. The resulting 12 prompts collectively detected 137 of 140 errors with OSS-20B and 90 of 140 with Qwen-35B, and matched annotated errors at $A$2 and $A$3 respectively. A subsequent verification-focused ensemble search optimized balanced accuracy on TrainProofs by refining the prompt set and the aggregation rule. After 30 iterations with OSS-120B, multiple ensemble-threshold combinations matched GPT-5.2 high reasoning on TrainProofs, and the best combination was selected using a 400-example holdout (Naik et al., 2 Apr 2026).

The deployed ensemble comprises 12 calls using 8 distinct prompts. A general grading run is executed 5 times independently to provide self-consistency signal, and 7 specialized prompts cover step-by-step checks, entailment analysis, theorem-usage analysis, topic-specific grading variants, and adversarial or skeptical checks. Aggregation uses threshold voting:

$A$4

with $A$5. The rationale given in the paper is that specialized prompts target complementary failure modes, while independent prompt calls reduce correlation among misjudgments. Threshold voting then filters idiosyncratic errors, increasing unanimity consistency and reducing prompt sensitivity. The same source reports that ensemble diversity outperforms either a single merged prompt or repeated identical prompts, which suffer correlated variance and unstable votes near decision boundaries (Naik et al., 2 Apr 2026).

The reported gains are substantial. Across models and datasets, the ensemble yields up to $A$6 balanced accuracy and $A$7 self-consistency. On ProofBench, Qwen3.5-35B improves from $A$8 balanced accuracy and $A$9 self-consistency to $\mathrm{BA}$0 balanced accuracy and $\mathrm{BA}$1 self-consistency, performing on par with frontier models such as Gemini 3.1 Pro and exceeding them in balanced accuracy on that dataset. On IMO-GradingBench, ensembled OSS-120B reaches $\mathrm{BA}$2, matching Gemini-3.1-Low, and $\mathrm{BA}$3, the lowest among compared models. These results are the paper’s main evidence that smaller models possess the underlying mathematical capability for frontier-grade proof verification but require more structured elicitation to realize it consistently (Naik et al., 2 Apr 2026).

5. Statistical reading, deployment significance, and limitations

The frontier construction is explicitly statistical rather than purely geometric. The paper reports means over three independent runs per configuration, with error bars given by runwise min–max. For frontier construction, it recommends confidence intervals for $\mathrm{BA}$4 via bootstrap over proofs and over prompts, binomial-proportion intervals or bootstrap for $\mathrm{BA}$5, and paired bootstrap tests to assess whether increases in $\mathrm{BA}$6 and reductions in $\mathrm{BA}$7 are statistically significant. This is significant because a point may appear non-dominated numerically while remaining indistinguishable from nearby points under sampling variability (Naik et al., 2 Apr 2026).

As deployment guidance, the proof-verification study distinguishes model choice from sensitivity management. If consistency and robustness are critical, frontier models have higher baseline self-consistency and lower prompt sensitivity. For cost-effective deployments, however, the paper recommends smaller open-source models such as OSS-120B or Qwen3.5-35B paired with the specialized ensemble and threshold voting near $\mathrm{BA}$8. It also recommends monitoring FPR and FNR rather than balanced accuracy alone, fixing random seeds and temperature, using repeated runs, and tracking both $\mathrm{BA}$9 and $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$0 over time. Evaluation across multiple rubric styles is presented as necessary for quantifying prompt sensitivity rather than treating a single prompt as representative (Naik et al., 2 Apr 2026).

The paper also describes several limitations. Binary correctness derived from 7-point grading compresses nuance, and different rubrics can create borderline cases that affect both balanced accuracy and consistency. Natural-language proofs are ambiguous, human labels are costly and subjective, and autoformalization remains challenging for free-form proofs. The ensemble method is demonstrated only for mathematics; transfer to law or finance would require domain-specific prompt design and failure-mode audits. Open research directions include reducing sensitivity without sacrificing mean performance through consistency-aware decoding, improved post-training, or reward shaping that penalizes flip-flops across runs and prompts; hybrid neuro-symbolic pipelines combining autoformalization with formal proof engines; learned meta-verifiers or adaptive thresholds; and fine-grained process-level labels that align verifier outputs with rubric components (Naik et al., 2 Apr 2026).

A recurrent misconception is that verification is intrinsically cheap because it is easier than generation. The paper’s results argue for a narrower claim. Verification may be easier in principle, but reliable verification remains a frontier capability because reproducibility and rubric robustness do not automatically emerge from raw problem-solving strength. In the frontier language, mean performance and sensitivity are partially separable, and inference-time intervention can move a system outward along both coordinates (Naik et al., 2 Apr 2026).

6. Cross-domain uses of the term

The expression also appears in several other research areas, although the axes and the object being fronted are domain-specific.

Domain	“Mean”	“Sensitivity”
Proof verification	Mean balanced accuracy	Self-inconsistency or prompt sensitivity
Topological dynamics	Mean equicontinuity or averaged orbit behavior	Mean sensitivity, regional mean sensitivity, strong mean sensitivity
Portfolio theory	Expected return	Risk or sensitivity under a chosen risk measure
Frontier econometrics	Mean inefficiency or mean deviation	Sensitivity to assumption relaxations
Robust estimation	Clean-data estimation error scale	Empirical sensitivity under $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$1 data modifications

In topological dynamics, the phrase marks the boundary between mean equicontinuity and mean sensitivity. For actions of $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$2-compact, locally compact amenable groups, regional mean sensitive pairs provide the exact criterion: a system is mean equicontinuous iff $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$3, and the maximal mean equicontinuous factor is obtained as $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$4 (Hauser et al., 16 Jan 2026). Earlier work established dichotomies between almost mean equicontinuity and mean sensitivity for transitive systems, and proved that every ergodic invariant measure of a mean equicontinuous system has discrete spectrum (Li et al., 2013). In cellular automata, however, the mean regime is more delicate: the “Pacman level 2” automaton is neither almost mean equicontinuous nor mean sensitive, showing that the corresponding dichotomy can fail when the cellular automaton commutes with a transitive shift but is not itself transitive (Baños et al., 2020). Measure-theoretic work further relates the frontier to sequence entropy, independence, and mean-sensitive tuples for ergodic amenable group actions (Liu et al., 14 Jan 2025, García-Ramos et al., 2022, Liu et al., 26 Jan 2025, Yang et al., 2024).

In finance, the expression is used for frontiers that trade expected return against a sensitivity or risk functional. Under normal mean-variance mixture returns and finite, law-invariant convex risk measures, the mean-risk frontier is obtained by solving a Markowitz problem with adjusted mean $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$5 and base covariance $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$6 rather than the true covariance of returns (Sayit, 2022). A different portfolio-theoretic use studies how efficient frontiers are rescaled under distribution misspecification and shows that shrinkage methods tend to rescale the sample efficient frontier, making comparisons at fixed risk aversion problematic (Paskaramoorthy et al., 2021).

In frontier econometrics, the term is tied to robustness of mean-based inefficiency conclusions under relaxed assumptions. One paper derives a breakdown frontier in the relaxation space $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$7 for conclusions about mean inefficiency, where $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$8 and $\frac{1}{2}(\mathrm{TPR}+\mathrm{TNR})$9 bound deviations of the noise and inefficiency densities from the parametric baseline; the frontier traces the maximal assumption relaxations under which a mean-based claim still holds (Acerenza et al., 28 Apr 2026). A related paper identifies the frontier structural function as the essential supremum of outcomes under assignment at the boundary and derives nonparametric lower bounds on mean deviation from variance and skewness alone, presenting this as a frontier of admissible mean deviations under weak assumptions (Ben-Moshe et al., 28 Apr 2025).

In robust statistics, the phrase is instantiated through empirical sensitivity. For Gaussian mean estimation, any estimator with optimal clean-data $C$0 error $C$1 must have empirical sensitivity at least

$C$2

and this lower bound is tight up to logarithmic factors. Here the frontier quantifies the best possible stability of accurate estimators under modification of at most $C$3 data points (Iverson et al., 21 May 2026).

These usages do not share a single mathematical template, but they do share a common organizing idea: a frontier is drawn between an averaged or mean-level objective and a formally specified notion of sensitivity or instability. This suggests that “Mean-Sensitivity Frontier” functions less as a fixed technical term than as a recurring research pattern for analyzing performance–robustness trade-offs across disciplines.