Selection, Recombination, or a Fresh Solve? A Candidate-Free Control for Single-Pass Test-Time Aggregation

Abstract: When every candidate is wrong, correct-candidate selection is unavailable, yet the aggregation call can still solve the problem afresh. A correct aggregate answer may therefore reflect recombination, fresh solving, or both. For efficient test-time reasoning, the relevant question is whether candidate context adds value beyond the additional generation pass. We introduce the missing candidate-free control under the same maximum output-token allowance and stratify by the number of correct candidates. Across AIME-2025 and HMMT-2025 with Qwen3-4B, candidate conditioning improves accuracy when multiple candidates are correct ($Δ<em>{\mathrm{cand}}$(c2+) = +0.290), lowers accuracy when every candidate is wrong ($Δ</em>{\mathrm{cand}}$(c0) = -0.123), and remains unresolved in the one-correct regime. The c2+ and c0 conclusions survive a conservative correction for the adaptive two-benchmark procedure. Under this counterfactual, the interpretation of all-wrong recovery reverses at this scale: conditioning on an all-wrong candidate pool lowers accuracy relative to a fresh solve. Original-format matching and placebo results characterize the failures descriptively but leave their mechanism unresolved. Within a separate structured intervention, explicit answer fields causally steer outputs toward their values; masking yields no measurable accuracy improvement, and equivalence with the original format was not established. The evidence is limited to one Qwen3-4B family, two mathematics benchmarks, first-answer-truncated candidate fragments, and single-pass prompted aggregation.

• The paper introduces a candidate-free baseline under matched output limits, separating genuine candidate-based gains from fresh problem solving in single-pass aggregation.
• Candidate conditioning improves accuracy when at least two candidates are correct (+0.290) but reduces it when all candidates are wrong (−0.123), while the one-correct regime remains unresolved.
• All-wrong aggregation often matches displayed candidate answers, and structured interventions show explicit answer fields can steer outputs, motivating candidate-free controls in future evaluations.

The missing control

Test-time aggregation methods—self-consistency, best-of-N, learned selectors, and single-pass refiners—are routinely credited with "recovering" correct answers on all-wrong candidate pools, where selection is impossible by construction. The paper's central observation is that this credit is unearned without a counterfactual: when every candidate is wrong, an aggregation call can still solve the problem afresh, so a correct aggregate answer may reflect recombination of candidate content, fresh solving, or both. Prior comparisons against candidate-dependent baselines (majority voting, best-of-N, learned selectors) cannot resolve this question because those baselines score zero at $c=0$; generation allowances also frequently differ across compared systems. The paper introduces the missing candidate-free control: a NO-CANDIDATE arm that solves from the problem alone under the same maximum output-token allowance as the aggregator, evaluated within strata defined by the number of correct candidates ($c \in \{0, 1, 2+\}$), following the stratification scheme of GSR (Wang et al., 27 Aug 2025).

The design uses five arms per candidate set ($K=4$): AGGREGATE (problem plus truncated candidate solutions), NO-CANDIDATE, ANSWER-ONLY (answers without reasoning), VOTING, and ORACLE. All generating arms share a 16,384-token output cap and decoding parameters; realized prompt length, output length, latency, and compute differ, so the comparison is neither compute-matched nor response-length-matched—a limitation stated explicitly. The primary statistic is the conditional net effect $$\Delta_{\mathrm{cand}}(c) = \mathbb{E}[Y_{\mathrm{AGGREGATE}} - Y_{\mathrm{NO\text{-}CANDIDATE}} \mid C=c]$$, tested with problem-clustered exact sign tests and Holm correction.

A regime-dependent sign reversal

On AIME-2025 and HMMT-2025 with Qwen3-4B-Instruct-2507 (Yang et al., 14 May 2025), the net effect of candidate conditioning changes sign across regimes. At c2+, conditioning improves accuracy substantially: pooled $\Delta_{\mathrm{cand}} = +0.290$ (raw $p=0.004$). At c0, conditioning on all-wrong candidates lowers accuracy relative to a fresh solve: $\Delta_{\mathrm{cand}} = -0.123$ (raw pooled $p=0.012$). The c1 regime is directional but unresolved (+0.242, $p=0.076$). Because the second benchmark was collected adaptively under a pre-committed extension rule, the paper applies a retrospective conservative two-stage correction; c2+ (adjusted $p=0.0133$) and c0 (adjusted $c \in \{0, 1, 2+\}$0, narrowly) survive at family $c \in \{0, 1, 2+\}$1, while c1 does not. A pre-registered process-level rerun under final-answer scoring strengthens the c0 deficit to $c \in \{0, 1, 2+\}$2 ($c \in \{0, 1, 2+\}$3), confirming robustness to the extraction-scoring rule.

Two further results sharpen the picture. First, a planted-recombination validation—constructed problems whose answers are recoverable only by composing two candidates—shows the harness can detect additive recombination (AGGREGATE 0.767 vs. 0.000 for both controls), while leaving the occurrence rate of recombination on real data unidentified. Second, in the regime where aggregation looks strongest (c2+), plurality voting numerically outperforms it (0.960 vs. 0.887), though not significantly. The implication is direct: for an untrained prompted $c \in \{0, 1, 2+\}$44B aggregator, the apparent all-wrong recovery regime is precisely where single-pass aggregation adds no net value over a fresh solve.

Displayed-answer matching is relevance-insensitive

At c0, wrong aggregate answers match displayed candidate answers far above coincidence: 0.845 versus 0.371 on AIME (paired 33/4, $c \in \{0, 1, 2+\}$5), with the same ordering on HMMT (0.892 vs. 0.506). The paper is careful not to claim causation—the per-problem matching–deficit correlation is null—and shows the loss concentrates on otherwise-solvable problems (pass@24 ≥ 1: $c \in \{0, 1, 2+\}$6 AIME, $c \in \{0, 1, 2+\}$7 HMMT), with near-zero effects on unsolvable ones.

A length-matched placebo control replaces candidates with off-topic solutions from other problems and reproduces much of the deficit (PLACEBO − NO-CANDIDATE = −0.164, CI excluding zero), while the genuine-versus-off-topic increment is indistinguishable from zero (+0.045, CI including zero). More than half of wrong placebo outputs (104/192) matched an off-topic displayed answer. The all-wrong deficit therefore does not require topically relevant candidate content, though whether harm arises from long-context degradation, task reframing, displayed-value steering, or their interaction remains unresolved; the algebraic decomposition into topical-content and generic-context terms requires a placebo-exchangeability assumption the paper does not claim.

Causal evidence from a structured intervention

A structured intervention re-rendered 220 all-wrong sets with byte-identical reasoning prefixes and manipulated only an explicit answer field across FULL, MASKED, and SUBSTITUTED conditions. Two Holm-corrected endpoints yield a dissociation: substituting values causally steers outputs toward them (+0.0682, adjusted $c \in \{0, 1, 2+\}$8), while masking yields no measurable accuracy recovery (−0.0045, adjusted $c \in \{0, 1, 2+\}$9). A bridge check comparing the structured rendering to the original prompt failed to establish equivalence within a pre-fixed ±0.05 margin, so the causal result is confined to the structured intervention and cannot be transferred to the original-format findings, which remain descriptive.

Relation to prior work

The paper positions its question—does reading candidates improve on using the aggregation-stage generation for a fresh solve?—as complementary to GSR (Wang et al., 27 Aug 2025), SSA (Qi et al., 10 Jun 2025), and GenSelect (Toshniwal et al., 23 Jul 2025). GSR's untrained prompted aggregator shows essentially no refinement gap over majority voting, consistent with the null found here; its trained refiner recovers ≤9% on all-wrong pools, interpreted as recombination but never tested against a candidate-free arm. SSA's error analysis reports copy-dominant gains in a trained aggregator, with limited evidence of synthesis. RSA (Venkatraman et al., 30 Sep 2025) demonstrates that recombination-plus-addition genuinely occurs in iterative settings—including with the exact model used here—but lacks a candidate-free arm, so its conditional net effect remains unmeasured. The distinction matters interpretively: a positive raw recovery rate coexists with a negative candidate-free contrast, meaning successful all-wrong answers do not by themselves evidence aggregation benefit.

Limitations and open questions

The evidence covers one Qwen3-4B model family, two 30-problem mathematics benchmarks (possibly contaminated), one deterministic set-construction seed, and first-answer-truncated candidate fragments—the c0 finding is established only for fragments, not full traces. The corrected c0 margin is narrow (adjusted $K=4$0 against $K=4$1). The RLVR variant nearly empties the informative strata, leaving model-class invariance unresolved, and its c2+ estimates are upper bounds due to asymmetric truncation. Prompt format is itself a confound: the canonical prompt displays explicit final-answer lines, making value-salience effects format-dependent. Timing measurements are marginal post-pool quantities, not end-to-end costs. The most substantive open questions are whether training, scale, or iteration produces a positive net recombination benefit, and whether alternative prompt designs mitigate displayed-answer matching; a cross-family replication (e.g., DeepSeek-class models) is identified as a designed next step.

Conclusion

By inserting a candidate-free generation under the aggregation stage's token allowance, the paper converts a single "recovery rate" into a conditional account: candidate context helps when multiple candidates are correct, harms when none are, and leaves the one-correct case open. All-wrong failures take the form of relevance-insensitive displayed-answer matching, and explicit answer fields causally steer outputs within a structured rendering whose equivalence to the original prompt was not established. For efficient-reasoning evaluation, the practical prescription is concrete: evaluations that credit correctness on all-wrong pools as aggregation benefit should include a candidate-free control, and candidate-added accuracy, marginal latency, and end-to-end cost should be reported as distinct quantities.