OpenDeepThink: Parallel Reasoning via Bradley--Terry Aggregation

Abstract: Test-time compute scaling is a primary axis for improving LLM reasoning. Existing methods primarily scale depth by extending a single reasoning trace. Scaling breadth by sampling multiple candidates in parallel is straightforward, but introduces a selection bottleneck: choosing the best candidate without a ground-truth verifier, since pointwise LLM judging is noisy and biased. To address this, we introduce OpenDeepThink, a population-based test-time compute framework that selects via pairwise Bradley-Terry comparison. Each generation, the LLM judges random pairs of candidates and aggregates votes via Bradley-Terry into a global ranking; top-ranked candidates are preserved and the top three quarters are mutated using the natural-language critiques produced during comparison; the bottom quarter is discarded. OpenDeepThink raises Gemini 3.1 Pro's effective Codeforces Elo by +405 points in eight sequential LLM-call rounds (~27 minutes wall-clock). The pipeline transfers across weaker and stronger models without retuning, and on the multi-domain HLE benchmark, gains appear concentrated in objectively verifiable domains and reverse in subjective ones. We release CF-73, a curated set of 73 expert-rated Codeforces problems with International Grandmaster annotation and 99% local-evaluation agreement against the official verdict.

• The paper introduces a population-based reasoning framework that uses BT aggregation to rank and select LLM-generated solutions without external verifiers.
• It integrates pairwise LLM comparisons with feedback-driven mutation, achieving significant improvements such as a +405 Elo increase in competitive programming tasks.
• Empirical results demonstrate robust performance in objective domains while revealing challenges in subjective areas, highlighting practical trade-offs in selection mechanisms.

OpenDeepThink: Population-Based Parallel Reasoning Through Bradley–Terry Aggregation

Overview

The paper "OpenDeepThink: Parallel Reasoning via Bradley--Terry Aggregation" (2605.15177) presents a population-based test-time compute framework for evolving and selecting LLM-generated solutions to complex reasoning tasks without reliance on ground-truth verifiers or reward models. The central innovation is leveraging pairwise LLM judgments aggregated via Bradley–Terry (BT) ranking to enable robust selection in high-noise settings and coupling this with feedback-driven mutation informed by natural-language critiques. The methodology targets competitive programming and multi-domain reasoning benchmarks, demonstrating substantial improvements in effective Codeforces Elo and delineating boundaries of applicability across objective/subjective domains.

Motivation and Problem Formulation

The dominant paradigm for improving LLM reasoning at inference has focused on sequential deepening: extending chains of thought, employing step-wise value functions, or iterative refinement. These approaches are susceptible to early missteps and lack robustness against noisy self-evaluation, particularly in complex tasks such as programming or open-ended arguments. Parallel sampling (best-of-N) introduces a selection bottleneck; without a ground-truth verifier, LLM judges engaged in pointwise assessment suffer from significant positive bias and poor recall on incorrect outputs, as established across several prior works.

OpenDeepThink addresses this bottleneck by operating over populations of candidate solutions, orchestrating parallel pairwise comparisons judged by the same LLM used for generation, and aggregating these comparisons through the Bradley–Terry model. This enables both robust head-to-head ranking and targeted mutation of non-elites, with the mutations informed by discriminative, natural-language critiques arising from pairwise competition.

Methodology

The pipeline operates as follows:

1. Candidate Sampling: An initial population of $n$ solutions is generated via parallel LLM calls.
2. Evolution Loop: Over $T$ generations, each candidate undergoes $K$ randomized pairwise comparisons. The judge produces both winner verdicts and brief natural-language rationales for each side.
3. Bradley–Terry Aggregation: Comparison outcomes are aggregated into global rankings using BT logistic models, which internalize opponent strength and mitigate sampling noise.
4. Selection and Mutation: The top 25% are preserved as elites. The top 75% (including elites) are mutated using aggregated negative feedback. The bottom 25% are discarded.
5. Final Selection: A dense round of pairwise comparisons (M per candidate) yields a final BT ranking, from which the submission is chosen.

The same LLM instance serves as both generator and judge. Critiques produced during pairwise comparisons, especially negative feedback, are recycled as prompts for mutation, facilitating both partial rescue and abandonment of flawed solution approaches.

Empirical Results

The framework is evaluated across 192 programming problems (CF-73 and NOI-119) and 82 multi-domain HLE questions:

• Competitive Programming: On Codeforces benchmarks using Gemini 3.1. Pro, OpenDeepThink raises effective Elo by +405 points. For the hardest tier (pass@1 ≤ 35%), pass@1 increases from 11% (random sampling) to 36% post-mutation, and the final BT top-1 accuracy reaches 50%, a net gain of 39 points over the baseline. Aggregation alone saturates performance on easy problems; evolution is essential on difficult instances.
• Cross-Model Transfer: Identical hyperparameters transfer to Gemini 3. Flash and Gemini 2.5. Pro, evidencing robust gains without model-specific tuning. The balance between gains from evolution and selection shifts along the capability spectrum.
• Multi-Domain HLE: Gains are concentrated in objectively verifiable domains, e.g., mathematics and physics (+5 to +17 points). In subjective domains (humanities/social sciences), performance declines by 25-30 points, reflecting the unreliability of pairwise LLM judgment as a proxy for correctness.
• Pairwise vs. Pointwise: Pairwise judgment achieves 86% selection accuracy versus 59% for pointwise. The gap persists after controlling for generation quality, underscoring the superiority of discriminative head-to-head comparisons as selection signal.

Feedback and Mutation Dynamics

Ablation studies reveal that negative feedback from pairwise critiques accounts for almost all mutation efficacy; positive feedback offers negligible incremental signal. At $K=4$, the rescue rate (WA→AC) nearly doubles compared with a no-feedback baseline. Increasing $K$ further regresses performance, indicating saturation in the mutator’s ability to process contrastive feedback. Gains from mutation are front-loaded, saturating by the second generation; dense comparison in the final round extracts additional top-1 accuracy.

Practical and Theoretical Implications

OpenDeepThink sets forth a paradigm for leveraging population-based, parallel reasoning at test time, distinguished by its training-free, verifier-free selection mechanism. The methodology scales compute breadth, not depth, and efficiently converts parallel API calls into accuracy gains without extending response latency. The use of BT aggregation as a soft verifier is critical—its effectiveness directly tracks the reliability of LLM pairwise judgments. Where LLMs can reliably discriminate correctness, the framework amplifies signal; where judgment is unreliable (e.g., subjective domains), iterative selection can amplify noise and degrade performance.

The methodology is robust within the Gemini family and theoretically extends to other LLM architectures, provided that pairwise judge accuracy is sufficient. However, there are clear limitations: prohibitive API call costs for latency-sensitive tasks, transferability uncertainties to other LLMs, and reliance on empirical prompt and selection ratios rather than controlled ablation.

Future Directions

Potential avenues for further research include:

• Systematic ablation of elite ratios, mutation license, and population sizes to optimize pipeline hyperparameters.
• Evaluation on architecturally distinct LLM families to generalize findings beyond Gemini.
• Exploration of hybrid approaches combining programmatic verifiers with BT aggregation in domains where partial ground-truth signals are available.
• Integration of domain-adaptive judge prompts or models to mitigate bias and improve reliability in subjective settings.
• Parallelization strategies for further reducing wall-clock latency and scaling up candidate populations.

Conclusion

OpenDeepThink systematically addresses the selection bottleneck in parallel LLM reasoning by unifying population-based candidate generation, pairwise comparison, and Bradley–Terry aggregation. The framework enables robust mutation and selection without external verifiers, yielding significant performance gains in competitive programming and other objectively scored domains. Its efficacy is contingent upon the reliability of pairwise LLM judgment; evolution and selection amplify signal or noise accordingly. The results underline the importance of discriminative, population-level reasoning mechanisms at inference and open new directions for scalable, training-free parallel search in complex problem domains.