Best-of-$\infty$ -- Asymptotic Performance of Test-Time Compute (2509.21091v1)

Abstract: We study best-of-$N$ for LLMs where the selection is based on majority voting. In particular, we analyze the limit $N \to \infty$, which we denote as Best-of-$\infty$. While this approach achieves impressive performance in the limit, it requires an infinite test-time budget. To address this, we propose an adaptive generation scheme that selects $N$ based on answer agreement, thereby efficiently allocating inference-time computation. Beyond adaptivity, we extend the framework to weighted ensembles of multiple LLMs, showing that such mixtures can outperform any individual model. The optimal ensemble weighting is formulated and efficiently computed as a mixed-integer linear program. Extensive experiments demonstrate the effectiveness of our approach.

• The paper introduces the best-of-∞ concept, proving that majority voting converges to the true answer as samples approach infinity.
• It presents an adaptive sampling algorithm that achieves best-of-∞ accuracy with 2x–5x fewer samples, enhancing computational efficiency.
• The MILP formulation optimally weights LLM ensembles, enabling performance gains over single models on complex reasoning benchmarks.

Best-of-$\infty$: Asymptotic Performance of Test-Time Compute

Introduction and Motivation

The paper investigates the theoretical and practical limits of test-time compute for LLMs via the best-of-$N$ (BoN) strategy, focusing on majority voting as the selection mechanism. The central question is: what is the asymptotic performance as $N \to \infty$ (best-of-$\infty$), and how can this be efficiently approximated in practice? The work further generalizes BoN to weighted ensembles of multiple LLMs, demonstrating that optimal mixtures can surpass the performance of any single model. The authors provide a principled adaptive sampling algorithm, a mixed-integer linear programming (MILP) formulation for ensemble weight optimization, and extensive empirical validation on challenging reasoning benchmarks.

Theoretical Framework: Best-of-$N$ and Best-of-$\infty$

The BoN approach generates $N$ candidate answers per problem and selects the most frequent (majority) answer. As $N$ increases, the probability of selecting the true majority converges to the population mode, yielding the best-of-$\infty$ performance. This is empirically validated across multiple datasets, where accuracy monotonically increases with $N$ and saturates at the asymptotic limit.

Figure 1: Accuracy of Best-of-N with majority voting as a function of N for GPT-OSS-20B across four datasets, illustrating the convergence to best-of-$\infty$.

Majority voting is robust to reward hacking and does not require additional modeling, unlike reward-based selection methods. However, literal realization of best-of-$\infty$ is infeasible due to unbounded compute requirements.

Adaptive Sampling: Efficient Approximation of Best-of-$\infty$

To address the impracticality of infinite sampling, the authors propose an adaptive sampling algorithm that dynamically determines the number of generations per problem. The algorithm samples until the majority answer is determined with high confidence, quantified via a Bayes factor computed under a Dirichlet process prior over the answer space. This nonparametric Bayesian approach accommodates both finite and infinite answer domains and adapts to the empirical distribution of generated answers.

Figure 2: Illustration of adaptive sampling, showing early stopping when answer agreement is high and continued sampling when disagreement persists.

Theoretical analysis (Theorem 1) establishes that, as the maximum sample size and Bayes factor threshold increase, the adaptive algorithm's accuracy converges to best-of-$\infty$ almost surely. Empirically, adaptive sampling achieves the same accuracy as fixed BoN with substantially fewer samples, yielding 2x–5x compute savings.

Figure 3: Cost-analysis of adaptive sampling versus fixed BoN on MATH500, demonstrating significant compute reduction for equivalent accuracy.

LLM Ensembles: Weighted Majority Voting and MILP Optimization

The framework is extended to ensembles of $K$ LLMs, where each generation is drawn from model $i$ with probability $w_i$. In the best-of-$\infty$ regime, the ensemble answer for each problem is deterministic: the answer with maximal weighted probability across models. The objective function—expected number of correct answers—is non-concave over the simplex of weights, precluding gradient-based optimization.

Figure 4: Visualization of the non-concave objective function $f(w)$ over the weight simplex for three LLMs and five problems; optimal solution is the intersection of polytopes.

The authors show that the optimal weights can be computed via a MILP, leveraging the polytope structure induced by majority voting. The MILP formulation scales to $K \approx 10$ and $N \approx 10^3$ in practice. A max-margin solution is adopted to improve finite-$N$ generalization.

Empirical Results: Scaling, Ensembles, and Generalization

Experiments span 11 open-weight LLMs (≤32B parameters) and four reasoning benchmarks (AIME2024, AIME2025, GPQA-DIAMOND, MATH500), with at least 80 generations per LLM-problem pair. Key findings include:

• Adaptive sampling matches best-of-$\infty$ accuracy with 2x–5x fewer samples than fixed BoN.
• LLM ensembles consistently outperform the best single model, with gains up to 3.3 percentage points in accuracy.
• Optimal weights learned on a small subset of problems generalize well to unseen data, achieving ensemble accuracy close to the best-of-$\infty$ limit.

  Figure 5: Performance comparison of a five-LLM ensemble on GPQA-Diamond; ensemble outperforms all single models and approaches best-of-$\infty$.

  

  Figure 6: Sample efficiency of weight learning on AIME2025; ensemble achieves 93.3% accuracy versus 90.0% for the best single LLM.

Transfer learning experiments show that weights learned on one dataset (AIME2024) transfer effectively to another (AIME2025), with ensemble accuracy matching or exceeding the best individual model in 64.2% of cases.

Comparison with Alternative Selection Methods

Majority voting is compared against reward models, self-certainty, and LLM-as-a-judge approaches. Across datasets and models, majority voting consistently outperforms these alternatives in the Bo5 setting, with reward models and self-certainty trailing by several percentage points.

Implementation Considerations

• Adaptive Sampling: Requires Bayesian posterior computation (Dirichlet process), typically estimated via Monte Carlo. Hyperparameters (concentration $\alpha$, Bayes factor threshold $B$) can be tuned for desired confidence/compute trade-off.
• MILP Optimization: Solved via open-source solvers (e.g., HiGHS), practical for moderate $K$ and $N$. Max-margin regularization improves robustness.
• Resource Requirements: Experiments require large-scale answer generation (≥80 samples per LLM/problem), but adaptive sampling amortizes compute cost.
• Deployment: Ensemble weights can be learned on a small labeled subset and applied to new problems, enabling efficient test-time scaling.

Implications and Future Directions

The results establish best-of-$\infty$ as a principled upper bound for test-time compute in LLM inference, with adaptive sampling providing a practical approximation. Weighted ensembles unlock complementary strengths across models, and MILP-based optimization offers a tractable solution for ensemble design. The findings suggest that, for reasoning tasks, scaling test-time compute via BoN and ensembles can yield greater gains than scaling model parameters alone.

Future work may explore:

• Extension to generative fusion and selection-then-regeneration ensembles.
• Integration with reward models and LLM-as-a-judge for hybrid selection.
• Scaling to larger model pools and more diverse reasoning domains.
• Theoretical analysis of sample complexity and generalization in adaptive sampling.

Conclusion

The paper provides a rigorous analysis of best-of-$N$ and best-of-$\infty$ strategies for LLM inference, introduces an efficient adaptive sampling algorithm, and formulates optimal ensemble weighting as a MILP. Empirical results demonstrate substantial accuracy and compute gains, with ensemble methods robustly outperforming single models. The work advances the understanding of test-time compute scaling and ensemble design for LLMs, with practical algorithms and open-source resources for further research.