Orchestrated Test-Time Scaling

Updated 14 July 2026

Orchestrated Test-Time Scaling is a structured, policy-driven method that allocates inference compute across steps, branches, and agents to optimize performance.
It integrates techniques such as bandit allocation, iterative refinement, and multi-agent coordination to focus compute on high-yield solution paths while reducing waste.
OTTS spans diverse applications—from large language models to vision and robotics—balancing accuracy, latency, and resource budgets in dynamic real-world scenarios.

Searching arXiv for papers on orchestrated test-time scaling and closely related formulations to ground the article. arxiv_search query="orchestrated test-time scaling OR strategic scaling test-time compute bandit OR collective test-time scaling OR Agentic Test-time Learning-to-Allocate Scaling" max_results=10 {"result":[{"arxiv_id":"(Qin et al., 1 Jun 2026)","title":"ATLAS: Agentic Test-time Learning-to-Allocate Scaling","authors":"Alexander Wei, Hyung Won Chung, Xian Li, Romal Thoppilan, Yi Tay, Jiahui Yu, Jonas Geiping, Quoc V. Le, Denny Zhou","abstract":"Test-time scaling has become a major way to improve LLM reasoning, but its orchestration has remained designer-engineered: a fixed sample budget, a fixed refinement loop, a fixed scoring rule, or a fixed search policy decides how compute is spent, leaving the model in charge of solving but not of orchestration. We introduce ATLAS, an agentic test-time scaling framework in which an LLM orchestrator owns the control loop end-to-end. Through a single action, explore, which dispatches a fresh independent solver on the original problem, the orchestrator decides whether to gather more evidence, when to stop, and how to synthesize the final answer; the action space is extensible, with each explore call optionally specifying solver, reasoning effort, or prompting strategy. We evaluate ATLAS on four benchmarks covering scientific question answering, code generation, and multimodal reasoning under a Claude Sonnet 4.6 backbone, where it reaches 56.00% on HLE-Verified, 82.29% on LiveCodeBench, 85.75% on GPQA-Diamond, and 23.71% on BabyVision while using far fewer API calls than fixed-workflow baselines. A multi-model extension, ATLAS-MM, that exposes solver choice as an additional action dimension further improves HLE-Verified to 60.00% and LiveCodeBench to 85.63%, with consistent gains on GPQA-Diamond and BabyVision. Ablations replacing the orchestrator’s direct synthesis with a separate integrator degrade or fail to improve accuracy on three of four benchmarks, consistent with the role of stateful evidence management in producing the gains.","categories":"cs.AI cs.CL","published":"2026-06-01"},{"arxiv_id":"(Zuo et al., 15 Jun 2025)","title":"Strategic Scaling of Test-Time Compute: A Bandit Learning Approach","authors":"Hanjun Dai, Ke Yang, Yuxiao Chen, Chenyang Tao, Ying Sheng, Danqi Chen, Xinyun Chen","abstract":"Scaling test-time compute has emerged as an effective strategy for improving the performance of LLMs. However, existing methods typically allocate compute uniformly across all queries, overlooking variation in query difficulty. To address this inefficiency, we formulate test-time compute allocation as a novel bandit learning problem and propose adaptive algorithms that estimate query difficulty on the fly and allocate compute accordingly. Compared to uniform allocation, our algorithms allocate more compute to challenging queries while maintaining accuracy on easier ones. Among challenging queries, our algorithms further learn to prioritize solvable instances, effectively reducing excessive computing on unsolvable queries. We theoretically prove that our algorithms achieve better compute efficiency than uniform allocation and empirically validate their effectiveness on math and code benchmarks. Specifically, our algorithms achieve up to an 11.10% performance improvement (15.04% relative) on the MATH-500 dataset and up to a 7.41% performance improvement (14.40% relative) on LiveCodeBench.","categories":"cs.LG cs.AI cs.CL stat.ML","published":"2025-06-15"},{"arxiv_id":"(Song et al., 5 Aug 2025)","title":"CTTS: Collective Test-Time Scaling","authors":"Xun Wang, Pei Guo, Sixing Wu, Yifan Wu, Yanan Zhang, Zongxia Li, Jia Zhang, Longyue Wang, Xixuan Song, Guanbo Wang, Guoyin Wang, Yiran Huang, Min Zhang, Chao Huang, Jie Zhou","abstract":"Test-time scaling (TTS) has emerged as a promising research field for enhancing the effectiveness of LLMs without extra training. However, most existing approaches, e.g., Best-of-N and Self-Consistency rely on a single agent interacting with a reward model (SA-SR), constrained by limited capabilities of a single test-time scaling (STTS) paradigm. On the other hand, recent works demonstrate that collective-agent methods can break through the upper bound of single-agent systems by orchestrating diverse models. Thus, in this paper, we take a first step towards exploring Collective Test-Time Scaling (CTTS). Consider the different interaction types of single and multiple models, we design three primary paradigms to investigate the optimal paradigm of CTTS: (1) single agent to multiple reward models (SA-MR); (2) multiple agents to single reward model (MA-SR); and (3) multiple agents to multiple reward models (MA-MR). Extensive experiments demonstrate that MA-MR consistently achieves the best performance. Based on this, we propose a novel framework named CTTS-MM that effectively leverages both multi-agent and multi-reward-model collaboration for enhanced inference. Specifically, for multi-agent collaboration, we propose an Agent Collaboration Search (ACS), which searches for the most effective combination of LLM agents from a large candidate pool; for multi-reward-model collaboration, we propose Mixture of Reword Models (MoR), which consists of a curated question pool and a Prior Reward model Ensemble Selection (PRES) to select the optimal combinations of reward models via Pair-wise Reward Ranking (PRR) metric. Experiments across seven mainstream benchmarks demonstrate that the proposed CTTS-MM consistently obtains superior performance.","categories":"cs.CL cs.AI","published":"2025-08-05"},{"arxiv_id":"(Yao et al., 13 Jun 2025)","title":"Simple Radiology VLLM Test-time Scaling with Thought Graph Traversal","authors":"Jingnan Guan, Xingyu Cheng, Jiang Bian, Wenjie Feng, Yixuan Yuan, Pengfei Liu, Yujia Bao","abstract":"Test-time scaling offers a promising way to improve the reasoning performance of vision-language large models (VLLMs) without additional training. In this paper, we explore a simple but effective approach for applying test-time scaling to radiology report generation. Specifically, we introduce a lightweight Thought Graph Traversal (TGT) framework that guides the model to reason through organ-specific findings in a medically coherent order. This framework integrates structured medical priors into the prompt, enabling deeper and more logical analysis with no changes to the underlying model. To further enhance reasoning depth, we apply a reasoning budget forcing strategy that adjusts the model's inference depth at test time by dynamically extending its generation process. This simple yet powerful combination allows a frozen radiology VLLM to self-correct and generate more accurate, consistent chest X-ray reports. Our method outperforms baseline prompting approaches on standard benchmarks, and also reveals dataset biases through traceable reasoning paths.","categories":"cs.CV cs.AI cs.LG","published":"2025-06-13"},{"arxiv_id":"(Kaya et al., 3 Oct 2025)","title":"Efficient Test-Time Scaling for Small Vision-LLMs","authors":"Xiangchi Zhang, Yuantao Gu, Jialin Wang, Sahar Abdelnabi, Mario Fritz, Francesco Croce, Zhun Zhong","abstract":"Small Vision-LLMs (VLMs) provide a computationally efficient alternative to larger models, at the cost of weaker generalization abilities and downstream task performance. These shortcomings could be addressed by test-time scaling techniques, but existing methods are typically computationally demanding, contradicting the resource-efficient design goals of small models. To address these limitations, we propose two novel and efficient test-time scaling strategies that leverage the model-internal features rather than external supervision: (i) Test-Time Augmentation (TTAug), which generates multiple augmented inputs and aggregates outputs at the token level without parameter updates, and (ii) Test-Time Adaptation (TTAdapt), which adapts model parameters during inference using consensus-based pseudolabels from TTAug. Through extensive experiments across nine benchmarks, we demonstrate consistent performance improvements while maintaining computational efficiency suitable for resource-constrained environments. The generality of our approach is demonstrated both within models at different scales and across different VLMs without additional tuning.","categories":"cs.CV cs.AI","published":"2025-10-03"},{"arxiv_id":"(Hao et al., 27 Sep 2025)","title":"Scaling LLM Test-Time Compute with Mobile NPU on Smartphones","authors":"Junyu Zhou, Yangjie Zhou, Jiaming Wei, Rui Han, Yawen Li, Xiaotao Li, Jiannan Wang, Han Guo, Huazhong Yang, Shanyu Tang, Yu Wang","abstract":"Deploying LLMs on mobile devices faces the challenge of insufficient performance in smaller models and excessive resource consumption in larger ones. This paper highlights that mobile Neural Processing Units (NPUs) have underutilized computational resources, particularly their matrix multiplication units, during typical LLM inference. To leverage this wasted compute capacity, we propose applying parallel test-time scaling techniques on mobile NPUs to enhance the performance of smaller LLMs. However, this approach confronts inherent NPU challenges, including inadequate hardware support for fine-grained quantization and low efficiency in general-purpose computations. To overcome these, we introduce two key techniques: a hardware-aware tile quantization scheme that aligns group quantization with NPU memory access patterns, and efficient LUT-based replacements for complex operations such as Softmax and dequantization. We design and implement an end-to-end inference system that leverages the NPU's compute capability to support test-time scaling on Qualcomm Snapdragon platforms. Experiments show our approach brings significant speedups: up to 19.0 for mixed-precision GEMM and 2.2 for Softmax. More importantly, we demonstrate that smaller models using test-time scaling can match or exceed the accuracy of larger models, achieving a new performance-cost Pareto frontier.","categories":"cs.AR cs.CL","published":"2025-09-27"},{"arxiv_id":"(Wang et al., 26 May 2025)","title":"Scaling over Scaling: Exploring Test-Time Scaling Plateau in Large Reasoning Models","authors":"Siyu Chen, Yunchen Zeng, Jiawei Xie, Jiaao Chen, Cong Xie, Minlin Chen","abstract":"Large reasoning models (LRMs) have exhibited the capacity of enhancing reasoning performance via internal test-time scaling. Building upon this, a promising direction is to further scale test-time compute to unlock even greater reasoning capabilities. However, as we push these scaling boundaries, systematically understanding the practical limits and achieving optimal resource allocation becomes a critical challenge. In this paper, we investigate the scaling plateau of test-time scaling and introduce the Test-Time Scaling Performance Model (TTSPM). We theoretically analyze two fundamental paradigms for such extended scaling, parallel scaling and sequential scaling, from a probabilistic modeling perspective. Our primary contribution is the derivation of the saturation point on the scaling budget for both strategies, identifying thresholds beyond which additional computation yields diminishing returns. Remarkably, despite their distinct mechanisms, both paradigms converge to a unified mathematical structure in their upper bounds. We empirically validate our theoretical findings on challenging reasoning benchmarks, including AIME, MATH-500, and GPQA, demonstrating the practical utility of these bounds for test-time resource allocation. We hope that this work provides insights into the cost-benefit trade-offs of test-time scaling, guiding the development of more resource-efficient inference strategies for large reasoning models.","categories":"cs.AI","published":"2025-05-26"},{"arxiv_id":"(Hoehn et al., 13 Jun 2025)","title":"Test-Time Scaling with Reflective Generative Model Selection","authors":"Yulei Qin, Lei Li, Wenjie Wang, Hongjia Li, Xinyan Yu, Xianheng Ma, Wenyuan Wang, Xinrun Wang, Hao Fei, Kam-Fai Wong","abstract":"Test-Time Scaling (TTS) methods, such as Best-of-N, leverage additional inference-time computation to improve LLMs' reasoning and generation. Despite their effectiveness, these methods are increasingly constrained by the limitations of relying on a single model for generation. In this work, we propose Reflective Generative Model Selection (RGMSelect), a multi-model TTS framework that decomposes the generation process into model selection and response synthesis. Instead of naively merging outputs from all available models, RGMSelect first introduces a two-stage preference optimization process to train a lightweight generator model selector from diverse generation traces, enabling it to dynamically identify the most suitable generator model for each instruction. It then leverages a powerful target model to synthesize a final answer conditioned on the selected model's output. Extensive experiments across 12 LLMs and three benchmarks show that RGMSelect consistently outperforms state-of-the-art TTS baselines, achieving gains of up to 14.5% over baseline and 6.5% over comparable multi-model methods.","categories":"cs.CL cs.AI","published":"2025-06-15"},{"arxiv_id":"(Yin et al., 7 Oct 2025)","title":"ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models","authors":"Jie Hu, Siqi Zhu, Xiaowei Zhang, Jiangjie Chen, Haoyang Li, Yijie Zhang, Yichao Jin, Huazhong Yang","abstract":"Test-time scaling has emerged as a transformative paradigm for enhancing the performance of large reasoning models, enabling dynamic allocation of computational resources during inference. However, as the landscape of reasoning models rapidly expands, a critical question remains: how can we systematically compare and evaluate the test-time scaling capabilities across different models? In this paper, we introduce ARISE (Adaptive Resolution-aware Scaling Evaluation), a novel metric specifically designed to assess the test-time scaling effectiveness of large reasoning models. Unlike existing evaluation approaches, ARISE incorporates two key innovations: (1) sample-level awareness that effectively penalizes negative scaling behaviors where increased computation leads to performance degradation, and (2) a dynamic sampling mechanism that mitigates the impact of accuracy fluctuations and token count instability on the final assessment. We conduct comprehensive experiments evaluating state-of-the-art reasoning models across diverse domains including mathematical reasoning, code generation, and agentic tasks. Our results demonstrate that ARISE provides a reliable and fine-grained measurement of test-time scaling capabilities, revealing significant variations in scaling efficiency across models. Notably, our evaluation identifies Claude Opus as exhibiting superior scaling characteristics compared to other contemporary reasoning models.","categories":"cs.AI cs.CL stat.ML","published":"2025-10-07"},{"arxiv_id":"(Li et al., 1 Feb 2026)","title":"Predicting and improving test-time scaling laws via reward tail-guided search","authors":"Tianyang Li, Jingyang Yuan, Miaoran Li, Junxuan Zhang, Yazhen Wang, Dimitris Bertsimas, Huan Wang","abstract":"Test-time scaling has emerged as a critical avenue for enhancing the reasoning capabilities of LLMs. Though the straight-forward ''best-of- $N$ '' (BoN) strategy has already demonstrated significant improvements in performance, it lacks principled guidance on the choice of $N$ , budget allocation, and multi-stage decision-making, thereby leaving substantial room for optimization. While many works have explored such optimization, rigorous theoretical guarantees remain limited. In this work, we propose new methodologies to predict and improve scaling properties via tail-guided search. By estimating the tail distribution of rewards, our method predicts the scaling law of LLMs without the need for exhaustive evaluations. Leveraging this prediction tool, we introduce Scaling-Law Guided (SLG) Search, a new test-time algorithm that dynamically allocates compute to identify and exploit intermediate states with the highest predicted potential. We theoretically prove that SLG achieves vanishing regret compared to perfect-information oracles, and achieves expected rewards that would otherwise require a polynomially larger compute budget required when using BoN. Empirically, we validate our framework across different LLMs and reward models, confirming that tail-guided allocation consistently achieves higher reward yields than Best-of- $N$ under identical compute budgets.","categories":"cs.LG cs.AI cs.CL stat.ML","published":"2026-02-01"}]} Orchestrated Test-Time Scaling (OTTS) denotes structured, policy-driven ways of allocating and sequencing inference compute at test time so that a model can explore solution trajectories more effectively within a fixed or progressively expanding budget (Yin et al., 7 Oct 2025). In contrast to naive scaling—simply increasing tokens or samples—OTTS explicitly governs how compute is spent across steps, branches, samples, tools, queries, or modules, and can include adaptive cross-query allocation, token-level aggregation and adaptation, graph-guided traversal, history-aware verification, agentic explore-or-stop control, and multi-agent or multi-reward coordination (Zuo et al., 15 Jun 2025, Kaya et al., 3 Oct 2025, Ye et al., 25 Jun 2026, Qin et al., 1 Jun 2026, Song et al., 5 Aug 2025). The term now spans a heterogeneous research area that includes LLMs, vision-LLMs, radiology VLLMs, embodied manipulation, agentic coding, text-to-SQL, recommendation, mobile inference, and evaluation methodology.

1. Conceptual scope and distinguishing features

OTTS is best understood as an inference-time control problem. One line of work defines it as “structured, policy-driven ways of allocating and sequencing inference compute at test time,” with practical policies including self-consistency with $k$ samples, iterative refinement, tree-of-thought branching, reflection loops, and tool-mediated steps (Yin et al., 7 Oct 2025). Another line emphasizes that orchestration can occur across a batch of queries rather than only within a single query: the bandit formulation treats each query as an arm and assigns more compute to challenging but solvable instances while stopping early on solved ones (Zuo et al., 15 Jun 2025). A third line treats orchestration as end-to-end control owned by an LLM orchestrator, as in ATLAS, where a controller repeatedly decides whether to “explore” again or to stop and synthesize a final answer from accumulated evidence (Qin et al., 1 Jun 2026).

A recurring misconception is that OTTS is equivalent to “more tokens” or “more samples.” Several papers explicitly reject that interpretation. ARISE contrasts OTTS with naive scaling by arguing that well-orchestrated policies channel compute into high-yield transitions while mitigating waste (Yin et al., 7 Oct 2025). The system-oriented study argues that compute-optimal is not always system-optimal, because latency, cost-per-token, communication overhead, and memory can reverse the practical ranking of strategies (Zhao et al., 23 Sep 2025). This suggests that OTTS is not merely a statistical scaling phenomenon; it is also a scheduling, systems, and control problem.

OTTS is also not confined to text-only reasoning. It appears in small VLMs as the coordination of Test-Time Augmentation and Test-Time Adaptation (Kaya et al., 3 Oct 2025), in radiology as Thought Graph Traversal plus reasoning budget forcing (Yao et al., 13 Jun 2025), in robotics as history-aware iterative refinement with reasoning and action verifiers (Ye et al., 25 Jun 2026), and in recommendation as deployment-time prediction merging across multiple models or seeds (Lyu et al., 8 Dec 2025). A plausible implication is that OTTS has become a general abstraction for inference-time resource allocation across modalities rather than a narrow LLM-specific trick.

2. Budgeted control primitives and formal structure

A recurring formal pattern is budget-constrained optimization. In the bandit formulation of strategic scaling, queries $x_i$ receive allocations $c(x_i)$ under a global budget $B$ , and the objective is

$\max_{\{c(x_i)\}} \frac{1}{n}\sum_{i=1}^n \mathrm{Metric}(x_i;c(x_i)) \quad \text{subject to} \quad \sum_i c(x_i)\le B$

with coverage and accuracy as the primary metrics (Zuo et al., 15 Jun 2025). In Text-to-SQL, the total cost is decomposed into internal, sequential, parallel, and selection components,

$C_{\mathrm{total}} = C_{\mathrm{int}} + T(q)C_{\mathrm{seq}} + N(q)C_{\mathrm{par}} + C_{\mathrm{sel}},$

making orchestration an explicit problem of allocating effort across generation, repair, and tournament selection (Wang et al., 29 Sep 2025).

Despite this common budgeted form, the controlled object differs across papers. In cross-query orchestration, the unit is a query; in Best-of- $N$ and self-consistency, it is a complete sampled trace; in small VLMs, it is the token-level next-token distribution aggregated over augmentations; in embodied robotics, it is a paired reasoning–action unit $N$ 0 scored by dual verifiers; in agentic coding, it is a compact trajectory summary rather than the raw rollout itself (Kaya et al., 3 Oct 2025, Ye et al., 25 Jun 2026, Kim et al., 16 Apr 2026).

The control signals used by OTTS are likewise diverse but structurally similar. The bandit orchestrator uses empirical average reward, gap to a correctness threshold, and response entropy to decide where to allocate the next unit of compute (Zuo et al., 15 Jun 2025). The small-VLM setting aggregates token probabilities across augmentations and derives consensus pseudolabels for test-time adaptation (Kaya et al., 3 Oct 2025). E-TTS combines reasoning and action scores multiplicatively,

$N$ 1

then applies an $N$ 2-greedy thresholded selector with feedback-guided resampling (Ye et al., 25 Jun 2026). ATLAS reduces orchestration to a single primitive action, explore, but expands the state through a Thought/Action/Observation trajectory whose accumulated evidence supports stopping or further exploration (Qin et al., 1 Jun 2026).

Two design motifs recur across these formulations. The first is early termination: elimination in the bandit setting, stopping at convergence in ATLAS, or ending refinement when no further gains appear. The second is selective intensification: concentrating budget on difficult queries, solvable hard instances, promising latent states, selected tool agents, or high-quality rollout summaries rather than applying uniform effort everywhere (Zuo et al., 15 Jun 2025, Li et al., 1 Feb 2026, Wong et al., 18 Feb 2026, Kim et al., 16 Apr 2026).

3. Representative orchestration regimes

The literature now contains several distinct but overlapping OTTS regimes.

Regime	Coordination unit	Representative instantiation
Cross-query allocation	Queries as arms	Elimination, UCB, Gap, Entropy (Zuo et al., 15 Jun 2025)
Token-level augmentation and adaptation	Aggregated next-token distributions	TTAug and TTAdapt (Kaya et al., 3 Oct 2025)
Graph-guided traversal	Organ and diagnostic question nodes	Thought Graph Traversal (Yao et al., 13 Jun 2025)
History-aware iterative refinement	Reasoning–action joint units	E-TTS (Ye et al., 25 Jun 2026)
Explore-or-stop orchestration	Independent solver calls	ATLAS and ATLAS-MM (Qin et al., 1 Jun 2026)
Multi-agent and multi-reward coordination	Candidate pools and reward ensembles	CTTS-MM (Song et al., 5 Aug 2025)

These regimes differ primarily in what is being orchestrated. Strategic scaling orchestrates compute across a batch of heterogeneous queries and proves a sample-complexity separation from uniform allocation, with $N$ 3 versus $N$ 4 under the Bernoulli model and oracle-threshold compatibility (Zuo et al., 15 Jun 2025). In small VLMs, orchestration coordinates a two-stage source-free loop: token-level TTAug builds a consensus distribution, and TTAdapt briefly updates the full VLM parameter set using consensus pseudolabels before resetting weights for the next question (Kaya et al., 3 Oct 2025).

Other regimes are more explicitly structural. In radiology, Thought Graph Traversal imposes a medically coherent Root $N$ 5 organ $N$ 6 question order, verifies each organ-specific answer, and dynamically extends the reasoning budget when uncertainty or inconsistency is detected (Yao et al., 13 Jun 2025). In robotics, E-TTS introduces a history buffer, dual verifiers, adaptive selection, and a feedback generator that closes the loop between rejected candidates and subsequent sampling (Ye et al., 25 Jun 2026). In agentic coding, long-horizon rollouts are first compressed into structured summaries $N$ 7, then subjected to Recursive Tournament Voting for selection and Parallel–Distill–Refine for reuse (Kim et al., 16 Apr 2026).

The concept also appears in less conventional settings. Team-of-Thoughts uses an orchestrator-tool paradigm with calibrated orchestrator choice and self-assessed tool-agent profiles to activate a small subset of heterogeneous models (Wong et al., 18 Feb 2026). LatentEvolve moves orchestration into latent space through alternating daytime scaling and nighttime scaling, combining episodic retrieval with slow consolidation into a latent weaver (Zhang et al., 29 Sep 2025). On smartphones, orchestration is hardware-aware: multiple candidate generations are run in parallel on mobile NPUs, and their outputs are selected through Best-of- $N$ 8 or beam search while tile-aware quantization and LUT-based kernels exploit otherwise idle matrix capacity (Hao et al., 27 Sep 2025).

4. Empirical behavior across domains

Empirical results are strong but heterogeneous. In cross-query bandit allocation, adaptive orchestration achieves up to an 11.10% performance improvement (15.04% relative) on MATH-500 and up to a 7.41% performance improvement (14.40% relative) on LiveCodeBench, with efficiency gains up to $N$ 9 on MATH-500 and about $N$ 0 to $N$ 1 in some settings (Zuo et al., 15 Jun 2025). In small VLMs, token-level aggregation raises mean accuracy from 43.8 to 47.9, and model-parameter adaptation yields the strongest overall gains, including COCO ROUGE-L from 9.1 to 16.9 with TTAug and to 35.9 with TTAdapt, although some tasks degrade under TTAdapt, indicating domain-dependent adaptation aggressiveness (Kaya et al., 3 Oct 2025).

In domain-specific multimodal systems, orchestration can be even more pronounced. Thought Graph Traversal improves chest X-ray report generation across GPT-4o, Qwen2.5-VL, and HuatuoGPT-Vision on IU X-Ray and MIMIC-CXR, and reveals that performance rises with more targeted diagnostic questions and with reasoning tokens up to a saturation point near 450 reasoning tokens (Yao et al., 13 Jun 2025). E-TTS reports up to a 33.14% increase in simulation and 26.62% in real-world manipulation scenarios without additional expert data collection or retraining (Ye et al., 25 Jun 2026). Agentic coding shows large gains when representation, selection, and reuse are coordinated: Claude-4.5-Opus improves from 70.9% to 77.6% on SWE-Bench Verified and from 46.9% to 59.1% on Terminal-Bench v2.0 (Kim et al., 16 Apr 2026).

Collective and agentic orchestration produce a different empirical profile. CTTS reports that MA-MR consistently achieves the best performance among SA-SR, SA-MR, and MA-SR paradigms, and CTTS-MM reaches an average accuracy of about 78.84% across seven benchmarks (Song et al., 5 Aug 2025). ATLAS reaches 56.00% on HLE-Verified, 82.29% on LiveCodeBench, 85.75% on GPQA-Diamond, and 23.71% on BabyVision while using far fewer API calls than fixed-workflow baselines; ATLAS-MM further improves HLE-Verified to 60.00% and LiveCodeBench to 85.63% (Qin et al., 1 Jun 2026). In Team-of-Thoughts, dynamic activation of heterogeneous tool agents yields 96.67% on AIME24 and 72.53% on LiveCodeBench, substantially above homogeneous role-play baselines (Wong et al., 18 Feb 2026).

Several papers also show that OTTS is not a single universally dominant recipe. “The Art of Scaling Test-Time Compute for LLMs” reports three trends: no single TTS strategy universally dominates, reasoning models separate into short-horizon and long-horizon categories, and for a given model type the optimal TTS performance scales monotonically with compute budget (Agarwal et al., 1 Dec 2025). This suggests that orchestration policy selection depends not only on task family but also on model-internal trace-quality behavior.

5. Evaluation, ranking, and system-optimality

As OTTS matured, evaluation itself became an object of study. ARISE introduces a sample-level, resolution-aware metric in which each transition from iteration $N$ 2 to $N$ 3 contributes

$N$ 4

so that negative scaling is penalized more strongly than positive scaling is rewarded (Yin et al., 7 Oct 2025). ARISE also adds dynamic sampling with defaults $N$ 5, $N$ 6, and $N$ 7, and reports that adaptive sampling reduced variance by 76.1% for ARISE versus 54.2% for SM under one comparison, while under identical budgets it achieved about 57.5% variance reduction versus about 31.4% for naive uniform multi-sampling (Yin et al., 7 Oct 2025).

Ranking models under TTS requires repeated-trial statistics rather than single-pass leaderboards. Scorio formalizes dense benchmark ranking with a binary tensor $N$ 8 and finds that, across 20 reasoning models and four Olympiad-style math benchmarks with up to $N$ 9 trials, most full-trial rankings agree closely with the Bayesian gold standard $k$ 0, with mean Kendall’s $k$ 1-- $k$ 2, and 19--34 methods recover exactly the same ordering (Hariri et al., 11 Mar 2026). In the single-trial regime, the best methods reach $k$ 3, and using greedy decoding as an empirical prior reduces variance at $k$ 4 by 16--52%, although it can bias rankings when greedy and stochastic sampling disagree (Hariri et al., 11 Mar 2026).

A separate systems perspective argues that ranking only by accuracy-versus-compute is inadequate. On long-sequence reasoning workloads, speculative decoding with a simple n-gram predictor consistently reduces latency compared to greedy decoding, whereas tensor parallelism often produces diminishing returns or even system-level regressions (Zhao et al., 23 Sep 2025). For DeepSeek-R1-Distilled-Qwen-14B, scaling to 4 GPUs improves latency by an average of 1.7× rather than proportionally, and tensor parallelism can increase cost-per-token relative to baseline single-GPU execution (Zhao et al., 23 Sep 2025). This is reinforced by the smartphone setting, where orchestrated parallel hypotheses on mobile NPUs achieve up to 19.0 for mixed-precision GEMM and 2.2 for Softmax, enabling smaller models with test-time scaling to match or exceed larger models under realistic mobile constraints (Hao et al., 27 Sep 2025). A plausible implication is that OTTS evaluation must jointly track accuracy, latency, token cost, memory, communication, and hardware utilization rather than only tokens or FLOPs.

6. Limitations, controversies, and emerging directions

The limitations reported across OTTS papers are strikingly consistent. Reward noise or misalignment remains a central failure mode in bandit-style allocation because oracle-threshold compatibility is only approximate in many practical settings (Zuo et al., 15 Jun 2025). In TTAug/TTAdapt, pseudolabel reliability, catastrophic forgetting or drift, sensitivity to $k$ 5, and calibration-sensitive weighting limit robustness, especially on domains such as OCRBench and TextVQA where TTAdapt can degrade performance (Kaya et al., 3 Oct 2025). Thought Graph Traversal depends on structured priors and curated question sets, and can amplify dataset biases even as it makes them visible through traceable reasoning paths (Yao et al., 13 Jun 2025). In robotics, verifier reliability, long-horizon memory drift, prompt sensitivity, and latency overhead remain unresolved, especially for highly dynamic or safety-critical settings (Ye et al., 25 Jun 2026).

Agentic orchestration introduces further issues. ATLAS can over-explore or under-explore depending on its stopping disposition, and its performance depends materially on orchestrator backbone strength and stateful evidence management (Qin et al., 1 Jun 2026). Collective systems such as CTTS-MM and Team-of-Thoughts reduce some single-model biases but inherit reward-model domain mismatch, tool-profile drift, and coordination failures (Song et al., 5 Aug 2025, Wong et al., 18 Feb 2026). Representation-heavy methods for coding agents can suffer from summary drift, over-pruning, or over-conditioning during refinement (Kim et al., 16 Apr 2026). In latent-space orchestration, memory contamination, sensitivity to hyperparameters such as $k$ 6, $k$ 7, $k$ 8, $k$ 9, and $x_i$ 0, and domain-shift failures in cosine-based retrieval remain open problems (Zhang et al., 29 Sep 2025).

Future directions recur with notable regularity. Strategic scaling identifies richer contexts, non-stationary or adversarial bandits, multi-objective orchestration, hierarchical scheduling, tool-use integration, and meta-learning or prior shaping as immediate extensions (Zuo et al., 15 Jun 2025). E-TTS emphasizes uncertainty-aware verifiers, learned weighting between reasoning and action scores, dynamic history management, and asynchronous acceleration (Ye et al., 25 Jun 2026). Agentic frameworks expose richer action spaces, broader multi-model pools, and stronger synthesis contracts as natural next steps (Qin et al., 1 Jun 2026). The systems view argues for a paradigm shift toward holistic, system-aware evaluations rather than compute-only scaling laws (Zhao et al., 23 Sep 2025).

Taken together, these lines of work indicate that OTTS is evolving from fixed, designer-engineered workflows toward adaptive controllers that estimate difficulty, expected information gain, or solution potential online. This suggests that the long-term trajectory of OTTS is not simply larger inference budgets, but increasingly explicit policies for deciding when, where, and in what form additional inference-time computation is worth spending.