Visual Test-Time Scaling (VTTS)

• VTTS is a paradigm of inference-time strategies that improves visual and multimodal performance by reallocating computational resources based on task demands.
• It employs mechanisms like candidate search, iterative perception, and token scaling to refine outputs through adaptive verification and branching.
• Empirical studies demonstrate that VTTS significantly enhances video generation, visual reasoning, and medical diagnosis, though benefits vary with task and model size.

Visual Test-Time Scaling (VTTS) denotes a family of inference-time strategies that improve visual or multimodal performance by allocating additional computation at test time rather than by changing model weights. In the current literature, that additional computation may appear as candidate search, self-consistency, iterative perception, tool-assisted zooming and cropping, visual-context pruning, extra reasoning tokens, or verifier-guided branching. VTTS has been studied in large vision-LLMs (LVLMs), multimodal LLMs (MLLMs), visual autoregressive models, GUI agents, visual grounding systems, medical visual question answering, and video generation; the common premise is that inference can be scaled or reallocated in a task-dependent way, although the benefits are neither uniform nor unlimited (Sammani et al., 27 Jun 2026).

1. Scope and conceptual framing

A central distinction in recent work is that VTTS is not restricted to “thinking longer” in text. In "Adaptive Visual Inference Scaling" (Jeddi et al., 10 Jun 2026), inference cost is decomposed into two coupled axes: Visual Context Scaling (VCS), which controls how much visual evidence is passed to the LLM, and Visual Reasoning Scaling (VRS), which controls how much inference-time reasoning search is performed. In "Test-time Scaling over Perception" (Jiang et al., 13 Apr 2026), the premise is sharper still: perception itself is treated as a scalable inference process. In "VideoChat-R1.5" (Yan et al., 25 Sep 2025), VTTS is defined through iterative perception during inference, so that the model can increase its perceptual compute rather than only its textual reasoning. In streaming video generation, "Stream-T1" (Tu et al., 6 May 2026) explicitly describes VTTS as a comprehensive test-time framework tailored to chunked autoregressive synthesis.

A useful summary of the literature is therefore not a single algorithm but a set of recurrent design choices organized around where extra inference budget is spent.

Setting	Inference-time mechanism	Representative papers
Visual generation	Candidate search, pruning, reward-guided branching	(Liu et al., 24 Mar 2025, Park et al., 26 Nov 2025, Tu et al., 6 May 2026)
Vision-language reasoning	Self-consistency, iterative perception, visual-context scaling	(Jeddi et al., 10 Jun 2026, Yan et al., 25 Sep 2025, Jiang et al., 13 Apr 2026)
Grounding and agents	Zooming, region proposals, token scaling, multi-agent collaboration	(Luo et al., 1 May 2025, Zhan et al., 20 Jan 2026, Yu et al., 5 Aug 2025)

The comprehensive LVLM study "On Test-Time Scaling for Vision-LLMs" (Sammani et al., 27 Jun 2026) further broadens the scope by evaluating nine test-time scaling methods across 13 instruction-tuned LVLMs and six benchmarks, including CoT prompting, Structured CoT, Plan-and-Solve, Self-Consistency, Self-Aggregation, Self-Refinement, Prompt Repetition, Describe-Answer, and Compositional Chain-of-Thought. That study is important because it treats VTTS not as a single branded framework but as a general inference paradigm for visual reasoning.

2. Core mechanisms

A dominant VTTS mechanism is candidate expansion followed by verification or pruning. In "GridAR" (Park et al., 26 Nov 2025), the canvas is partitioned into row-wise grids; multiple partial candidates are generated for the same region; infeasible ones are pruned early by a verifier; and accepted partials are fixed as anchors for later decoding. This directly contrasts with Best-of-$N$, which finishes full images before ranking them. In "Video-T1" (Liu et al., 24 Mar 2025), the search space is the trajectory from Gaussian noise to video, and the principal alternatives are an intuitive linear search strategy and Tree-of-Frames (ToF), which adaptively expands and prunes video branches in an autoregressive manner. In "Stream-T1" (Tu et al., 6 May 2026), the chunk-wise autoregressive structure makes beam-style exploration practical because synthesis is localized and uses few denoising steps.

A second mechanism is iterative perception. "VideoChat-R1.5" (Yan et al., 25 Sep 2025) alternates between global and focused views through Iterative Perception (ITP): the model first processes the visual input at coarse scale, predicts candidate spatio-temporal regions, and then re-processes the input with attention concentrated on those regions in later rounds. "TTSP" (Jiang et al., 13 Apr 2026) formalizes the same intuition as a response to the Grounding Paradox: deciding where to look requires the evidence that can only be obtained after looking. Its exploratory-trace formulation is explicitly probabilistic:

$$P\bigl(\exists\, i: \mathrm{IoU}(\hat R_i, R^*) \ge \delta \bigr) = 1 - \prod_{i=1}^{K} P\bigl(\mathrm{IoU}(\hat R_i, R^*) < \delta\bigr).$$

Increasing the number of diverse traces raises the probability that at least one trace covers the essential evidence region (Jiang et al., 13 Apr 2026).

A third mechanism is aggregation over multiple reasoning paths or descriptions. In zero-shot medical diagnosis, a vision-LLM generates multiple descriptions of the image, an LLM produces a diagnosis for each description, and the final class probability is approximated by averaging over outputs:

$$p(\hat{y}=1 \mid x) \approx \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(a^{(i)}=\boxed{1}).$$

This converts description diversity into more reliable final decisions without training or fine-tuning (Byun et al., 11 Jun 2025). In AVIS, the analogous reasoning-side mechanism is adaptive self-consistency: a learned difficulty predictor selects the number of reasoning rollouts, and all rollouts reuse a shared prefilling pass and KV cache through shared-prefill inference (Jeddi et al., 10 Jun 2026).

A fourth mechanism is test-time token or budget scaling. In "Scaling Test-time Inference for Visual Grounding" (Zhan et al., 20 Jan 2026), small VLMs are prompted to generate more intermediate reasoning tokens before outputting a grounding box. In MACT (Yu et al., 5 Aug 2025), the judgment agent uses internal scaling through budget forcing, which enforces a minimum token budget for correctness analysis. In these systems, the scaled resource is not visual branching alone but the length or depth of inference.

3. VTTS in image and video generation

VTTS entered visual generation through search-based reinterpretations of sampling. "Video-T1" (Liu et al., 24 Mar 2025) casts test-time scaling for video generation as a search problem over noise trajectories, using a generator, one or more test-time verifiers, and heuristic search. The simplest strategy is linear Best-of-$N$ search; the more efficient one is Tree-of-Frames, where only promising branches are expanded. The paper reports that TTS boosts VBench scores by 2-19% depending on dimension and baseline quality, and that ToF achieves similar or better video quality than linear search at much lower computation. Its practical complexity is stated as $O(N+T)$ when branching is sparse (Liu et al., 24 Mar 2025).

"Stream-T1" (Tu et al., 6 May 2026) extends the same logic to streaming video generation and argues that chunk-level synthesis with few denoising steps is intrinsically suited for TTS. Its first unit, Stream-Scaled Noise Propagation, correlates adjacent chunks by propagating optimized historical noise:

$$x_T^n = \beta x_T^{n-1} + \sqrt{1-\beta^2}\,\epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$

The second unit, Stream-Scaled Reward Pruning, combines short-term frame-level and long-term sliding-window video-level rewards to balance local spatial aesthetics with global temporal coherence. The third unit, Stream-Scaled Memory Sinking, routes evicted KV-cache context into reward-guided memory pathways so that previously generated visual information can anchor later chunks. On 5s VBench and 30s MovieGen evaluations, the framework improves temporal consistency, motion smoothness, and frame-level visual quality, and it outperforms Best-of-$N$ and vanilla beam search in VTTS (Tu et al., 6 May 2026).

For visual autoregressive image generation, "GridAR" (Park et al., 26 Nov 2025) replaces raster-scan Best-of-$N$ with grid-partitioned progressive generation, early pruning of infeasible partials, and layout-specified prompt reformulation. The paper reports that with $N=4$, GridAR outperforms Best-of-$N$ with $$P\bigl(\exists\, i: \mathrm{IoU}(\hat R_i, R^*) \ge \delta \bigr) = 1 - \prod_{i=1}^{K} P\bigl(\mathrm{IoU}(\hat R_i, R^*) < \delta\bigr).$$0 by 14.4% on T2I-CompBench++ while reducing cost by 25.6%, and that it also improves semantic preservation on PIE-Bench by 13.9% over larger-$$P\bigl(\exists\, i: \mathrm{IoU}(\hat R_i, R^*) \ge \delta \bigr) = 1 - \prod_{i=1}^{K} P\bigl(\mathrm{IoU}(\hat R_i, R^*) < \delta\bigr).$$1 baselines (Park et al., 26 Nov 2025). The key point is that VTTS in generation need not mean repeated full-length decoding; it can mean localized branching before a full image or full video exists.

"Thinking in Frames" (Li et al., 28 Jan 2026) places video generation inside visual reasoning rather than media synthesis. In Maze Navigation, it reports a Visual Test-Time Scaling Law: OOD performance increases steadily as the inference budget is scaled from 61 to 121 frames, and higher frame budgets can induce emergent self-correction, where an initially wrong trajectory is reversed and repaired. The same paper also shows that this behavior is task-dependent: Tangram Puzzle does not exhibit the same monotonic improvement because geometric consistency becomes the bottleneck (Li et al., 28 Jan 2026).

4. VTTS in reasoning, grounding, documents, and agents

In visual grounding, VTTS is often used to substitute compute for parameter scale. "Scaling Test-time Inference for Visual Grounding" (Zhan et al., 20 Jan 2026) observes that small and large VLMs have nearly the same visual encoder size and that the major difference lies in the LLM. Its Efficient visual Grounding LLMs (EGM) therefore scale the number of generated reasoning tokens at inference. On RefCOCO, EGM-Qwen3-VL-8B reaches 91.4 IoU with 737ms average latency, while Qwen3-VL-235B reaches 90.5 IoU with 4,320ms, making the smaller model 5.9x faster (Zhan et al., 20 Jan 2026). The same framework is extended to amodal grounding, where the model predicts visible and occluded parts.

For GUI agents, "RegionFocus" (Luo et al., 1 May 2025) implements VTTS as dynamic zooming over promising sub-regions. Region proposals are triggered by environment feedback or self-judgment, local actions are predicted independently in cropped regions, and an image-as-map mechanism overlays landmarks to avoid retrying failed regions. Applied to Qwen2.5-VL-72B, RegionFocus achieves 61.6% grounding performance on ScreenSpot-Pro; on top of two state-of-the-art open vision LLM agents, the paper reports 28+\% gains on ScreenSpot-Pro and 24+\% gains on WebVoyager, with detailed improvements of 28.9% and 31.8% on ScreenSpot-Pro and 24.2% and 34.3% on WebVoyager for specific backbones (Luo et al., 1 May 2025).

In document understanding and VQA, MACT (Yu et al., 5 Aug 2025) distributes VTTS across specialized agents. The planning agent uses parallel scaling via diverse plan generation, the execution agent uses sequential scaling with step-level candidate selection by a reward model, the judgment agent uses budget forcing, and the answer agent uses no scaling. The three variants of MACT hold the top three positions in average scores, leading in 13 of the 15 benchmarks, and removing agent-wise hybrid TTS reduces average accuracy by 3.7% (Yu et al., 5 Aug 2025).

In multimodal reasoning over video and images, "VideoChat-R1.5" (Yan et al., 25 Sep 2025) combines iterative perception with reinforcement learning and spatio-temporal supervision. The VTTS-80K dataset provides question-answer pairs, spatio-temporal clues, and chain-of-thought explanations so that iterative region selection can be trained with GRPO. The paper reports an average increase of over 5\% across more than 15 benchmarks, with gains of +5.4% for Qwen2.5-VL-7B and +6.3% for Qwen2.5-VL-3B on major benchmarks (Yan et al., 25 Sep 2025).

In clinical visual question answering, test-time scaling is used to stabilize zero-shot diagnosis. The two-stage Describe-then-Diagnose framework in (Byun et al., 11 Jun 2025) improves PneumoniaMNIST AUC from 0.517 in the single-sample setting to 0.821 with TTS at $$P\bigl(\exists\, i: \mathrm{IoU}(\hat R_i, R^*) \ge \delta \bigr) = 1 - \prod_{i=1}^{K} P\bigl(\mathrm{IoU}(\hat R_i, R^*) < \delta\bigr).$$2, and the paper reports substantial gains on PathMNIST and RetinaMNIST as well. A stated empirical finding is that performance follows a power law as the number of TTS samples increases (Byun et al., 11 Jun 2025).

Finally, AVIS (Jeddi et al., 10 Jun 2026) addresses deployment cost directly. It realizes VCS through Key Diversity Visual (KDV) pruning, a training-free $$P\bigl(\exists\, i: \mathrm{IoU}(\hat R_i, R^*) \ge \delta \bigr) = 1 - \prod_{i=1}^{K} P\bigl(\mathrm{IoU}(\hat R_i, R^*) < \delta\bigr).$$3 key-based rule for removing redundant visual tokens before prefilling, and realizes VRS through adaptive self-consistency with a learned difficulty predictor. The framework is compatible with shared-prefill inference and remains effective on top of RL post-trained VLMs while keeping compute and latency low (Jeddi et al., 10 Jun 2026).

5. Empirical regularities, scaling laws, and failure modes

The broadest empirical pattern is that VTTS is highly task-dependent. The comprehensive LVLM study in (Sammani et al., 27 Jun 2026) reports that, contrary to earlier LLM findings, small, well-performing models benefit the most from test-time scaling, with improvements of up to around 30\%, often reaching or surpassing large-model performance. Yet the same study also reports that LVLMs lose focus when given more compute than necessary, and that visual information is encoded early in the reasoning chain, after which the chain is dominated by text-only reasoning and the contribution of image tokens drops significantly (Sammani et al., 27 Jun 2026).

A second regularity is that reasoning-heavy benchmarks benefit more than perception-heavy ones. "Limits and Gains of Test-Time Scaling in Vision-Language Reasoning" (Ahmadpour et al., 11 Dec 2025) finds that closed-source models consistently benefit from structured reasoning and iterative Self-Refinement, whereas open-source VLMs show inconsistent behavior: external verification provides the most reliable gains, while iterative refinement often degrades performance. The same study finds that TTS yields clear improvements on multi-step reasoning tasks such as MathVista and MMMU but offers only limited gains on perception-focused benchmarks such as MMBench (Ahmadpour et al., 11 Dec 2025).

A third regularity is the existence of a saturation point. Although "Scaling over Scaling" (Wang et al., 26 May 2025) studies large reasoning models rather than VTTS specifically, its Test-Time Scaling Performance Model provides a useful formal account of diminishing returns:

$$P\bigl(\exists\, i: \mathrm{IoU}(\hat R_i, R^*) \ge \delta \bigr) = 1 - \prod_{i=1}^{K} P\bigl(\mathrm{IoU}(\hat R_i, R^*) < \delta\bigr).$$4

with marginal gain

$$P\bigl(\exists\, i: \mathrm{IoU}(\hat R_i, R^*) \ge \delta \bigr) = 1 - \prod_{i=1}^{K} P\bigl(\mathrm{IoU}(\hat R_i, R^*) < \delta\bigr).$$5

and saturation point

$$P\bigl(\exists\, i: \mathrm{IoU}(\hat R_i, R^*) \ge \delta \bigr) = 1 - \prod_{i=1}^{K} P\bigl(\mathrm{IoU}(\hat R_i, R^*) < \delta\bigr).$$6

This suggests that many VTTS systems should be expected to exhibit rapid early gains followed by diminishing returns, a pattern explicitly reported in video generation, visual reasoning, and diagnosis settings (Wang et al., 26 May 2025).

A fourth regularity is that more test-time compute is not the only route to improvement. "Rethinking Dense Optical Flow without Test-Time Scaling" (Chanda et al., 8 May 2026) argues that strong visual semantic and geometric priors from frozen foundation models can reduce, if not overcome, the need for iterative refinement at test time. Its single-pass method reaches 2.81 EPE on Sintel Final without refinement, outperforming SEA-RAFT under comparable training conditions and improving over GMFlow without refinement (Chanda et al., 8 May 2026). This is a direct counterpoint to any strong claim that VTTS is universally necessary.

6. Verification, adaptation, and broader significance

Verification has emerged as one of the most stable design patterns in VTTS-adjacent systems. The survey "Trust but Verify!" (Venktesh et al., 20 Aug 2025) treats verifiers as the central mechanism of effective test-time scaling, categorizing them by supervision type, verification level, and output modality. Outcome Reward Models, Process Reward Models, prompt-based critics, symbolic verifiers, and RL-based verifiers all serve the same systems role: scoring or filtering candidate outputs so that extra inference-time compute is not spent uniformly, but selectively.

Visual systems repeatedly instantiate this verifier-centric pattern. "Video-T1" (Liu et al., 24 Mar 2025) uses a multi-verifier ensemble to rank candidate videos. "GridAR" (Park et al., 26 Nov 2025) uses a verifier to label partial candidates as possible or impossible. "Stream-T1" (Tu et al., 6 May 2026) fuses image-level and video-level reward models to prune chunk candidates. MACT (Yu et al., 5 Aug 2025) scores execution candidates with a pre-trained reward model and supplements this with mixed reward modeling. TTSP (Jiang et al., 13 Apr 2026) filters exploratory traces using entropy-based confidence estimation and then aggregates outputs by confidence-weighted voting. AVIS (Jeddi et al., 10 Jun 2026) adds a different form of verification pressure through difficulty prediction and token pruning rather than explicit candidate scoring.

Taken together, these works indicate that VTTS is increasingly about adaptive allocation rather than uniform overgeneration. Additional inference budget can be spent on branching, on looking again, on seeing more or less of the image, on generating longer reasoning traces, or on evaluating candidate trajectories more carefully. The literature also indicates that the most effective allocation depends on model family, task structure, and whether the bottleneck is perceptual evidence, language reasoning, temporal consistency, or calibration (Sammani et al., 27 Jun 2026). A plausible implication is that future VTTS research will continue to move from static “more compute everywhere” heuristics toward policies that jointly manage perception, reasoning, and verification under explicit latency and memory constraints.