Chain of Foresight-Focus Thought (CoFFT)

• Chain of Foresight-Focus Thought (CoFFT) is a visual reasoning framework that iteratively aligns textual reasoning with focused visual input to reduce irrelevant distractions.
• It implements a three-stage loop—diverse sample generation, dual foresight decoding, and visual focus adjustment—to incrementally update the reasoning path.
• Inspired by human visual cognition, CoFFT not only boosts VLM performance on cluttered images but also extends to embodied multimodal tasks via frameworks like EM-CoT and HALO.

Searching arXiv for the specified CoFFT and HALO papers to ground the article in current literature. Chain of Foresight-Focus Thought (CoFFT) is a visual reasoning framework for vision-LLMs (VLMs) that iteratively couples prospective reasoning with adaptive visual selection. It was introduced as a training-free approach designed to mitigate interference from irrelevant visual content by emulating a human-inspired cycle in which reasoning guides where to look next, and revised visual focus informs subsequent reasoning (Zhang et al., 26 Sep 2025). In later embodied vision-language-action work, the same conceptual structure was extended into Embodied Multimodal Chain-of-Thought (EM-CoT), where CoFFT-like decomposition appears as a sequence of textual reasoning, visual foresight, and action prediction for robotic control (Shou et al., 24 Feb 2026). Taken together, these formulations define CoFFT as a broader reasoning paradigm centered on explicit foresight and focus rather than a single monolithic inference pass.

1. Conceptual basis and problem formulation

CoFFT was proposed in response to a specific failure mode of VLMs: when images contain large amounts of irrelevant or distracting content, the models are susceptible to interference, excessive task-irrelevant reasoning, or hallucinations (Zhang et al., 26 Sep 2025). The underlying diagnosis is that conventional reasoning pipelines do not discover and process the required regions precisely during inference. Existing single-pass or purely language-based chain-of-thought methods, including Monte Carlo Tree Search and Predictive Decoding, were described as unable to dynamically refine visual input, which leaves them exposed to salient but irrelevant features (Zhang et al., 26 Sep 2025).

The conceptual inspiration is human visual cognition. In this account, humans apply “foresight” to evaluate which regions are likely to support future reasoning steps and then shift visual focus to those regions. CoFFT operationalizes this as an iterative cycle in which reasoning guides focus and focus guides reasoning (Zhang et al., 26 Sep 2025). Its two core notions are therefore “Foresight Thought,” which generates and evaluates multiple short candidate reasoning paths, and “Focus Thought,” which adjusts the model’s visual input toward an informative sub-region before reasoning continues (Zhang et al., 26 Sep 2025).

A common misconception is to treat CoFFT as merely another chain-of-thought prompting strategy. The published formulation is narrower and more specific: it is not only a reasoning-trace mechanism but an iterative procedure that jointly scores candidate reasoning paths and changes the visual input itself through attention-based cropping (Zhang et al., 26 Sep 2025). This distinguishes it from approaches that expand reasoning depth without modifying perceptual focus.

2. Iterative three-stage loop

CoFFT is organized as a repeated three-stage loop: Diverse Sample Generation (DSG), Dual Foresight Decoding (DFD), and Visual Focus Adjustment (VFA) (Zhang et al., 26 Sep 2025). At iteration $t$, the method maintains a current reasoning chain $R_t$, a current visual focus image $V_t$, and the question $Q$.

In the first stage, the model samples $k$ diverse reasoning continuations, each with length up to $l$:

$$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$

Here, $M$ is the VLM and $T_i \in [T_{\min},T_{\max}]$ is a temperature chosen to promote diversity (Zhang et al., 26 Sep 2025). The reported sampling strategy uses temperatures in $\{0.4, 0.5, \dots, 1.0\}$ without replacement, with weights halved after each use and reset once exhausted (Zhang et al., 26 Sep 2025). The purpose is exploration of multiple reasoning “foresights.”

In the second stage, CoFFT evaluates each candidate using both a visual criterion and a reasoning-progression criterion, then appends only the first step of the optimal sample to the reasoning chain (Zhang et al., 26 Sep 2025). This first-step update is central: the method does not commit to an entire sampled path, but instead uses limited lookahead to decide the next reasoning increment.

In the third stage, the framework updates visual focus by computing a crop-score map and selecting a rectangular window from a predefined family of windows spanning 40%–90% of the original image dimensions (Zhang et al., 26 Sep 2025). If the best window exceeds a threshold relative to the global mean, the system crops and upscales that region for the next iteration; otherwise it reverts to the full image (Zhang et al., 26 Sep 2025). The loop repeats until the answer is reached or “REASONING_COMPLETE” is emitted (Zhang et al., 26 Sep 2025).

Stage	Function	Output
Diverse Sample Generation	Generate $R_t$0 candidate reasoning paths with varied temperatures	$R_t$1
Dual Foresight Decoding	Score candidates by visual focus and reasoning progression	selected sample $R_t$2
Visual Focus Adjustment	Crop and magnify the most informative region or keep full image	next visual input $R_t$3

This architecture makes CoFFT an iterative inference controller rather than a learned end-to-end model. A plausible implication is that its principal novelty lies in coordinating search over reasoning space with search over perceptual focus.

3. Dual scoring and focus adjustment

The defining technical feature of CoFFT is Dual Foresight Decoding, which combines a visual focus score $R_t$4 and a reasoning progression score $R_t$5 for each candidate sample $R_t$6 (Zhang et al., 26 Sep 2025).

The visual score is built from a “relative attention” map that normalizes task-specific attention against a descriptive baseline:

$R_t$7

where $R_t$8 “Describe the image in detail.” The score is then defined as

$R_t$9

This combines attention-map cosine similarity with thresholded intersection-over-union, thereby tying a candidate reasoning path to the question-relevant visual distribution (Zhang et al., 26 Sep 2025).

The reasoning progression score measures whether adding early steps of a candidate increases the mean log-probability relative to the current reasoning chain:

$V_t$0

where $V_t$1 is the mean log-probability of the current chain and $V_t$2 is the mean log-probability of the chain extended by the first $V_t$3 steps of $V_t$4 (Zhang et al., 26 Sep 2025). This makes CoFFT sensitive both to where a reasoning path is visually grounded and to whether it improves the model’s internal continuation confidence.

The combined selection rule is

$V_t$5

with empirically $V_t$6 (Zhang et al., 26 Sep 2025). The updated chain is

$V_t$7

Visual Focus Adjustment then constructs a question-relevance map,

$V_t$8

a future-relevance map $V_t$9, and the combined crop-score map

$Q$0

with empirically $Q$1 (Zhang et al., 26 Sep 2025). The best window $Q$2 is chosen by maximizing mean crop-score $Q$3, and cropping occurs only if

$Q$4

where

$Q$5

(Zhang et al., 26 Sep 2025). The paper also gives the map-update notation

$Q$6

where $Q$7 captures the attention concentration induced by $Q$8 (Zhang et al., 26 Sep 2025).

These design choices clarify that CoFFT is not gradient-based test-time optimization. It is explicitly described as gradient-free in the map-update notation and operates by decoding, scoring, and cropping during inference (Zhang et al., 26 Sep 2025).

4. Empirical performance and efficiency

CoFFT was evaluated on Qwen2.5-VL-Instruct (7B and 32B), InternVL2.5-Instruct (8B), and Llava-Next (7B) across seven datasets: MathVista, MathVision, M3CoT, MMStar, Charxiv, SeekWorld-China, and SeekWorld-Global (Zhang et al., 26 Sep 2025). The reported metric is Pass@1 accuracy, averaged over test sets, while efficiency is summarized by $Q$9, where $k$0 is tokens generated and $k$1 is model parameters (Zhang et al., 26 Sep 2025).

Across all VLMs and seven datasets, CoFFT is reported to yield consistent improvements of 3.1%–5.8% absolute over baseline single-pass inference (Zhang et al., 26 Sep 2025). The paper gives the following representative averages over six benchmarks.

Model	Baseline	CoFFT
Qwen2.5-VL-7B	42.72	48.19
Llava-Next-7B	22.41	25.54
InternVL2.5-8B	40.84	45.30
Qwen2.5-VL-32B	47.05	52.96

The Qwen2.5-VL-32B result corresponds to a +5.91% gain, while the 7B, Llava-Next, and InternVL2.5 variants show +5.47%, +3.13%, and +4.46% respectively (Zhang et al., 26 Sep 2025). The work therefore frames CoFFT as model-agnostic inference-time augmentation rather than a method tied to a single backbone.

The computational overhead is substantial but bounded relative to more exhaustive search. For Qwen2.5-VL-7B, baseline FLOPS are reported as $k$2, CoFFT FLOPS as $k$3, and full Monte Carlo Tree Search as $k$4 (Zhang et al., 26 Sep 2025). The paper states that CoFFT remains far more efficient than full Monte Carlo Tree Search while delivering larger gains (Zhang et al., 26 Sep 2025).

Sensitivity studies further characterize the cost-performance tradeoff. With fixed $k$5, increasing $k$6 from 2 to 4 to 10 raises FLOPS from $k$7 to $k$8 to $k$9, while improving MathVista from 68.8% to 70.4% to 72.2% and SeekWorld-China from 34.32% to 35.12% to 37.27% (Zhang et al., 26 Sep 2025). With fixed $l$0, increasing $l$1 from 3 to 5 to 7 raises FLOPS from $l$2 to $l$3 to $l$4, with MathVista at 68.8%, 70.4%, and 70.7%, and SeekWorld-China at 33.51%, 35.12%, and 35.92% respectively (Zhang et al., 26 Sep 2025). This suggests a gradual scaling behavior rather than a sharply diminishing regime, although the paper does not formalize an optimal operating point.

5. Ablations, limitations, and methodological boundaries

Ablation results indicate that both Dual Foresight Decoding and Visual Focus Adjustment are necessary for the full effect. On Qwen2.5-VL-7B, removing Dual Foresight Decoding reduces average accuracy from 48.19% to 44.31%, and removing Visual Focus Adjustment reduces it to 45.31% (Zhang et al., 26 Sep 2025). The paper therefore concludes that both components are critical and that their synergy yields the largest boost (Zhang et al., 26 Sep 2025).

The method was also compared with combinations involving existing techniques on MathVista and SeekWorld-China. DyFo + Predictive Decoding reached an average of 51.02%, VFA + Predictive Decoding 50.24%, and CoFFT (VFA+DFD) 52.76% (Zhang et al., 26 Sep 2025). Within the evidence provided, this positions CoFFT as more than a simple crop heuristic layered on top of predictive decoding.

The limitations are explicit. First, CoFFT incurs additional computational cost due to iterative sampling and evaluation (Zhang et al., 26 Sep 2025). Second, performance can degrade if cropping omits globally relevant context, and careful tuning of $l$5 and $l$6 is needed (Zhang et al., 26 Sep 2025). Third, when the base VLM already excels with the full image, iterative cropping can sometimes degrade performance by over-focusing (Zhang et al., 26 Sep 2025). These caveats are important because they delimit the class of tasks for which adaptive focus is advantageous.

A related misconception is that “more focus” is always beneficial. The published limitations directly reject that interpretation: over-focusing can remove necessary global context, so the full-image fallback is not a peripheral detail but part of the method’s error-control mechanism (Zhang et al., 26 Sep 2025).

6. Extension to embodied multimodal reasoning

The HALO framework extends the CoFFT concept into embodied vision-language-action modeling through Embodied Multimodal Chain-of-Thought (EM-CoT) (Shou et al., 24 Feb 2026). In that formulation, EM-CoT is stated to extend CoFFT to embodied vision-language-action by explicitly decomposing policy learning into three human-like cognitive stages: “thinking” (textual reasoning), “imagination” (visual foresight), and “execution” (action prediction) (Shou et al., 24 Feb 2026). The relation is made explicit: EM-CoT corresponds to CoFFT by conditioning motor outputs not only on past observations but also on an intermediate, interpretable reasoning trace and imagined future states, ensuring that every control command is both semantically planned and visually grounded (Shou et al., 24 Feb 2026).

The HALO formalization defines a policy decomposition over instructions $l$7, visual observation history $l$8, and next action chunk $l$9:

• textual task reasoning:

$$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$0

with next-token cross-entropy loss

$$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$1

• visual subgoal foresight:

$$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$2

trained with a diffusion-style flow-matching objective in latent space;

• action prediction:

$$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$3

trained with an $$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$4 flow-matching loss (Shou et al., 24 Feb 2026).

The fine-tuning objective is given as

$$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$5

(Shou et al., 24 Feb 2026). Architecturally, HALO uses a Mixture-of-Transformers (MoT) with three expert lanes initialized from a Qwen2.5-1.5B LLM: a Multimodal Understanding Expert, a Visual Generation Expert, and an Action Prediction Expert (Shou et al., 24 Feb 2026). Cross-expert collaboration occurs through a shared self-attention mechanism and special control tokens that route representations to reasoning, visual, or action experts, while attention masks enforce causal masking on text generation, bidirectional intra-frame attention for visual tokens, and no leakage from noisy prediction tokens (Shou et al., 24 Feb 2026).

The quantitative results in HALO show that this embodied analogue of CoFFT-like reasoning improves robotic performance. On RoboTwin 2.0, the baseline policy pi_0 achieves 46.4% on Easy and 16.3% on Hard, HALO without EM-CoT reaches 75.3% and 21.2%, and HALO with EM-CoT reaches 80.5% and 26.4%, corresponding to gains of +34.1% and +10.1% over the baseline (Shou et al., 24 Feb 2026). Ablations show that removing visual subgoals yields 77.8% and 18.3%, removing textual reasoning yields 76.1% and 22.5%, and full EM-CoT attains 80.5% and 26.4% (Shou et al., 24 Feb 2026). In real-world experiments on four long-horizon tasks, HALO reportedly achieves up to 90% success under basic settings and maintains >65% under novel distractions, outperforming To variants by 20–30% (Shou et al., 24 Feb 2026).

This extension is significant because it reframes CoFFT from a VLM inference method into a more general reasoning principle. A plausible implication is that the original foresight-focus cycle can be interpreted as a template for decomposing sequential decision-making into interpretable intermediate stages, provided those stages remain semantically planned and visually grounded.

7. Position in the research landscape

Within the literature summarized here, CoFFT occupies a specific place among methods for multimodal reasoning under visual clutter. It differs from single-pass inference by making reasoning iterative; it differs from purely language-based search methods by dynamically refining the visual input; and in the embodied setting it differs from monolithic VLA policies by explicitly separating semantic reasoning, visual foresight, and action prediction (Zhang et al., 26 Sep 2025, Shou et al., 24 Feb 2026).

The main strengths attributed to CoFFT are that it excels on complex, cluttered images requiring fine-grained detail, that benefits grow with model size, and that it is training-free in its original VLM formulation (Zhang et al., 26 Sep 2025). The main limitations are the extra computational cost and the possibility of over-focusing (Zhang et al., 26 Sep 2025). HALO addresses a different bottleneck: the supervision gap for embodied multimodal chain-of-thought, which it approaches through automated multimodal annotation at scale and a two-stage training recipe (Shou et al., 24 Feb 2026). The paper argues that cross-expert attention and specialized token routing preserve modality-specific inductive biases and avoid interference seen in monolithic models (Shou et al., 24 Feb 2026).

Future directions named in the CoFFT work include adaptive stopping criteria, learned policies to select $$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$6 and $$\mathcal{S}_t = \{s_{t,1},\,s_{t,2},\dots,s_{t,k}\},\quad s_{t,i} = \mathrm{Decode}\bigl(M;\;V_{t-1},Q,R_{t-1},T_i,l\bigr).$$7 per instance, and integration with lightweight object detectors to guide cropping (Zhang et al., 26 Sep 2025). In the embodied line, the evidence provided suggests a parallel trajectory in which automated multimodal data annotation and staged optimization are used to support long-horizon reasoning and out-of-distribution robustness (Shou et al., 24 Feb 2026).

As a research topic, CoFFT therefore denotes more than a single algorithmic contribution. In its original form, it is an inference-time loop of diverse sample generation, dual foresight decoding, and visual focus adjustment for VLMs (Zhang et al., 26 Sep 2025). In its embodied extension, it becomes a structural principle for decomposing policy learning into “thinking,” “imagination,” and “execution” (Shou et al., 24 Feb 2026). Across both settings, the unifying premise is that multimodal intelligence benefits when future reasoning needs are anticipated before committing either to the next textual step or to the next perceptual or motor act.