Qwen3-VL: Advanced Vision-Language Reasoning

• Qwen3-VL is a vision-language model that integrates reflection-driven visual re-attention to enhance multi-step reasoning on complex visual tasks.
• The model employs a staged training pipeline combining supervised fine-tuning and reinforcement learning to optimize output policy and mitigate visual hallucinations.
• It demonstrates state-of-the-art performance by using COPY and ROUTE strategies to maintain visual context and boost accuracy in visual question answering benchmarks.

Qwen-LookAgain (Qwen-LA) is a vision-language reasoning model designed to enhance the fidelity and reliability of autoregressive Vision-LLMs (VLMs) when solving complex visual tasks. Built atop the Qwen2.5-VL-Instruct (7B) multimodal decoder, Qwen-LA introduces reflection-driven visual re-attention mechanisms and reinforcement learning-based output policy optimization, achieving state-of-the-art performance on both visual question answering (QA) and hallucination mitigation benchmarks (Chu et al., 29 May 2025).

1. Architecture and Training Workflow

Qwen-LA employs a staged training and optimization pipeline integrating data distillation, supervised fine-tuning (SFT), and reinforcement learning (RL). The backbone is Qwen2.5-VL-7B-Instruct, a standard multimodal decoder processing a text prompt $x$ of length $L_x$ and an image prompt $c$ mapped to $L_c$ discrete visual tokens by a frozen vision encoder. Autoregressive decoding yields output $y$.

The training workflow proceeds as follows:

1. Cold-Start SFT: The backbone undergoes SFT on 2,000 “cold-start” examples, each containing a single <REFLECTION> block (augmented via GPT-4O and manual validation).
2. Balanced Reflective Policy Optimization (BRPO): BRPO, a tailored RL algorithm, is applied to elicit spontaneous insertion of multiple <REFLECTION>...</REFLECTION> blocks during reasoning, producing the Qwen-Zero policy.
3. Data Distillation and Final SFT: Qwen-Zero generates 40,000 reasoning+reflection examples (all human-verified/corrected), which are used to fully SFT the backbone, activating Visual Re-attention operations.
4. Visual Re-attention at Reflection: During reasoning, encountering a <REFLECTION> token triggers Visual Token COPY (VTC) or Visual Token ROUTE (VTR) operations, forcing the model to re-ingest visual context at the critical reasoning juncture.

Final model variants include Qwen-LA-COPY (VTC; full visual token re-insertion) and Qwen-LA-ROUTE (VTR; top $m\%$ visual tokens by attention routed at reflection).

2. Balanced Reflective Policy Optimization (BRPO)

BRPO is a rule-based RL objective inspired by Generalized Reflective Policy Optimization (GRPO), designed to balance the timing and quantity of vision-text reflections within an autoregressive sequence. For each input $q$:

• Generate $G$ candidate outputs $\{o_i\}$ with the old policy $\pi_{old}$.
• Assign scalar rewards $r_i$ based on:
  1. Format adherence: $+1$ if output follows the required segment structure and order.
  2. Accuracy: $+1$ if the <CONCLUSION> matches ground truth via regex.
  3. Reflection balance: Penalizes deviation from the target number/length of <REFLECTION> blocks.

• Compute intra-group normalized advantages $A_i$.

The clipped policy-gradient update (style of PPO with KL regularization) is given by:

$$\mathcal{L}_{\text{BRPO}}(\theta) = \frac{1}{G}\sum_{i=1}^G \min\left( \frac{\pi_\theta(o_i\mid q)}{\pi_{old}(o_i\mid q)} A_i,\; \mathrm{clip}\left(\frac{\pi_\theta(o_i\mid q)}{\pi_{old}(o_i\mid q)}, 1-\epsilon, 1+\epsilon\right)A_i \right) - \beta \mathrm{KL}(\pi_\theta \| \pi_{ref})$$

This process enables the model to autonomously determine reflection frequency and length, maximizing segment compliance and answer correctness while maintaining informativeness during extended reasoning.

3. Theoretical Analysis of Visual Attention Decay

The model formalizes how mutual information between generated tokens ($y$) and the visual input ($c$) decays as the output sequence grows. Let $L_x$, $L_c$, $L_y$ denote the lengths of the text, visual input, and generated output (excluding reflections), with $L_{total} = L_x + L_c + L_y$.

• Theorem 3.1 (Attention decay):

$I(y; c|x) \leq \frac{L_c}{L_{total}} H(y|x, c)$

As $L_y$ increases, $L_{total}$ grows and $L_c / L_{total}$ drops, leading to diminishing mutual information and weaker conditioning on the original image.

• Theorem 3.2 (Reinsertion boosts attention):

If $k>0$ visual tokens are reinserted before each reflection (i.e., $L_c' = L_c + k$, $L_{total}' = L_{total} + k$), the c-information ratio increases:

$\frac{L_c'}{L_{total}'} > \frac{L_c}{L_{total}}$

This elevates the theoretical upper bound on visual attention, motivating explicit re-introduction of visual tokens during extended reasoning.

4. Visual Re-attention Operations

Re-attention is operationalized through two deterministic token routing strategies activated at the <REFLECTION> pivot:

• Visual Token COPY (VTC): All $L_c$ visual embeddings are prepended to the decoder’s input prior to generating the reflection segment. This approach doubles the visual representation at each reflection, substantially raising the image-to-text token ratio.
• Visual Token ROUTE (VTR): For the current prefix, average attention weights are computed per visual token ($$\text{attn}_j = \frac{1}{L_y} \sum_{i=1}^{L_y} \text{attn}_{i,j}$$). The top $m\%$ of tokens by weight are identified, and only these are prepended before the reflection. This strategy trades off re-attention intensity for computational and inference efficiency.

Both strategies directly address the limitations of text-only reflection by ensuring actual visual content is refreshed in the latent state at each reflective reasoning turn.

5. Empirical Results on Visual QA and Hallucination

Qwen-LA sets new benchmarks on both accuracy and hallucination metrics across several visual QA datasets and hallucination tests, as shown below.

Visual QA Accuracy:

Model	MMMU	MMMU-Pro	MMBench	MMStar	MathVision
Qwen2.5-VL-Instruct (7B)	58.6	41.0	82.6	63.9	25.1
Llama-3.2-11B-Vision-Instr.	50.7	33.0	64.9	46.6	12.4
LLaVA-CoT (11B)	51.2	31.7	73.8	57.8	15.6
Vision-R1 (7B)	56.2	36.1	81.5	61.4	25.5
Qwen-LA-COPY (ours)	60.3	41.7	82.7	65.9	26.4
Qwen-LA-ROUTE (ours)	59.1	41.3	82.8	64.6	25.8

Hallucination Benchmarking:

Method	CHAIR_i↓	CHAIR_s↓	POPE↑	MMHAL↑	MME↑
Qwen2.5-VL-Instr.	9.4	37.1	88.7	3.68	2309.4
Qwen-LA-COPY (ours)	3.7	9.8	90.2	3.82	2330.8
Qwen-LA-ROUTE (ours)	5.6	11.2	88.5	3.73	2322.6

Qwen-LA-COPY achieves the lowest hallucination rates (as measured by CHAIR metrics) and highest QA accuracy, with an inference cost of approximately twice the generation time compared to baselines. Qwen-LA-ROUTE offers a computability–performance trade-off by selectively routing a subset of visual tokens ($m=50\%$) during reflection.

6. Implications and Comparative Context

Qwen-LA demonstrates that purely textual reflection, though beneficial for standard LMs, is insufficient for mitigating hallucinations in VLM reasoning. The explicit vision-text reflection process—by reinserting image-derived tokens at key reasoning junctures—substantially curbs hallucinations and maintains high visual attention.

A plausible implication is that future VLMs and VLRMs seeking accurate long-form vision-language reasoning should incorporate structured visual re-attention, particularly when task demands require extended multi-step reasoning. The reinforcement learning-driven approach to structuring reflection (via BRPO) further highlights the synergy between RL and structured autoregressive modeling in controlling model introspection and visual grounding (Chu et al., 29 May 2025).