Starling-LM-7B-alpha: Vision-Language Reasoning Model

• Starling-LM-7B-alpha is a vision-language reasoning model with 7B parameters that integrates dynamic visual token reinjection through reflection mechanisms.
• It employs Balanced Reflective Policy Optimization (BRPO) to interleave reflective stages, enhancing visual grounding and improving accuracy on complex QA tasks.
• Empirical evaluations highlight that its advanced multimodal design significantly reduces hallucinations while outperforming previous benchmarks.

Starling-LM-7B-alpha is a vision-language reasoning model operating at the 7B parameter scale, designed as an advanced extension of the Qwen2.5-VL-Instruct backbone. The model introduces a policy-learned, reflection-based multimodal reasoning workflow, formalizes the theoretical dynamics of visual attention decay during extended inference, and implements robust mitigation mechanisms that integrate vision-text reflections at both the architectural and algorithmic levels. Starling-LM-7B-alpha aims to improve both reasoning accuracy on complex visual QA tasks and to significantly reduce hallucinations by strictly guiding the model to re-attend to visual cues during generation (Chu et al., 29 May 2025).

1. Theoretical Analysis of Visual Attention Decay

A central issue in inference with Vision-Language Reasoning Models (VLRMs) is the decay of attention to visual tokens as multi-step generation progresses. Visual attention in Starling-LM-7B-alpha is formally quantified via the mutual information $I(y; c \mid x)$ between the generated output tokens $y$ and the initial set of visual tokens $c$, conditioned on the text prompt $x$. Under uniform entropy assumptions across the concatenated input sequence, the attention bound decays according to:

$$I(y; c \mid x) \lesssim \frac{L_c}{L_x + L_c + L_y} H(y \mid x, c)$$

with $L_x$ as the number of text prompt tokens, $L_c$ for visual prompt tokens, and $L_y$ for generated tokens. As $L_y$ increases during reasoning, the fraction $\frac{L_c}{L_x + L_c + L_y}$ decreases, empirically confirming the model's progressively weakened grounding in the initial image [(Chu et al., 29 May 2025), Theorem 3.1]. Reintroduction of $k$ visual tokens at any generation stage raises the upper bound on mutual information:

$$\frac{L_c + k}{L_x + L_c + L_y + k} > \frac{L_c}{L_x + L_c + L_y}$$

demonstrating that explicit reinjection of image features during reflection empirically boosts the model's effective visual grounding [(Chu et al., 29 May 2025), Theorem 3.2].

2. Balanced Reflective Policy Optimization (BRPO)

Starling-LM-7B-alpha employs Balanced Reflective Policy Optimization (BRPO), an RL algorithm built atop the GRPO framework, to learn when and how to trigger vision-text reflection blocks during sequence generation. The base policy $\pi_\theta$ (initialized from Qwen2.5-VL-Instruct) is fine-tuned to interleave output blocks—<SUMMARY>, <CAPTION>, <REASONING>, zero or more <REFLECTION> segments, and a final <CONCLUSION>—with the agent learning autonomously the frequency and length of reflections (Chu et al., 29 May 2025).

BRPO rewards are multi-faceted:

• Format reward $r^{fmt}$ for correct block order and syntax.
• Answer correctness $r^{acc}$ via regular-expression match of <CONCLUSION>.
• Reflection-balance reward penalizing deviation from a target average reflection length $\lambda$ (set to 100 tokens).

Policy optimization includes intra-batch normalization of rewards, clipped surrogate objectives, and a KL penalty toward a reference policy. This incentivizes nuanced reflection behavior and stabilizes training:

$$\mathcal{L}_{\mathrm{BRPO}(\theta)} = \frac{1}{G} \sum_{i=1}^{G} \min\Bigl( \frac{\pi_\theta(o_i | q)}{\pi_{\text{old}}(o_i | q)} A_i, \operatorname{clip}\Bigl(\frac{\pi_\theta(o_i | q)}{\pi_{\text{old}}(o_i | q)}, 1-\epsilon, 1+\epsilon \Bigr) A_i \Bigr) - \beta D_{KL}(\pi_\theta \| \pi_\text{ref})$$

where advantage $A_i$ is computed by group-level reward normalization. The policy gradient is stabilized by clipping and KL penalty, guiding the emergence of multiple, balanced, and sometimes "empty" reflection stages (Chu et al., 29 May 2025).

3. Vision-Text Reflection Process and Visual Token Reinjection

The reflection workflow is implemented by special <REFLECTION> tokens in the autoregressive output stream. Starling-LM-7B-alpha presents no separate gating network; the learned policy directly decides when to signal visual reflection. On emission of <REFLECTION>, the model injects visual tokens via either of two mechanisms:

• Visual Token COPY (VTC): The entire original sequence of visual embeddings $c = [c_1, ..., c_{L_c}]$ is inserted at the reflection point, extending the current context. Subsequent reflection text is decoded with immediate access to all visual cues (Chu et al., 29 May 2025).
• Visual Token ROUTE (VTR): Attention weights $\mathrm{attn}_{i,j}$ (from prior tokens $y_i$ to each visual token $c_j$) are averaged, ranked, and the top $m\%$ of tokens (indices $J$) are selected and injected at the <REFLECTION> marker—enabling a more focused reattending mechanism based on previously attended content.

Both methods enforce legitimate re-grounding in the visual domain and allow the model to flexibly adapt reflection content and context in response to intermediate reasoning.

4. Training and Inference Pipelines

Starling-LM-7B-alpha's training regime integrates multiple procedural stages:

• Cold-start initialization: 2,000 seed examples, each carrying a GPT-4o-generated reflection, are used to prime the policy (bootstrapping via Qwen2.5-VL-Instruct).
• BRPO fine-tuning: Full RL updates over a 10,000-sample mixed multimodal-math corpus, yielding a naturally reflective agent (Qwen-Zero).
• Distillation: Qwen-Zero is used to label 40,000 LLaVA-CoT questions; the set is refined by human and model verification (Qwen-Zero-40K).
• Supervised fine-tuning: SFT is applied with cross-entropy over token sequences, in which either VTC or VTR are force-applied upon <REFLECTION> tokens to co-train the model on visual-token reinjection dynamics.

Training loss per position $t$ is standard cross-entropy:

$$\mathcal{L}_{\mathrm{SFT}} = -\sum_t \log p_\theta\bigl(y_t | y_{<t}, x, c'\bigr)$$

where $c'$ denotes the possibly augmented visual token context.

Inference pipeline:

1. The image is encoded to visual tokens $c$.
2. Text prompt $x$ is supplied.
3. Decoding proceeds autoregressively for tokens $y_t$.
4. Each <REFLECTION> emission triggers immediate reinjection of visual tokens via VTC or VTR.
5. Decoding terminates at </CONCLUSION> or maximum length (Chu et al., 29 May 2025).

5. Empirical Evaluation and Results

Starling-LM-7B-alpha achieves leading performance across a range of visual QA and hallucination benchmarks. For the 7B COPY variant:

• Visual QA accuracy (QA tasks):

| Task | Qwen-LA-COPY | Qwen2.5-VL-Instruct | |-------------|--------------|---------------------| | MMMU | 60.3% | 58.6% | | MMMU-Pro | 41.7% | 41.0% | | MMBench | 82.7% | 82.6% | | MMStar | 65.9% | 63.9% | | MathVision | 26.4% | 25.1% |

• Hallucination metrics (lower is better):

| Metric | Qwen-LA-COPY | Base Model | |-------------|--------------|---------------------| | CHAIR$_i$ | 3.7% | 9.4% | | CHAIR$_s$ | 9.8% | 37.1% | | POPE F1 | 90.2 | 88.7 | | MMHAL BENCH | 3.82 | 3.68 | | MME | 2330.8 | 2309.4 |

The ROUTE variant yields near-identical QA accuracy with slightly higher hallucination metrics but reduced inference overhead, offering a practical tradeoff. These results indicate effective suppression of visual hallucinations and strengthened accuracy on extended multimodal reasoning tasks relevant to the 7B deployment scale (Chu et al., 29 May 2025).

6. Significance, Limitations, and Future Directions

Starling-LM-7B-alpha demonstrates empirically and theoretically that vision-text reflection—mediated by BRPO and enforced via token reinjection—is both necessary and sufficient to arrest the decay of visual attention in extended vision-language reasoning. The flexible, learnable reflection policy ensures the model is not rigidly tied to fixed reflection frequency or length, adapting to diverse input complexities and inference requirements.

A plausible implication is that similar reflection-based grounding mechanisms should be integrated into future large-scale VLRMs to maintain visual fidelity over long generation trajectories. The approach sidesteps explicit contrastive losses and specialized hallucination suppressors, relying instead on policy optimization and context-aware grounding.

Limitations discussed include the computational cost associated with multiple reflections and increased visual-token context, and the open challenge of aligning reflection block length and content with downstream reasoning requirements. Future directions noted are exploring further improvements in modality alignment, chain-of-thought reasoning in visual contexts, and generalized extensions to other backbone architectures and scaling regimes.

In summary, Starling-LM-7B-alpha (Qwen-LookAgain-7B) provides a rigorous blueprint for achieving leading accuracy and hallucination resistance in vision-LLMs through formal attention analysis, policy-learned reflection, and multimodal token reinjection (Chu et al., 29 May 2025).