Improving Reasoning in Vision-Language Models via Perception Verified Self-Training

Abstract: Achieving human-like reasoning in Vision-LLMs (VLMs) remains a long-standing challenge. Recent approaches leverage Chain-of-Thought (CoT) rationales generated by human annotators or proprietary models to improve reasoning, which is costly and difficult to scale. Self-training offers a promising alternative by using models own outputs as supervision. However, existing methods often suffer from visual hallucinations -- where rationales describe non-existent visual content, and language shortcuts -- where predictions rely on textual priors rather than true visual grounding, as rationales are typically filtered only by answer correctness without verifying visual perception. To address this limitation, we propose a perception-verified self-training framework that enforces visually grounded reasoning. First, our method employs a CoT template (caption-reasoning-conclusion) that disentangles perception from reasoning, enabling independent verification of visual understanding. To compensate for the absence of ground-truth captions, we propose PerceptEval, an unsupervised method that evaluates caption quality based on its alignment with visual and textual elements present in the image. Using caption verification together with answer correctness, we partition the data into three subsets: easy (correct caption and conclusion), medium (correct caption but incorrect conclusion), and hard (incorrect caption). Building on this partitioning, we design a two-stage curriculum learning strategy. In Stage 1, the model is trained on easy examples and subsequently in Stage 2, medium samples are incorporated through a caption-guided reasoning enhancement procedure that regenerates reasoning conditioned on verified captions. Only regenerated samples with the correct conclusions are retained.

• The paper introduces a novel two-stage self-training framework that disentangles perception from reasoning using an unsupervised PerceptEval module.
• It leverages OCR and FG-CLIP metrics for caption validation, effectively reducing visual hallucinations and reliance on language shortcuts.
• Empirical results demonstrate up to 16% improvement in test accuracy and robust performance across diverse multimodal domains.

Improving Reasoning in Vision-LLMs via Perception Verified Self-Training

---

Introduction

The challenge of achieving robust, human-like reasoning in Vision-LLMs (VLMs) is accentuated by the prevalence of visual hallucinations and reliance on language shortcuts due to insufficient supervision on intermediate perceptual components of multimodal data. While chain-of-thought (CoT) rationales have improved reasoning in LLMs, scaling these techniques to VLMs for multimodal tasks faces significant obstacles, primarily stemming from the cost and complexity of collecting high-quality, visually-grounded rationales. The paper "Improving Reasoning in Vision-LLMs via Perception Verified Self-Training" (2606.22158) addresses these issues by proposing a structured self-training pipeline that leverages unsupervised verification of perception to jointly enhance visual grounding and reasoning.

---

Framework Overview and Methodology

The proposed framework operates in two principal stages, utilizing a novel Caption-Reasoning-Conclusion template. This disentangles the perceptual component from the reasoning process, enabling independent verification and iterative self-improvement. The central innovation is PerceptEval, an unsupervised module for caption quality assessment, which combines OCR-text alignment and FG-CLIP-based visual-similarity metrics to ensure high fidelity in perception.

Figure 1: Overview of the self-training pipeline. Stage-1 uses easy cases for fine-tuning; Stage-2 incorporates medium-difficulty cases after reasoning enhancement guided by verified captions.

To further maximize learning signal quality, data is partitioned as follows:

• Easy cases: Both caption and conclusion are correct.
• Medium cases: Caption correct, conclusion incorrect.
• Hard cases: Caption incorrect.

The two-stage curriculum learning scheme fine-tunes the VLM first on easy samples to stabilize the perceptual and reasoning pathways, then introduces medium cases via a caption-embedded prompting mechanism, which regenerates reasoning grounded on previously verified captions. Only samples producing correct conclusions are retained, ensuring all supervision is perceptually grounded.

---

Caption Validation: PerceptEval

PerceptEval robustly validates generated captions by leveraging both textual content extracted via OCR and visual similarity:

• OCR text agreement: Cosine similarity between Sentence-Transformer embeddings of model-generated captions and OCR-derived text summaries.
• FG-CLIP visual alignment: Visual-language similarity between the image and generated caption focusing on object and scene recognition.
• Domain-adaptive thresholding: Text area coverage ratio adjusts the weighting between OCR and FG-CLIP similarity, allowing robust filtering in both text-heavy and visually rich settings.

This dual filtering mechanism dramatically reduces the incidence of hallucinated and incomplete perceptual inputs, restricting training to high-quality, visually grounded rationales.

---

Quantitative and Qualitative Results

Extensive experiments across multiple domains (commonsense, natural science, language science, social science) and VLM architectures (LLaVA-v1.5, Qwen2-VL) demonstrate that perception-verified self-training yields significant gains—up to 16% improvement over prior self-training baselines such as STaR and R3V. The results are robust across both small and large backbone models and transfer well to out-of-distribution settings via evaluation on benchmarks like MMMU.

• Numerical results: The method achieves test accuracies of 64.93% (Language Science) and 72.74% (Commonsense) on LLaVA-v1.5-7B, outperforming both STaR and R3V.
• Token efficiency: Rationales generated are concise yet detailed (69–93 tokens), optimizing explicit reasoning without excessive verbosity.

Figure 2: Human subjective evaluation: rationales produced by the proposed method are preferred overall, indicating improved logical and visual coherence.

Qualitative analyses reveal substantial reductions in both visual hallucinations and language shortcut exploitation. Baselines routinely propagate perceptual errors in reasoning, especially when answer filtering alone is used; in contrast, the perception-verified approach grounds reasoning in both visual and textual evidence.

Figure 3: Validation accuracy progression illustrates the efficacy of curriculum learning. Stage 2 (dark blue) shows sharp improvement after the staged introduction of medium samples.

---

Scalability, Data Efficiency, and Ablations

The selective curriculum enables superior performance with reduced reliance on large quantities of training data, highlighting the importance of quality over quantity in rationale selection.

Figure 4: Test accuracy versus the proportion of training samples utilized. The proposed method attains higher accuracy with consistently less data than direct SFT or other self-training baselines.

Ablation studies confirm:

• Caption filtering via PerceptEval provides critical feedback for perceptual accuracy.
• Gradual introduction of medium samples (via curriculum learning) avoids early saturation or instability.
• FG-CLIP and OCR-based filtering are synergistic, especially in domains with heterogeneous image content.

---

Implications and Future Directions

This framework demonstrates that disentangling perception from reasoning, combined with rigorous perceptual verification and curriculum learning, is a necessary paradigm shift in VLM self-training. Practically, this approach reduces dependence on expensive, manually annotated multimodal rationales, enabling scalable, cost-efficient improvement of VLMs. Theoretically, it establishes that intermediate verification mechanisms are required for robust visual grounding in reasoning-centric tasks.

The method is extensible to larger models and diverse domains. Future work should address:

• Integration with advanced RL-based self-training algorithms (e.g., DPO, MCTS-based optimizations).
• Adaptive rationale generation (reason-only-if-necessary) for further inference efficiency.
• Leveraging reliable components of hard cases for improved robustness.

---

Conclusion

Separation and independent verification of perception and reasoning in VLM self-training fundamentally enhance multimodal reasoning capabilities. PerceptEval delivers unsupervised caption quality control, and curriculum learning stabilizes iterative improvement. The approach consistently outperforms established baselines and is both scalable and cost-efficient, suggesting a clear path forward for advancing VLM reasoning via perceptually grounded self-training (2606.22158).