No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers (2512.08889v1)

Abstract: Visual reasoning is challenging, requiring both precise object grounding and understanding complex spatial relationships. Existing methods fall into two camps: language-only chain-of-thought approaches, which demand large-scale (image, query, answer) supervision, and program-synthesis approaches which use pre-trained models and avoid training, but suffer from flawed logic and erroneous grounding. We propose an annotation-free training framework that improves both reasoning and grounding. Our framework uses AI-powered verifiers: an LLM verifier refines LLM reasoning via reinforcement learning, while a VLM verifier strengthens visual grounding through automated hard-negative mining, eliminating the need for ground truth labels. This design combines the strengths of modern AI systems: advanced language-only reasoning models for decomposing spatial queries into simpler subtasks, and strong vision specialist models improved via performant VLM critics. We evaluate our approach across diverse spatial reasoning tasks, and show that our method improves visual reasoning and surpasses open-source and proprietary models, while with our improved visual grounding model we further outperform recent text-only visual reasoning methods. Project webpage: https://glab-caltech.github.io/valor/

• The paper introduces VALOR, a framework that uses multimodal verifiers to train visual reasoning models without ground-truth annotations.
• It employs reinforcement learning for LLM-based program synthesis and VLM pseudo-labeling, leading to significant accuracy gains on diverse benchmarks.
• The approach enhances spatial grounding and robustness, outperforming traditional supervised and unsupervised methods in complex visual reasoning tasks.

Annotation-Free Training of Visual Reasoners via Multimodal Verifiers

Motivation and Problem Setting

Visual reasoning requires both high-fidelity visual grounding and sophisticated multi-step programmatic reasoning across object attributes and spatial relations. Traditional approaches either employ chain-of-thought LLMs that demand extensive (image, query, answer) supervision or program synthesis methods that utilize pre-trained vision specialists with fixed logic. The former is data-hungry and prone to errors in visual understanding; the latter suffers from misaligned spatial reasoning due to improper grounding. This work introduces VALOR—a scalable, annotation-free training paradigm leveraging multimodal verifiers to systematically tune both logic (LLM) and visual grounding (VLM/vision specialists), circumventing the need for ground-truth labels.

Figure 1: VALOR enables accurate reasoning and grounding by decomposing visual reasoning tasks using multimodal verifiers, eliminating ground truth supervision.

VALOR Framework: Architecture and Training Protocol

VALOR unifies two synergistic learning signals: an LLM verifier for logic/program synthesis via RL and a VLM verifier for vision specialist grounding via SFT. The architecture features a language-only Qwen3-8B LLM generating plans and Python programs using a fixed vision specialist API (object detection, depth estimation, VQA). The training loop employs high-capacity verifiers—Gemini-2.5-Flash for LLM and GPT-5-mini for VLM evaluation—to critique and reward program outputs at fine-grained levels.

Figure 2: VALOR system overview; LLM verifiers drive RL training for program synthesis, while VLM verifiers supervise vision grounding models through pseudo-labeling and SFT.

LLM Program Synthesis and Reward Modeling

VALOR’s RL phase uses the Qwen3-8B LLM to infer a plan and executable Python code for spatial queries, leveraging an API with functions for object detection (gd_detect), depth estimation (depth), and attribute extraction (vqa). The LLM output is evaluated by a composite binary reward model covering format, syntax, logic, attribute, spatial, and adherence criteria. Deterministic interpreters assess format and syntax, while frozen LLM verifiers (Gemini-2.5-Flash) judge semantic aspects. The RL objective employs Group Relative Policy Optimization (GRPO), maximizing expected advantage while regularizing policy drift.

Figure 3: (a) LLM verifiers reward outputs for semantic correctness across logic, attribute, spatial, and code adherence domains; (b) VLM verifiers filter detector overpredictions in staged pseudo-labeling for grounding model improvement.

Vision Specialist Tuning via VLM Verification

The grounding module—GroundingDINO—is fine-tuned via VLM-based pseudo-labeling. Object predictions from a detector with relaxed thresholds are subject to three-stage VLM verification: coarse filtering of objects, per-crop semantic validation, and deduplication of overlapping instances. The final high-precision pseudo-labels are recycled as training data. This process obviates the need for manually annotated bounding boxes and scales to large image distributions. Empirical analysis indicates a $75\%$ precision in final pseudo-labels, with the deduplication stage yielding the most substantial improvements.

Experimental Evaluation and Analysis

The efficacy of VALOR is demonstrated across a spectrum of visual reasoning benchmarks (Omni3D-Bench, RoboSpatial, VSR, TallyQA, GQA, RealWorldQA, BLINK, CountBenchQA), assessing both programmatic reasoning and grounded perception. Key findings include:

• Verifier-Improved Reasoning: RL-tuned VALOR-RL yields performance boosts on challenging benchmarks—$+6.4\%$ accuracy on Omni3D-Bench versus vanilla Qwen3-8B.
• Verifier-Improved Grounding: Enhanced visual grounding via VLM pseudo-labeling confers cumulative gains, e.g., $+8.3\%$ on CountBenchQA, $+7.7\%$ on RoboSpatial, compared to the detector’s original version.
• Comparison with Competing Methods: VALOR surpasses RL-tuned VLMs (GRIT, ViGoRL) trained with labeled data, annotation-free program synthesis baselines (VisProg, ViperGPT, VADAR), and direct-answer proprietary/open-source VLMs (e.g., GPT-4o, Gemini-2.5-Flash, GPT-5-mini) on reasoning-heavy tasks.

Figure 4: Qualitative comparison of reasoning and outputs for VALOR, LLMs with tool-use, and RL-tuned VLMs across complex spatial queries.

Grounding Analysis and Ablations

The study further contrasts visual grounding failure modes and corrections. GRIT and ViGoRL, which generate box coordinates through text, frequently omit relevant objects or misalign bounding predictions. VALOR, by explicit tool invocation and verifier-driven adjustment, robustly identifies and grounds salient instances required for spatial inference.

Figure 5: Left: Grounding accuracy contrast between VALOR, GRIT, ViGoRL. Right: Verifier-based tuning of visual grounding corrects overprediction errors in VALOR-RL baseline.

VALOR’s data scalability was investigated, revealing monotonic accuracy improvements for both reasoning and grounding as synthetic training sample cardinality increases. Ablation with supervised fine-tuning (SFT) demonstrates that RL remains superior for program-fusion tasks demanding sparse, high-quality feedback.

Implications, Limitations, and Prospective Directions

VALOR’s pseudo-supervised protocol addresses fundamental limitations in large-scale vision-language alignment: label scarcity, domain adaptation, and systemic error propagation in compositional reasoning. By marshalling verifiers as primary learning signals, both logic and perception modules become more robust in 2D/3D scenarios. Failures on hypothetical queries and crowded object arrangements indicate remaining challenges at the intersection of spatial compositionality and robust grounding. Integration of VLM verifiers into the RL loop (rather than post-hoc pseudo-labeling) and targeted harvesting of hard negatives promise further improvement.

Future efforts should investigate compatibility of strong VLMs (e.g., Gemini-2.0-Flash) as grounding modules, direct joint RL for reasoning and grounding, and systematic query generation strategies for better domain coverage. Finally, the distinction between verifier reliability and generator capacity—elucidated by the inability of powerful VLMs to produce accurate groundings yet serving effectively as critics—should inform LLM/VLM architecture co-development.

Conclusion

VALOR constitutes a principled, annotation-free framework for training visual reasoners by leveraging high-capacity multimodal verifiers. Empirical evidence across demanding benchmarks substantiates its superiority over both supervised and unsupervised traditional approaches in visual program synthesis and grounding. The paradigm offers a scalable, label-agnostic alternative for spatial reasoning tasks and sets the stage for fully verifier-centric multimodal AI systems.