Vero-600K: RL Dataset & GBDT Benchmark

• Vero-600K is a comprehensive dataset featuring 600K curated samples for RL post-training of vision-language models across six distinct tasks.
• It employs a task-routed reward mechanism with precise formatting rules and overlong penalties, leading to significant performance improvements.
• In GBDT systems, Vero-600K benchmarks demonstrate 1 s/tree processing speed and superior scalability for high-dimensional, large-instance tabular workloads.

Vero-600K denotes two distinct entities in contemporary machine learning research: (1) a purpose-built, task-balanced reinforcement learning dataset for post-training vision-LLMs, and (2) the approximate scale at which the Vero distributed GBDT system is empirically benchmarked on high-dimensional, large-instance tabular data workloads. Both meanings reflect advances in dataset curation and efficient large-scale learning architectures, sharing an emphasis on scalability and reproducibility in open research contexts.

1. Definition and Scope

Vero-600K, in its primary usage, refers to a 600,000-sample dataset for reinforcement learning (RL)–based post-training of vision-LLMs (VLMs), curated from 59 high-quality source datasets, and partitioned into six semantically distinct visual reasoning tasks. In distributed gradient boosting decision tree (GBDT) literature, “600K” specifies the instance count for large-scale empirical benchmarking, notably the RCV1 dataset (N ≈ 697,000). Across both senses, Vero-600K signifies rigorous, large-scale experimental settings central to open multimodal AI and scalable tabular machine learning (Sarch et al., 6 Apr 2026, Fu et al., 2019).

2. Dataset Construction and Taxonomy

Vero-600K is assembled to maximize breadth and balance for RL fine-tuning of VLMs. Its construction follows a strictly-defined taxonomy split into six categories, with 100,000 samples per category:

Category	Example Sources	# Sources
Chart OCR & Table Reasoning	ChartQA, InfographicVQA, ArxivQA, CoSyn-Chart	9
STEM & Scientific Figures	MathVision, MathVista, AI2D, Geo170K, RAVEN	13
Spatial Action & Embodied Reasoning	ST-VQA, GameQA, Robo2VLM, VisualJigsaw	8
Knowledge Recognition & Commonsense VQA	GQA, FVQA, VQAv2, A-OKVQA	12
Grounding, Counting & Search	AerialVG, CountBenchQA, ScreenSpotPro	11
Captioning & Instruction Following	pixmo-cap, MM-IFEval, Flickr30K-Entities	6

The dataset undergoes layered filtering: heuristics (removing datasets with <1,000 samples or low image resolution), manual quality control (<5% annotation error), model-based prompt screening (irrelevance, ambiguity, unsupported numerics), and answer normalization (removing units, dropping multi-value or ambiguous answers). Uniform sampling across categories at training time (“equal ratio”) outperforms sampling based on difficulty, length, or image area (Sarch et al., 6 Apr 2026).

3. Reward Design and RL Formulation

Vero-600K is paired with a highly structured, task-routed reward design to address the heterogeneity of answer formats encountered in open visual reasoning. For each image–question pair $(v, q)$, the RL rollout produces output $y = (z, a) \sim \pi_\theta$. The total reward function is: $$R(y, y^*) = (1-\alpha) R_{\text{acc}}(y, y^*) + \alpha R_{\text{fmt}}(y) + R_{\text{overlong}}(y), \quad \text{with } \alpha = 0.2$$

• Accuracy reward $R_{\text{acc}}$ is realized via category-specific verifiers: exact string match, multiple choice extraction, math-verified numerics, set/list checks, validated ordering, web action field match, grounding IoU/F1, click evaluations, programmatic instruction-following constraint satisfaction, and LLM-as-judge scoring (Qwen3-32B).
• Format reward $R_{\text{fmt}}$ incentivizes output structure: chain-of-thought in > ..., answers in <answer>...</answer>, and a single \boxed{…} for discrete responses (full, partial, or zero credit for formatting errors).
• Overlong penalty $R_{\text{overlong}}$ imposes a linear penalty for outputs exceeding a buffer $B$ tokens from the model’s context window, promoting concise completion.

This reward routing outperforms monolithic verifier baselines by +5.4 points on aggregate. The approach enables consistent, on-policy evaluation across tasks with disparate output structures (Sarch et al., 6 Apr 2026).

4. RL Training Protocol, Scaling, and Empirical Findings

Vero-600K is used to train four 7–8B parameter VLMs via RL (Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, Qwen3-VL-8B-Thinking, MiMo-VL-7B-SFT) using Group Sequence PPO (GSPO) with asymmetric clipping, batch size 256, micro-batch 128, learning rate $1 \times 10^{-6}$, and 2,000 steps. RL training yields substantially greater gains than supervised fine-tuning (+4.8 percentage points vs +0.4) with continued performance scaling across the full data pass. Among alternative RL and data-mixing strategies, GSPO with task-routed rewards and uniform data mixture delivers the most stable and highest mean scores.

On VeroEval (a suite of 30 benchmarks), Vero-600K RL yields average improvements of +5.5 pp (Qwen3-VL-8B-Instruct base), +8.5 pp for Chart OCR, +6.4 pp STEM, +8.6 pp Grounding & Counting, and consistent superiority over prior open RL recipes by 3–15 pp across competencies (Sarch et al., 6 Apr 2026).

5. Application in Distributed Tabular GBDT Workloads

In the context of GBDT systems, “Vero-600K” refers to a vertical-partitioned, row-store data management strategy optimized for workloads such as RCV1 (N ≈ 697,000 instances, D ≈ 47,000 features):

• Data Strategy: The feature matrix is partitioned by columns across W workers; each worker stores all N samples and D/W features in Compressed Sparse Row (CSR) format, indexed per tree node.
• Efficiency: Histogram memory per worker scales as $$(\frac{D}{W}) \times q \times C \times 8 \times 2 \times 2^{L-2}$$, minimizing memory for large D. Communication per layer is a bitmap of $\lceil N/8\rceil$ bits, invariant to D and C.
• Empirical Results (RCV1, 5 workers, L=8, q=20, T=100): Vero achieves 1 s/tree (0.9 s computation + 0.1 s communication), outperforming XGBoost (17× faster), LightGBM (5.6× faster), and alternatives. Per-worker RAM stays below 1 GB (Fu et al., 2019).

This approach is most advantageous for high-dimensional, modest-bandwidth, or memory-constrained environments, especially when D ≫ n, tree depth or class count is large, or parallelism is bounded by $y = (z, a) \sim \pi_\theta$0 histogram and $y = (z, a) \sim \pi_\theta$1 index updates per layer.

6. Best Practices, Limitations, and Reuse Guidelines

Research employing Vero-600K emphasizes:

• Broad and balanced data coverage as the primary factor in generalizable visual reasoning. Uniform mixtures mitigate negative transfer between tasks.
• Task-routed reward mechanisms as essential for heterogeneous output domains; single-verifier regimes generate excessive reward noise.
• Inclusion of open-ended instruction-following is required to retain fluent visual chat modalities and avoid collapse to highly terse outputs.
• Reproducibility is achieved through public release of all data, filtering and reward scripts, and trained models, supporting further experimentation and benchmarking in multimodal RL and chain-of-thought research.

In large-scale GBDT, vertical-partitioned row-store should be preferred in high-D workloads; horizontal partitioning is favored when D is low and n is extremely large, with column-store avoided for most tabular applications.

7. Impact and Future Directions

Vero-600K represents a modular foundation for post-training VLMs using RL in diverse domains, providing both a task-diverse, quality-controlled dataset and an empirical validation of scalable, balanced data mixtures with task-aware rewards. In GBDT systems, it supports efficient, high-throughput distributed learning at hundreds of thousands of instances and feature dimensions. Its release is poised to influence further research on multimodal RL, curriculum design, data mixing strategies, and distributed machine learning system architectures (Sarch et al., 6 Apr 2026, Fu et al., 2019).