STGR-RL-36k: Reinforcement Learning Video Reasoning

• STGR-RL-36k is a large-scale reinforcement learning corpus combining video QA with explicit spatio-temporal annotations for evidence localization.
• It integrates key timestamps, object bounding boxes, and structured reasoning traces to enable reward-based optimization in video models.
• Empirical results show that using STGR-RL-36k boosts spatio-temporal alignment and model accuracy compared to pure supervised fine-tuning.

STGR-RL-36k denotes a large-scale reinforcement learning (RL) training corpus for grounded video reasoning, introduced in "Open-o3 Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence" (Meng et al., 23 Oct 2025). It is designed to enable and benchmark models that generate not only textual answers about video content, but also explicit spatio-temporal evidence—localizing when (timestamps) and where (object bounding boxes) supporting visual cues appear within dynamic scenes.

1. Purpose and Motivation

Traditional video reasoning corpora focus on either generic question–answer pairs (e.g., VideoQA benchmarks), temporal localization (temporal grounding datasets), or static spatial localization (frame/image-level datasets with bounding boxes). However, these are fundamentally incomplete for RL training aimed at evidence-grounded reasoning: existing datasets seldom couple temporal and spatial evidence or provide reasoning traces that explicitly bind answer steps to time and region.

STGR-RL-36k is curated to fill this void. It contains examples where answers, key timestamps, bounding box locations, and structured, annotation-aligned reasoning traces are jointly supervised to enable reward-based learning that directly targets answer accuracy, evidence alignment in both temporal and spatial domains, and rigorous output format correctness.

2. Dataset Construction and Composition

STGR-RL-36k is constructed as a composite of multiple data sources with both inherited and newly annotated samples. The total size is approximately 36,100 instances, precisely quantified as:

Type	Source(s)	Size (k)
Temporal Grounding	Time-R1, TVG-RL	5.2
Spatial Grounding	VisCoT	5.0
Spatio-Temporal	Gemini 2.5 Pro pipeline, VideoEspresso	10.9
General VideoQA	Video-R1	15.0

Annotations in each instance include (as applicable): question–answer pairs, timestamped keyframes, 1–3 prominent object bounding boxes per annotated frame, and a step-wise reasoning trace using a strict markup:

<obj>object_name</obj><box>[x_min, y_min, x_max, y_max]</box>at<t>timestamp</t>s

The full category distribution per the dataset’s own analysis is 14.4% temporal, 13.9% spatial, 30.3% spatio-temporal, and 41.7% general QA.

Data sources are carefully filtered and post-processed. For example, bounding box crops are auto-verified by a strong vision-LLM (Qwen2.5-VL-7B) to remove semantic outliers, and both annotation and reasoning chains are consistency-checked to guarantee every object/timestamp/box mention is matched.

3. Role in RL Training and Reward Computation

STGR-RL-36k is employed as the core RL dataset in a two-stage training regime:

1. Cold-start: Model is initialized on STGR-CoT-30k (demonstration data) via supervised fine-tuning (SFT).
2. RL stage: Model undergoes RL using STGR-RL-36k, with scalar rewards jointly reflecting answer correctness, temporal alignment (using intervals or timestamp-point supervision), spatial localization (spatial reward gated by timestamp proximity), and output format validity.

The RL optimization uses Group Sequence Policy Optimization (GSPO). For a query-completion pair $(x, y)$, the total scalar reward combines accuracy, "thinking" (evidence localization), and format components. The thinking reward is only computable due to the spatio-temporal synchronization in STGR-RL-36k, as the reward function parses the answer and reasoning trace for explicit $<$obj$>$, $<$box$>$, $<$t$>$ markup.

4. Annotation Schema and Evidence Structure

The annotation protocol is distinguished by:

• Explicit keyframes (1–5 per exemplar)
• 1–3 salient object boxes per keyframe (each $[x_{\min}, y_{\min}, x_{\max}, y_{\max}]$)
• Timestamps in seconds, referenced both in keyframes and within reasoning
• Structured reasoning process, where each evidence mention is strictly formatted
• For video sources without time-aligned region annotation, frame indices are back-converted to absolute timestamps

The dataset sources span both curated resources and automated annotation via Gemini 2.5 Pro + PLM-Rdcap, followed by robust filtering, semantic verification, and consistency pruning.

5. Empirical Impact and Ablation Results

Multi-stage ablation studies demonstrate that RL using STGR-RL-36k both surpasses supervised fine-tuning alone and enables precise spatio-temporal alignment unattainable by models trained only on text-only or single-modality data. Notable results (on V-STAR):

• Pure SFT mAM: 28.5, Pure RL (GSPO): 30.4, SFT+RL (GSPO): 33.7
• Full Open-o3 Video (with STGR-RL-36k spatio-temporal data): 33.7 mAM, 46.6 mLGM; w/o spatio-temporal: 28.3 mAM, 36.2 mLGM

Including high-quality spatio-temporal examples from STGR-RL-36k yields substantial performance improvements versus conventional QA or unsynchronized data sources, especially in metrics assessing evidence localization.

6. Limitations and Dataset Scope

STGR-RL-36k is primarily a training corpus, not a standalone evaluation benchmark. It lacks public test splits, inter-annotator agreement metrics, and licensing granularity per external source. Annotation quality, while high, may still exhibit errors; the authors describe mitigation but do not claim perfection. The dataset is relatively sparse in examples featuring long videos, complex scenes, or small objects—a recognized limitation for broader deployment.

There is no audio or multimodal supervision. Complex reasoning involving multi-hop inference beyond direct evidence localization is only partially addressed. The dataset is specifically constructed for the RL reward structure of Open-o3 Video and similar architectures.

7. Relationship to Prior Work and Distinctives

STGR-RL-36k is novel in unifying answer, spatial, temporal, and reasoning-trace supervision for video reasoning models. Prior datasets typically supply only isolated supervision types, limiting their utility for RL workflows requiring reward verifiability across evidence modalities. STGR-CoT-30k serves the SFT phase and is similar in concept but smaller and less spatio-temporal-heavy; STGR-RL-36k is indispensable for reward-driven optimization with tightly synchronized evidence.

No variant of "STGR-RL-36k" is documented as an alias for existing video tracking/pose datasets such as APT-36K (Yang et al., 2022), nor is it related to spatiotemporal Gaussian representation methods (STGR) for medical imaging (Xie et al., 28 Mar 2025), which do not refer to this dataset or reinforcement learning at 36k scale.

---

STGR-RL-36k is the principal RL corpus enabling explicit, spatio-temporally grounded video reasoning in the Open-o3 Video pipeline, supplying the structured evidence required for both accurate answering and synchronized, verifiable grounding supervision (Meng et al., 23 Oct 2025).