VideoSSR: Video Self-Supervised Reinforcement Learning (2511.06281v1)

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has substantially advanced the video understanding capabilities of Multimodal LLMs (MLLMs). However, the rapid progress of MLLMs is outpacing the complexity of existing video datasets, while the manual annotation of new, high-quality data remains prohibitively expensive. This work investigates a pivotal question: Can the rich, intrinsic information within videos be harnessed to self-generate high-quality, verifiable training data? To investigate this, we introduce three self-supervised pretext tasks: Anomaly Grounding, Object Counting, and Temporal Jigsaw. We construct the Video Intrinsic Understanding Benchmark (VIUBench) to validate their difficulty, revealing that current state-of-the-art MLLMs struggle significantly on these tasks. Building upon these pretext tasks, we develop the VideoSSR-30K dataset and propose VideoSSR, a novel video self-supervised reinforcement learning framework for RLVR. Extensive experiments across 17 benchmarks, spanning four major video domains (General Video QA, Long Video QA, Temporal Grounding, and Complex Reasoning), demonstrate that VideoSSR consistently enhances model performance, yielding an average improvement of over 5\%. These results establish VideoSSR as a potent foundational framework for developing more advanced video understanding in MLLMs. The code is available at https://github.com/lcqysl/VideoSSR.

• The paper introduces a self-supervised reinforcement learning framework that leverages three video pretext tasks to generate scalable, annotation-free training data.
• It employs adjustable task difficulty and tailored reward shaping to overcome sparse reward signals, significantly improving video reasoning performance.
• The approach replaces manual annotation with intrinsic video signals, enabling robust, bias-free evaluation and better generalizability across multimodal benchmarks.

VideoSSR: Advancing Multimodal LLMs via Video Self-Supervised Reinforcement Learning

Motivation and Background

With the proliferation of Multimodal LLMs (MLLMs), video understanding has seen rapid progress, largely driven by Reinforcement Learning with Verifiable Rewards (RLVR). Modern datasets such as LongVideoReason and ReWatch enable RLVR for video reasoning by providing verifiable, synthetic annotations. However, as MLLMs become more capable, these datasets increasingly fail to present sufficient challenge or reliable reward signals—evidenced by their highly bimodal outcome distributions, especially for advanced models like Qwen3-VL, leading to vanishing gradient issues during RL optimization.

Figure 1: Distribution of answer correctness on ReWatch and LongVideoReason, revealing the dominance of zero and maximum scores for most questions, particularly for Qwen3-VL.

Manual annotation at scale is prohibitively expensive and often introduces biases, particularly when annotator models are less capable than the targets. This raises the question: can intrinsic video signals themselves be leveraged for scalable, challenging, and annotation-free RLVR training in MLLMs? "VideoSSR: Video Self-Supervised Reinforcement Learning" (2511.06281) proposes a comprehensive framework addressing this critical challenge.

Methodology: Self-Supervised Video Pretext Tasks

VideoSSR introduces three self-supervised video pretext tasks—Anomaly Grounding, Object Counting, and Temporal Jigsaw— designed with parametrically adjustable difficulty, thus producing verifiable question-answer pairs directly from raw videos with no dependence on external annotations.

Figure 2: Overview of the three self-supervised pretext tasks: (a) Anomaly Grounding, (b) Object Counting, and (c) Temporal Jigsaw.

• Anomaly Grounding: A random temporal segment in a video is perturbed (e.g., via channel swap, rotation, mirroring, zoom, shuffling, or other fine-grained/spatial/temporal transformations), and the task is accurate localization of the anomalous interval.
• Object Counting: Primitive geometric shapes (circles, squares, triangles), procedurally generated, are superimposed onto randomly selected frames; the model must count instances of each shape across frames.
• Temporal Jigsaw: The video is split into $n$ equal segments and shuffled; the model's task is to reconstruct the original segment order, directly assessing its temporal reasoning capacity.

Each task is crafted for adjustable hardness (e.g., more shapes, more segments), enabling flexible curriculum and ongoing benchmark relevance as models improve.

Figure 3: An example of Object Counting with ground truth (circles, squares, triangles) = 3, 2, 3.

Figure 4: Temporal Jigsaw example: shuffled sequence (ground truth order: 452316).

VIUBench: Diagnosing Intrinsic Video Understanding Bottlenecks

The Video Intrinsic Understanding Bench (VIUBench) is introduced to assess fine-grained, spatial, and temporal perception in SOTA closed- and open-source models via the above pretext tasks. VIUBench's results reveal consistent, substantial performance gaps, even for advanced closed models like GPT-5 (average score 58.7) and severe underperformance for open models (Qwen3-VL-8B: 19.5).

More importantly, the difficulty and discriminatory power of these tasks is inherently scalable. Increasing the number of shapes or segments results in pronounced performance drops (see object counting and jigsaw tasks), underscoring the flexibility and diagnostic utility of these tasks for continuous evaluation as models improve.

VideoSSR-30K: Scalable Self-Supervised Training Data

Using these three pretext tasks, the VideoSSR-30K dataset is constructed—comprising 30,000 question-answer pairs entirely independent of model/human annotation. Each task's proportion and subtype diversity are systematically controlled, as visualized below.

Figure 5: Task distribution in VIUBench and VideoSSR-30K for balanced evaluation and training.

This construction ensures high signal diversity while facilitating parametric difficulty tuning, supporting continual benchmarking and curriculum learning.

Reinforcement Learning Framework and Reward Shaping

VideoSSR applies RLVR—using GRPO as the backbone—with each pretext task accompanied by a tailored smooth reward function to combat the sparse signal problem associated with strict correctness-based rewards.

• Anomaly Grounding: Reward is the Mean Intersection over Union (mIoU) between predicted and ground truth anomaly intervals, yielding a dense [0,1] reward.
• Object Counting: Reward is $$R_\text{count} = \frac{1}{K} \sum_{k=1}^K \max(0, 1 - |\hat{y}_k - y_k|/(y_k+\epsilon))$$, reflecting normalized count error across $K$ shapes.
• Temporal Jigsaw: Reward is structure-aware: $R_\text{jigsaw} = 1 - E_\text{jigsaw}/E_{\max}$, with $E_\text{jigsaw}$ the sum of positional displacements between prediction and ground truth, penalizing permutation errors smoothly.

This reward shaping enables stable and efficient RLVR even for intrinsically challenging tasks, preventing zero-variance outcomes during training.

Experimental Results and Ablation Analysis

VideoSSR-8B, built on Qwen3-VL-8B-Instruct and trained for 1 epoch on VideoSSR-30K (8 H200 GPUs, 16 hours), is evaluated across 17 mainstream video benchmarks (General Video QA, Long Video QA, Temporal Grounding, and Complex Reasoning). Performance is consistently and significantly enhanced:

Figure 6: Performance comparison on four video tasks, demonstrating consistent improvement for VideoSSR.

Highlights:

• General Video QA: Improvements on temporally sensitive datasets (VinoGround: +10.6), and consistent gains on MVBench, TempCompass, AoTBench.
• Temporal Grounding: Dramatic zero-shot advances, e.g., QVHighlights (+15.9 mIoU), and solid gains on ActivityNet and CharadesSTA.
• Complex Reasoning: Substantial increase on VCRBench (+9.0), strongly linked with jigsaw task.
• Ablations:
  • All three pretext tasks individually contribute to Video-MME improvements.
  • Mixed-task training outperforms scaling of any single pretext to 30k samples.
  • SOTA open-source models fine-tuned on existing annotated datasets do not match VideoSSR's performance; in some cases, annotation bias induces degradation.
  • Among 14 anomaly perturbations, not all yield equal gains, with some temporal perturbations leading to negative transfer, likely due to the model's reliance on non-visual cues (e.g., timestamps).
  • 

  • Figure 7: Mixed-task training is more effective than scaling any single pretext task at fixed data scale.

  • 

  • Figure 8: Ablation results across 14 anomaly grounding perturbations: not all perturbations are equally valuable.

Implementation Considerations and Scalability

The entire approach is highly resource-efficient due to:

• Complete avoidance of manual/model annotation.
• Inherent scalability via parametric pretext generation.
• Efficient RLVR enabled by reward shaping.
• On par or improved compute cost compared to annotation-dependent paradigms.

For practical implementation, integrating VideoSSR into new MLLMs is simple: pretext task data pipelines can interface with any model capable of frame-based video input; reward functions are modular and directly compatible with GRPO-type RLVR methods. Scaling to longer videos requires further engineering, but the core methodology is agnostic to video length and task difficulty.

Implications, Limitations, and Future Directions

VideoSSR represents a robust, annotation-free framework for unlocking the next phase of video reasoning in MLLMs via self-supervision. Its strengths are:

• Independence from costly, bias-prone external annotation.
• Dynamic scalability in challenge through parametric task design.
• Applicability across diverse video reasoning domains.
• Robustness against reward signal collapse in RLVR.

However, as noted in the discussion, primary experiments are limited to shorter videos due to computational constraints, and only three pretext tasks were fully explored. Expanding the taxonomy of self-supervised tasks, designing adaptive curricula, and scaling to dense, long-form video will be critical for future advances.

Conclusion

VideoSSR demonstrates that intrinsic video properties—operationalized through anomaly grounding, object counting, and temporal jigsaw—provide a scalable, high-signal foundation for self-supervised RLVR in MLLMs. This approach consistently outperforms RLVR on static, annotated datasets and circumvents the scalability and bias bottlenecks of traditional annotation pipelines. By enabling progressionally harder evaluation and training as models advance, VideoSSR is poised to drive the next generation of robust, generalizable video-LLMs.