SloMo-44K Dataset

• SloMo-44K dataset is a curated collection of 44,632 slow-motion video segments with detailed annotations, enabling benchmarking in realistic temporal reasoning and deblurring.
• It employs a robust multi-stage pipeline, including shot segmentation, overlay filtering, and self-supervised speed estimation, to ensure high precision in slow-motion content selection.
• The dataset supports diverse applications such as speed estimation, temporal super-resolution, video generation, and practical deblurring, offering a valuable benchmark for real-world challenges.

The SloMo-44K dataset is a large-scale resource comprising 44,632 real-world slow-motion video segments, curated for research in video temporal reasoning, temporally controllable video generation, and realistic image deblurring tasks. Assembled via automated pipelines leveraging self-supervised models and visual-language systems, SloMo-44K contains approximately 18 million frames drawn from over 18,000 source videos, with coverage spanning a diverse array of content domains and capture conditions. The dataset enables systematic benchmarking of algorithms for slow-motion recognition, temporal super-resolution, speed estimation, and practical deblurring in naturalistic settings, providing dense per-clip annotations of speed and scene attributes as well as standardized synthetic blur constructions for controlled evaluation (Wu et al., 23 Apr 2026, Noki et al., 24 Jun 2025).

1. Dataset Construction and Composition

SloMo-44K was constructed from publicly available high-frame-rate videos crawled from YouTube, Vimeo, and Flickr. Candidate videos were identified using queries indicative of slow-motion content (“high frame rate,” “high-speed camera,” etc.). The pipeline included multiple stages:

• Shot segmentation: TransNetv2 was applied to divide videos into homogeneous shots.
• Overlay filtering: DPText-DETR (OCR) discarded clips with heavy text/subtitle overlays.
• CGI/screen-capture removal: Qwen2.5-VL classifier excluded synthetic or recorded desktop footage.
• Quality assessment: A COVER-based model filtered out low-resolution or poorly compressed clips.

Segments then underwent automatic detection of playback speed changes using a VideoMAEv2-based detector, trained by cross-modal supervision on synchronized audio (with pitch-shift events as indicators of speed change). Inference located change points, segmenting shots into temporally uniform sub-clips.

To reliably select slow-motion segments, a two-stage filtering integrated (i) Gemini 2.5 VLM prompting to localize slow-motion regions and (ii) fine-tuned VideoMAEv2 (ViT backbone) clip-level classification; only segments with both high predicted slow-motion probability (>0.998) and VLM localization (>10%) were retained, achieving ~98% slow-motion precision at ~44% recall.

Each retained segment was annotated via a self-supervised speed estimator trained for log-equivariance to temporal acceleration, then refined iteratively on ultra-slow clips by repeated acceleration/re-estimation cycles (3×). Dense multi-granularity captions and scene/style/attribute descriptors were produced using InternVL3.

2. Dataset Statistics and Annotation Protocol

SloMo-44K comprises:

• 44,632 slow-motion video segments (1–20 s each)
• Sourced from 18,235 full-length videos
• ≈18 million frames (raw capture rates up to >10,000 fps)
• Playback speeds distributed log-uniformly from 0.01× to 1.0× real time

Annotations per segment include:

• Pseudo-speed scalar (continuous, e.g., 0.05, 0.10, … 1.0)
• A binary flag for slow-motion status
• Fine-grained dense captions (scene category, style, shot type, lighting, atmosphere)
• Extracted scene nouns, verbs, and attributes

Table 1 summarizes key statistics.

Property	Value/Description	Notes
# segments	44,632	1–20 s per segment
# full videos	18,235	Diverse sources
# frames	≈18M	Up to >10K fps capture
Speed labels	[0.01, 1.0]× (log-uniform)	Pseudo-labeled
Caption vocabulary	Thousands of unique words	Extracted with InternVL3

A held-out subset (“SloMo-44K-Test”) was reserved for temporal super-resolution evaluation, with supplementary speed estimation and speed-change detection splits also constructed (Wu et al., 23 Apr 2026).

3. Blur Synthesis, Ground Truth, and Benchmarking

SloMo-44K includes a large-scale set of blur-sharp image pairs (“Deblurring in the Wild”), constructed from smartphone high-speed video. Using Apple iPhone 15 Pro in 240 fps slow-motion mode, raw frames at 1920×1080 were stacked in non-overlapping 30-frame windows to simulate continuous exposure:

• Blurred image:

$$B' = \frac{1}{N} \sum_{i=0}^{N-1} F_i, \quad N = 30$$

with $T = N/240 = 1/8$ s effective exposure.

• Sharp ground truth: the centermost frame $F_{\mathrm{GT}} = F_{\lfloor N/2 \rfloor}$ minimizes temporal misalignment.

Algorithm 1 in (Noki et al., 24 Jun 2025) details the pipeline: accumulate sum, select center, normalize, store uint8 images. The dataset contains 42,045 blur-sharp pairs spanning 843 scenes, train/test split of 37,841/4,204. Scenes include both indoor and outdoor environments, a spectrum of lighting conditions, and diverse camera/object motion including complex depth parallax.

Evaluation of SOTA deblurring models (Restormer, MPRNet, NAFNet, FFTformer, DMPHN, MIMO-UNet++, AdaRevD) demonstrates systematic performance degradation relative to legacy benchmarks (none surpassing the blurry input baseline of 32.38 dB PSNR and 0.777 SSIM), emphasizing the challenge of real-world smartphone blur (Noki et al., 24 Jun 2025).

4. Core Methodologies and Algorithms

SloMo-44K’s generation and annotation pipeline leverages several technical advances:

• Self-supervised speed estimation (equivariance loss): For each video $V$ and its $k$-accelerated variant $V^k$,

$$L_1 = \left[ \log f_\theta(V^k) - \log (k \cdot f_\theta(V)) \right]^2$$

calibrated on Adobe240fps with supervised regression.

• Temporal super-resolution synthetic blur: Constructed by frame-averaging high-FPS input and temporal downsampling.
• Speed-conditioned video generation: Target speeds are bucketized on a log scale from 0.01 to 1.0×; time embeddings are enriched by MLP transforms of positional encodings and framewise speed modulation.

Models for both generation and enhancement are based on Wan2.1 I2V-14B or VACE-14B architectures, with LoRA adapters finetuned over 1–2 days on 4×A100 GPUs (learning rates: $1\times 10^{-5}$ for speed modules, $1\times 10^{-4}$ for LoRA adaptation) (Wu et al., 23 Apr 2026).

5. Distributional and Phenomenological Characteristics

The dataset exhibits wide variation in both temporal and spatial phenomena:

• Playback speeds: Log-uniform, ranging from extreme slow motion (0.01×) to real-time (1.0×).
• Scene domains: Sports, animals, fluid dynamics, everyday objects, and more.
• Motion types: Linear, rotational, arbitrary 2D, including non-uniform blur from depth, occlusions.
• Lighting: Varies from high-contrast daylight to complex mixed indoor settings.
• Challenge features: Significant non-uniform blur, camera shake, artifacts from smartphone ISP pipelines, and frequent lack of ground-truth FPS metadata.

6. Usage Recommendations and Best Practices

To ensure reproducibility, the following are recommended:

• Employ the provided train/val/test splits and utilize the supplied random seeds.
• Reproduce all filtering and selection steps: TransNetv2, DPText-DETR OCR filtering, Qwen2.5-VL CGI detection.
• For robust slow-motion selection, combine VideoLLM (Gemini 2.5) with a fine-tuned VideoMAEv2 ViT classifier and balance speed buckets during model training.
• For temporal super-resolution, generate synthetic blur according to the SloMo-44K protocols to match real-world low-FPS motion degradation.

Project resources (including download links, data formats, speed/caption files, and preprocessing scripts) are available at https://seeing-fast-and-slow.github.io/ (Wu et al., 23 Apr 2026). License terms are guided by the project page and presumed to permit research applications.

7. Applications, Challenges, and Research Impact

SloMo-44K supports research in:

• Video temporal reasoning: Benchmarking model capacity to perceive, estimate, and condition on the ‘flow’ of time.
• Temporal super-resolution: Transforming blurry, low-FPS sequences to high-FPS, sharp outputs via learned temporal interpolation and restoration.
• Video generation: Conditioning generative models on explicit playback speed for controllable motion synthesis.
• Practical deblurring: Realistic training and evaluation of deblurring networks reflecting in-the-wild consumer capture statistics.
• Downstream vision tasks: Enhanced pre-processing for SLAM, AR/VR, object tracking, and video stabilization.

Major challenges include the absence of ground truth for true playback speed in most wild slow-motion footage, scene-wise heterogeneity (mixed speeds, editing), and complex artifacts (motion blur, parallax, occlusions). The dataset’s breadth and annotation depth make it a robust, challenging benchmark for studying motion and time in computer vision, and for developing models that generalize to diverse spatiotemporal phenomena (Wu et al., 23 Apr 2026, Noki et al., 24 Jun 2025).