---
title: SloMo-44K Dataset
url: https://www.emergentmind.com/topics/slomo-44k-dataset
type: topic
---

# SloMo-44K Dataset

The SloMo-44K dataset is a large-scale resource comprising 44,632 real-world slow-motion video segments, curated for research in video temporal reasoning, temporally controllable video generation, and realistic image deblurring tasks. Assembled via automated pipelines leveraging self-supervised models and visual-language systems, SloMo-44K contains approximately 18 million frames drawn from over 18,000 source videos, with coverage spanning a diverse array of content domains and capture conditions. The dataset enables systematic benchmarking of algorithms for slow-motion recognition, temporal super-resolution, speed estimation, and practical deblurring in naturalistic settings, providing dense per-clip annotations of speed and scene attributes as well as standardized synthetic blur constructions for controlled evaluation [2604.21931, 2506.19445].

## 1. Dataset Construction and Composition

SloMo-44K was constructed from publicly available high-frame-rate videos crawled from YouTube, Vimeo, and Flickr. Candidate videos were identified using queries indicative of slow-motion content (“high frame rate,” “high-speed camera,” etc.). The pipeline included multiple stages:

- **Shot segmentation**: TransNetv2 was applied to divide videos into homogeneous shots.
- **Overlay filtering**: DPText-DETR (OCR) discarded clips with heavy text/subtitle overlays.
- **CGI/screen-capture removal**: Qwen2.5-VL classifier excluded synthetic or recorded desktop footage.
- **Quality assessment**: A COVER-based model filtered out low-resolution or poorly compressed clips.

Segments then underwent automatic detection of playback speed changes using a VideoMAEv2-based detector, trained by cross-modal supervision on synchronized audio (with pitch-shift events as indicators of speed change). Inference located change points, segmenting shots into temporally uniform sub-clips.

To reliably select slow-motion segments, a two-stage filtering integrated (i) Gemini 2.5 VLM prompting to localize slow-motion regions and (ii) fine-tuned VideoMAEv2 (ViT backbone) clip-level classification; only segments with both high predicted slow-motion probability (>0.998) and VLM localization (>10%) were retained, achieving ~98% slow-motion precision at ~44% recall.

Each retained segment was annotated via a self-supervised speed estimator trained for log-equivariance to temporal acceleration, then refined iteratively on ultra-slow clips by repeated acceleration/re-estimation cycles (3×). Dense multi-granularity captions and scene/style/attribute descriptors were produced using InternVL3.

## 2. Dataset Statistics and Annotation Protocol

SloMo-44K comprises:
- 44,632 slow-motion video segments (1–20 s each)
- Sourced from 18,235 full-length videos
- ≈18 million frames (raw capture rates up to >10,000 fps)
- Playback speeds distributed log-uniformly from 0.01× to 1.0× real time

Annotations per segment include:
- Pseudo-speed scalar (continuous, e.g., 0.05, 0.10, … 1.0)
- A binary flag for slow-motion status
- Fine-grained dense captions (scene category, style, shot type, lighting, atmosphere)
- Extracted scene nouns, verbs, and attributes

Table 1 summarizes key statistics.

| Property           | Value/Description                         | Notes                                                   |
|--------------------|-------------------------------------------|---------------------------------------------------------|
| # segments         | 44,632                                    | 1–20 s per segment                                      |
| # full videos      | 18,235                                    | Diverse sources                                         |
| # frames           | ≈18M                                      | Up to >10K fps capture                                 |
| Speed labels       | [0.01, 1.0]× (log-uniform)                | Pseudo-labeled                                          |
| Caption vocabulary | Thousands of unique words                 | Extracted with InternVL3                                |

A held-out subset (“SloMo-44K-Test”) was reserved for temporal super-resolution evaluation, with supplementary speed estimation and speed-change detection splits also constructed [2604.21931].

## 3. Blur Synthesis, Ground Truth, and Benchmarking

SloMo-44K includes a large-scale set of blur-sharp image pairs (“Deblurring in the Wild”), constructed from smartphone high-speed video. Using Apple iPhone 15 Pro in 240 fps slow-motion mode, raw frames at 1920×1080 were stacked in non-overlapping 30-frame windows to simulate continuous exposure:

- **Blurred image**: 
  $$
  B' = \frac{1}{N} \sum_{i=0}^{N-1} F_i, \quad N = 30
  $$
  with $T = N/240 = 1/8$ s effective exposure.

- **Sharp ground truth**: the centermost frame $F_{\mathrm{GT}} = F_{\lfloor N/2 \rfloor}$ minimizes temporal misalignment.

Algorithm 1 in [2506.19445] details the pipeline: accumulate sum, select center, normalize, store uint8 images. The dataset contains 42,045 blur-sharp pairs spanning 843 scenes, train/test split of 37,841/4,204. Scenes include both indoor and outdoor environments, a spectrum of lighting conditions, and diverse camera/object motion including complex depth parallax.

Evaluation of SOTA deblurring models (Restormer, MPRNet, NAFNet, FFTformer, DMPHN, MIMO-UNet++, AdaRevD) demonstrates systematic performance degradation relative to legacy benchmarks (none surpassing the blurry input baseline of 32.38 dB PSNR and 0.777 SSIM), emphasizing the challenge of real-world smartphone blur [2506.19445].

## 4. Core Methodologies and Algorithms

SloMo-44K’s generation and annotation pipeline leverages several technical advances:

- **Self-supervised speed estimation** (equivariance loss): For each video $V$ and its $k$-accelerated variant $V^k$,
  $$
  L_1 = \left[ \log f_\theta(V^k) - \log (k \cdot f_\theta(V)) \right]^2
  $$
  calibrated on Adobe240fps with supervised regression.
- **Temporal super-resolution synthetic blur**: Constructed by frame-averaging high-FPS input and temporal downsampling.
- **Speed-conditioned video generation**: Target speeds are bucketized on a log scale from 0.01 to 1.0×; time embeddings are enriched by MLP transforms of positional encodings and framewise speed modulation.

Models for both generation and enhancement are based on Wan2.1 I2V-14B or VACE-14B architectures, with LoRA adapters finetuned over 1–2 days on 4×A100 GPUs (learning rates: $1\times 10^{-5}$ for speed modules, $1\times 10^{-4}$ for LoRA adaptation) [2604.21931].

## 5. Distributional and Phenomenological Characteristics

The dataset exhibits wide variation in both temporal and spatial phenomena:
- **Playback speeds**: Log-uniform, ranging from extreme slow motion (0.01×) to real-time (1.0×).
- **Scene domains**: Sports, animals, fluid dynamics, everyday objects, and more.
- **Motion types**: Linear, rotational, arbitrary 2D, including non-uniform blur from depth, occlusions.
- **Lighting**: Varies from high-contrast daylight to complex mixed indoor settings.
- **Challenge features**: Significant non-uniform blur, camera shake, artifacts from smartphone ISP pipelines, and frequent lack of ground-truth FPS metadata.

## 6. Usage Recommendations and Best Practices

To ensure reproducibility, the following are recommended:
- Employ the provided train/val/test splits and utilize the supplied random seeds.
- Reproduce all filtering and selection steps: TransNetv2, DPText-DETR OCR filtering, Qwen2.5-VL CGI detection.
- For robust slow-motion selection, combine VideoLLM (Gemini 2.5) with a fine-tuned VideoMAEv2 ViT classifier and balance speed buckets during model training.
- For temporal super-resolution, generate synthetic blur according to the SloMo-44K protocols to match real-world low-FPS motion degradation.

Project resources (including download links, data formats, speed/caption files, and preprocessing scripts) are available at https://seeing-fast-and-slow.github.io/ [2604.21931]. License terms are guided by the project page and presumed to permit research applications.

## 7. Applications, Challenges, and Research Impact

SloMo-44K supports research in:
- **Video temporal reasoning**: Benchmarking model capacity to perceive, estimate, and condition on the ‘flow’ of time.
- **Temporal super-resolution**: Transforming blurry, low-FPS sequences to high-FPS, sharp outputs via learned temporal interpolation and restoration.
- **Video generation**: Conditioning generative models on explicit playback speed for controllable motion synthesis.
- **Practical deblurring**: Realistic training and evaluation of deblurring networks reflecting in-the-wild consumer capture statistics.
- **Downstream vision tasks**: Enhanced pre-processing for SLAM, AR/VR, object tracking, and video stabilization.

Major challenges include the absence of ground truth for true playback speed in most wild slow-motion footage, scene-wise heterogeneity (mixed speeds, editing), and complex artifacts (motion blur, parallax, occlusions). The dataset’s breadth and annotation depth make it a robust, challenging benchmark for studying motion and time in computer vision, and for developing models that generalize to diverse spatiotemporal phenomena [2604.21931, 2506.19445].

Source: https://www.emergentmind.com/topics/slomo-44k-dataset