---
title: 'UniStereo: Unified Stereo Dataset'
url: https://www.emergentmind.com/topics/unistereo-dataset
type: topic
---

# UniStereo: Unified Stereo Dataset

UniStereo is the first large-scale, unified dataset designed for stereo video conversion research, explicitly constructed to support both parallel and converged (toe-in) stereo formats. It enables fair benchmarking and robust training of monocular-to-stereo video conversion models for applications such as virtual reality (VR) and 3D cinema. UniStereo addresses limitations of previous datasets by providing format-diverse, standardized training and evaluation splits, paired with per-clip textual captions to support multi-modal research and avoid evaluation mismatches across different stereo geometries [2512.16915].

## 1. Motivation, Design, and Objectives

UniStereo responds to the rapidly increasing demand for high-quality stereo content driven by widespread adoption of stereoscopic displays. Traditional 3D video acquisition remains expensive, while manual 2D-to-3D workflows are labor-intensive. Contemporary automatic approaches—dominated by the depth-warp-inpaint pipeline—suffer from error propagation, intrinsic depth ambiguity, and lack of robustness to differing stereo geometries. Previous methods have been limited by reliance on private datasets or on content in a single stereo format, undermining the reproducibility and comparability of results.

UniStereo was constructed with three main objectives:
- Provide a unified corpus for both parallel and converged stereo geometries.
- Supply standardized test splits for each format to ensure consistent evaluation.
- Include paired textual captions to advance text-conditioned mono-to-stereo research [2512.16915].

## 2. Dataset Composition and Structure

UniStereo comprises three subsets: Stereo4D (parallel), 3DMovie (converged), and UE5-Syn (synthetic, parallel). Each subset contributes diverse and complementary data sources, with all clips standardized to 81 frames (5 seconds at 16 fps), 832×480 resolution.

| Subset    | Source                        | Format    | Approx. Clips | Training / Test Clips | Domains / Notables                       |
|-----------|-------------------------------|-----------|:-------------:|:--------------------:|------------------------------------------|
| Stereo4D  | VR180 YouTube (~7,000 videos) | Parallel  | ~100,000      | 58,000 / 400         | Indoor/outdoor, moving/static, specular  |
| 3DMovie   | 142 commercial 3D films       | Converged | ~44,900       | 44,879 / 400         | Live-action, animation, cinematic scenes |
| UE5-Syn   | Unreal Engine 5 synthetic     | Parallel  | 200           | n/a / 200            | 28 animal classes, day/night, synthetic  |

Each video comprises fixed 81-frame segments, resulting in a total of approximately 103,000 stereo pairs. The dataset’s domain diversity includes indoor/outdoor environments, dynamic/static content, and a balance of reflective and transparent surfaces, challenging models to generalize across varied parallax and scene content.

## 3. Data Acquisition and Preprocessing Pipeline

### Stereo4D (Parallel)
- Sourced from ~7,000 VR180 YouTube videos; includes camera2world matrices and rectification parameters.
- Rectification to perspective views (HFOV = 90°), cropped to 832×480.
- Downsampled to 16 fps and sliced to 81-frame clips; short clips are discarded.
- Captions generated for left views using ShareGPT4Video [2512.16915].

### 3DMovie (Converged)
- Manually curated 142 true converged films in side-by-side (SBS) format; excludes pseudo-stereo and top-bottom layouts.
- Clips segmented using PySceneDetect; only segments ≥81 frames retained.
- Manual trimming removes credits/overlays; strict odd-indexed sampling reduces redundancy.
- Symmetric border cropping eliminates visual artifacts.
- Frames resized to 832×480; left-view captions created analogously.

3DMovie assumes temporal alignment from SBS encoding, requiring no further calibration. Stereo4D provides per-clip geometric calibration data, enabling, for example, on-the-fly computation of disparity from geometry if required.

## 4. Annotations, Quality Assurance, and Distribution

Annotations consist of paired left/right stereo video clips, which serve as ground-truth for supervised learning. While no explicit pixelwise depth maps or disparity maps are provided, Stereo4D’s geometric metadata allows derivation for evaluation purposes. Left-eye captions, obtained via ShareGPT4Video, offer a foundation for text-conditioned baseline and multi-modal task development.

Quality control includes:
- Manual verification of converged geometry in 3DMovie to prevent mislabeling.
- Automated filtration of non-informative segments (e.g., logos, credits).
- Uniform, symmetric border cropping to maintain data integrity [2512.16915].

Distribution is managed via a public repository (https://github.com/KlingTeam/StereoPilot), with download scripts enabling retrieval of video sources and metadata; users acquire raw assets using tools such as yt-dlp or from institutional archives.

## 5. Geometric Formulation and Statistical Properties

### Stereo Geometry
#### Parallel
- $x_l = f X / Z$, $x_r = f (X-B)/Z$
- Disparity: $d = x_l - x_r = f B / Z$ → $Z = f B / d$

#### Converged
- Cameras rotated by $\theta$ about vertical axis, baseline $B$
- For left camera: $[X_L;Y_L;Z_L] = R(-\theta)[X+B/2;Y;Z]$, right: $[X_R;Y_R;Z_R] = R(+\theta)[X-B/2;Y;Z]$
- $x_l = f X_L/Z_L$, $x_r = f X_R/Z_R$
- Rotation matrix: $R(\theta) = \begin{bmatrix} \cos\theta & 0 & -\sin\theta \\ 0 & 1 & 0 \\ \sin\theta & 0 & \cos\theta \end{bmatrix}$

### Statistical Highlights
- Baseline $B$ in Stereo4D: 6–8 cm, per-clip rectification supplied.
- 3DMovie convergence angles distributed for cinematic zero-disparity planes at ~5–10 m.
- Disparity spans $\approx \pm$32 px at the prescribed resolution.
- Focal length $f \approx 416$ px from $f = (832/2)/\tan(90^\circ/2)$.
- Scene types: even split between static/dynamic, indoor/outdoor; moving objects, specular materials, and complex lighting are well-represented [2512.16915].

## 6. Benchmarking Protocols and Impact

UniStereo enables fair, cross-format evaluation through standardized test splits and dataset-wide diversity. The introduction of a learnable “domain switcher” mechanism in associated models such as StereoPilot allows end-to-end adaptation to both stereo geometries. Evaluation protocols follow the prescribed splits: 58,000 Stereo4D clips + 44,879 3DMovie clips for training, and 400 reserved clips per subset for testing (no predefined validation split is published).

### Baseline Metrics and Results

Metrics: SSIM, MS-SSIM, PSNR, LPIPS, SIOU, and inference latency per 81-frame clip on a single GPU.

| Method               | SSIM (Par/Conv) | PSNR (Par/Conv) | LPIPS (Par/Conv) | SIOU (Par/Conv) | Latency      |
|----------------------|----------------:|----------------:|-----------------:|----------------:|-------------:|
| StereoDiffusion      | 0.642 / 0.678   | 20.54 / 20.70   | 0.245 / 0.341    | 0.252 / 0.181   | 60 min       |
| StereoCrafter        | 0.553 / 0.706   | 17.67 / 23.79   | 0.298 / 0.203    | 0.226 / 0.213   | 1 min        |
| Mono2Stereo          | 0.649 / 0.795   | 20.89 / 25.76   | 0.222 / 0.191    | 0.241 / 0.201   | 15 min       |
| StereoPilot          | 0.861 / 0.837   | 27.74 / 27.86   | 0.087 / 0.122    | 0.408 / 0.260   | 11 s         |

StereoPilot, trained on UniStereo, demonstrates leading fidelity and perceptual alignment—PSNR improvement by 7 dB over prior state-of-the-art and LPIPS reduced by half. Inference time is reduced to 11 seconds per clip, versus minutes or hours for previous approaches. This suggests UniStereo is critical for advancing real-time, robust, and generalizable mono-to-stereo models [2512.16915].

## 7. Significance and Research Implications

UniStereo establishes the first regime for equitable evaluation of mono-to-stereo conversion across both prevailing geometric conventions in stereoscopic video. Its careful design—unified geometry coverage, exhaustive preprocessing, domain-balanced splits, and multi-modal annotations—directly addresses format-specific biases that previously hampered the field. The dataset lowers the barrier to reproducible benchmarking, catalyzing robust model development (as in the StereoPilot framework) and facilitating new research on text-conditioned conversion and generalization across diverse content domains. The public data release, detailed metadata, and alignment with open-source evaluation protocols are designed to advance the scientific rigor and comparability in stereo video research [2512.16915].

Source: https://www.emergentmind.com/topics/unistereo-dataset