---
title: Kubric MOVi-C Dataset for Object-Centric Learning
url: https://www.emergentmind.com/topics/kubric-movi-c-dataset
type: topic
---

# Kubric MOVi-C Dataset for Object-Centric Learning

The Kubric MOVi-C dataset constitutes the third tier of the Multi-Object Video (MOVi) series, explicitly designed to advance object-centric discovery and segmentation methodologies within highly realistic and dynamic synthetic video environments. It targets benchmarking of unsupervised object discovery, motion-based grouping, and assessment of temporal consistency for methods at the intersection of visual perception and object-centric learning. MOVi-C distinguishes itself by leveraging photorealistically rendered, scanned household objects against real-world High Dynamic Range Image (HDRI) backgrounds, thereby introducing a level of texture and scene fidelity absent from earlier MOVi dataset tiers [2203.03570].

## 1. Hierarchical Position and Comparative Context

MOVi-C sits at an intermediate complexity level within the MOVi series. Unlike MOVi-A and MOVi-B—which rely on geometric primitives (spheres, cubes, cones, cylinders, pastel colorings, and uniform lighting) and simple scene arrangements—MOVi-C employs 5–10 high-quality 3D object scans per scene, drawn from both Google Scanned Objects (GSO, approximately 1,000 unique meshes) and a 51,300-model subset of ShapeNetCore.v2 encompassing 55 object categories. Each object carries real-world Physically-Based Rendering (PBR) textures. Environments feature randomized HDRI backgrounds from Polyhaven, and lighting is further modulated via a synthetic point source with color jitter. In contrast, MOVi-D increases object counts and introduces sparse movers, and MOVi-E adds a moving camera trajectory to further complicate temporal analysis [2203.03570].

| Dataset | Object Set | Scene Complexity | Camera Motion |
|---------|------------|------------------|--------------|
| MOVi-A  | 8 primitives | Flat, simple | Static |
| MOVi-B  | 8 primitives + cones, capsules | Checkerboard, pastel | Static |
| MOVi-C  | 5–10 scanned meshes | HDRI backgrounds | Static |
| MOVi-D  | up to 23 (dynamic + static) | Mixed motion | Static |
| MOVi-E  | up to 23 | Mixed, moving camera | Dynamic |

## 2. Data Generation Pipeline and Framework Architecture

The Kubric Python framework orchestrates MOVi-C data synthesis by interfacing PyBullet for physical simulation and Blender’s Cycles for photorealistic rendering. The workflow encompasses:

- **Scene Initialization**: A `Scene` object encapsulates global settings (gravity, frame count, camera intrinsics, lighting).
- **Asset Ingestion**: Assets sourced via KuBasic, ShapeNetCore.v2, or GSO manifests provide mesh data, real-world PBR textures, and URDF collision geometry.
- **Physics Simulation**: PyBullet manages rigid-body dynamics, executing 24 simulation steps ($\Delta t=1/24$ s per step) to produce per-frame object trajectories and collision logs. Physical parameters include standard gravity ($g = [0,0,-9.81]$), restitution (0.5), and friction (0.4).
- **Rendering**: Blender Cycles, synchronized via shared scene graphs, interpolates object motion and ray-traces output modalities at 64 samples per pixel, 256×256 px resolution, with maximum bounce count set to 8 and no denoising.
- **Export**: RGB, instance segmentation, depth, optical flow, and surface normals are exported in standardized formats, augmented by JSON metadata per scene [2203.03570].

Orchestration ensures that each asset is congruently represented in both simulation and rendering domains—position, orientation, and scale are mirrored and governed by the same random initializations.

## 3. Scene, Object, and Motion Configuration

Each MOVi-C sequence instantiates $n \sim Uniform\{5, ..., 10\}$ objects per scene, with object placement assured non-overlapping via rejection sampling. Object scale per instance is sampled as $s \sim \mathcal{U}([0.8,1.2])$, allowing modest in-category variation. Appearance leverages real-world textures mapped from the associated 3D mesh, with scene backdrops stochastically drawn from an HDRI pool and lighting augmented by a point-light of intensity $I \sim \mathcal{U}([5,15])$ W, color sampled componentwise as $\mathcal{U}([0.8,1.2])$.

Initial object velocities follow $\|\vec{v}\| \sim \mathcal{U}(0,2)$ units/s, with directions sampled uniformly from the unit sphere. No extraneous external forces act beyond gravity. Rotational inertia tensors derive from convex-decomposed URDFs, ensuring accurate dynamic response. This parameterization provides a high-variance, yet physically plausible, set of motion conditions for object discovery methods to exploit [2203.03570].

## 4. Camera Model, Lighting, and Rendering Modalities

The camera adopts a Blender perspective model with focal length $f = 50$ mm, sensor width 36 mm, yielding 256×256 pixel outputs. Its location is randomized about the scene center: elevation $\theta \sim \mathcal{U}(30^\circ, 60^\circ)$, azimuth $\phi \sim \mathcal{U}(0^\circ, 360^\circ)$, but is always static per sequence in MOVi-C (unlike MOVi-E). Backgrounds rely on Polyhaven-provided HDRI, delivering varied, realistic global illumination.

Output modalities include:

- **RGB**: 8-bit PNG per frame
- **Instance Segmentation**: 16-bit PNG (object IDs)
- **Depth**: 32-bit float EXR
- **Optical Flow**: .flo format (Middlebury convention)
- **Surface Normals**: 32-bit EXR in world coordinates

Rendering uses Cycles with 64 samples per pixel and a maximum of 8 light bounces, without denoising or motion blur; the system supports automated depth-of-field should it be enabled [2203.03570].

## 5. Annotations, Metadata, Storage, and Dataset Statistics

Rich ground-truth annotation accompanies every sequence:

- **Pixel-level**: instance masks, optical flow vectors, depth maps, normal maps
- **Object-centric**: 3D pose ($R \in SO(3), t \in \mathbb{R}^3$), bounding box, mass, friction, velocity, per-object collision logs
- **Camera**: intrinsic matrix $K$, extrinsic $[R|t]$

Data organization is per-scene:

```
MOVi-C/
  scene_00000/
    metadata.json
    rgba/000.png … 023.png
    seg/000.png … 023.png
    depth/000.exr … 023.exr
    flow/000.flo … 023.flo
    norm/000.exr … 023.exr
```

The full dataset comprises 100,000 video sequences (24 frames each), totaling approximately 2.4 million frames and ~1.5 TB. Dataset splits are 80% training (80,000 scenes), 10% validation (10,000 scenes), and 10% test (10,000 scenes). The object count per scene is mean $\approx 7.5 \pm 1.7$, with a pool of 1,000 unique object meshes and 55 ShapeNet categories [2203.03570].

## 6. Application Domains and Benchmarks

MOVi-C directly targets the evaluation of unsupervised object discovery and segmentation from video, with additional utility for examining temporal consistency and the role of motion cues in object grouping. Practical use cases include:

- **Slot-attention and spatial-slot models**: assessment of robustness to real-world textures and backgrounds
- **Motion-based grouping**: evaluation of generalization to complex object geometry
- **Temporal segmentation**: measuring trajectory consistency

Reported benchmarks for unsupervised object discovery (Foreground Adjusted Rand Index, ARI) are as follows:

| Method      | Foreground ARI (%)          |
|-------------|-----------------------------|
| SAVi [Anon ’22]     | $47.0 \pm 0.3$    |
| SIMONe [Kabra ’21]  | $19.8 \pm 0.5$    |
| SAVi + BBox Init Cue| $73.5 \pm 0.3$    |

A plausible implication is that the complexity and realism of MOVi-C substantially challenge current object-centric video models, especially with respect to segmentation under realistic appearance and lighting statistics [2203.03570].

## 7. Significance, Limitations, and Future Context

MOVi-C, as part of the Kubric datasets, exemplifies the scalability and annotation richness afforded by synthetic data generation. Its design addresses the limitations of uniform, low-level synthetic scenes and facilitates controlled experimentation with complex objects and photorealistic environments. This framework enables reproducible, large-scale evaluation across object segmentation, motion analysis, and temporal consistency tasks. A plausible implication is that future research leveraging MOVi-C will contribute to closing the gap between synthetic and real data performance, but performance benchmarks indicate existing methods face substantially increased challenges as photorealism and scene complexity rise [2203.03570].

Source: https://www.emergentmind.com/topics/kubric-movi-c-dataset