InteriorVideo: Synthetic Indoor Video Dataset

• InteriorVideo is a synthetic video dataset offering precise, physically based ground-truth intrinsic channels including albedo, normals, roughness, metallicity, and irradiance for dynamic indoor scenes.
• It features 1,154 videos from 297 diverse indoor scenes with detailed metadata, enabling controlled experiments in neural video rendering.
• The dataset supports the development of neural rendering and video diffusion models through smooth camera trajectory generation and comprehensive supervision of material, geometry, and lighting.

InteriorVideo is a synthetic video dataset specifically constructed to support research in intrinsic-guided neural video rendering and video diffusion models. It offers precise, physically based ground-truth supervision for color, geometry, material, and lighting in dynamic indoor scenes by providing dense per-frame intrinsic channels and metadata aligned with each rendered frame.

1. Dataset Composition and Structure

InteriorVideo encompasses 1,154 rooms extracted from 297 distinct indoor scenes. Each room contains a single contiguous video sequence, resulting in 1,154 separate videos. Each video comprises 120 frames rendered at approximately 10 frames per second, resulting in a total of 138,480 frames and approximately 3.8 hours of video data. The native resolution for all images is 640×448 pixels. On average, each scene contributes around four rooms, enabling coverage of a broad spatial and semantic variation within the dataset. The database is organized hierarchically by split (train/test), scene, and then room, ensuring efficient access patterns for large-scale experiments.

2. Intrinsic Channel Supervision

A defining attribute of InteriorVideo is the inclusion of five ground-truth intrinsic channels alongside the conventional RGB “beauty” render for every frame:

• Albedo: RGB per-pixel material base color, encoding diffuse reflectance as defined in the physically based rendering (PBR) engine.
• Surface Normals: World- or camera-space normal vectors encoded in RGB, extracted directly from the underlying triangle mesh at each shading point.
• Roughness: Scalar channel (single-band) representing microfacet distribution from the PBR model, with 0 indicating a perfectly smooth surface and 1 denoting fully rough.
• Metallicity: Scalar map indicating material “metalness,” sourced directly from material parameterization (0 for dielectrics, 1 for metals).
• Irradiance: Per-pixel ambient illumination (RGB, HDR) computed by path tracing the hemispherical integral, with Monte Carlo noise minimized using Intel Open Image Denoise (OIDN). The irradiance at pixel location $\mathbf{x}$ is computed as:

$$\mathrm{irradiance}(\mathbf{x}) = \int_\Omega L_i(\mathbf{x}, \omega)\;\mathrm{max}(n \cdot \omega, 0)\,d\omega$$

where $L_i$ is incoming radiance, $n$ is the normal vector.

This comprehensive channel set enables direct supervision for learning physically meaningful disentanglement of material, geometry, and lighting phenomena, crucial for controllable neural rendering and photorealistic video synthesis (Huang et al., 9 Oct 2025).

3. Camera Trajectory Generation

Camera paths through each room are generated to maximize perceptual richness and ensure smooth temporal dynamics:

• The virtual camera is initialized at a height of 1.7 meters, positioned either at a randomly chosen corner or midpoint of the room’s interior bounding rectangle.
• At each timestep ($\sim$0.05 m step length, $\sim$0.5 m/s at 10 FPS), 900 collision-free candidate camera positions in the local neighborhood are sampled.
• For each candidate $i$, the count $N_i$ of distinct visible objects in its view frustum is computed, and the position is weighted as $w_i = (N_i)^2$ to favor visually complex shot compositions.
• This strategy produces smoothly varying, content-aware trajectories with an average optical-flow magnitude of 3.16 px per frame, ensuring both wide spatial coverage and perceptual continuity for model training and evaluation.

4. Organization, Metadata, and Splits

Data are distributed in a directory tree reflecting experimental splits:

• Train: 235 scenes, 917 rooms, 110,040 frames
• Test: 60 scenes, 237 rooms, 28,440 frames

Within each room directory:

• frames/: Contains RGB beauty passes and all intrinsic channel outputs per frame.
• cams.json: List of per-frame camera extrinsics (4×4), intrinsics, and frame indices.
• scene_labels.txt (optional): Scene and room IDs, room type, and semantic annotations.

Framewise images and irradiance are stored as .png (RGB and scalar channels) and HDR .exr (irradiance) files, facilitating high-fidelity training regimes for geometry, reflectance, and lighting estimation. The dataset structure directly supports per-room, per-scene, and split-wise selection for controlled experiments (Huang et al., 9 Oct 2025).

5. Licensing, Citation, and Research Applications

InteriorVideo is publicly released under the Creative Commons Attribution-NonCommercial (CC BY-NC 4.0) license, with unrestricted academic and research use contingent on proper attribution. The full dataset and metadata are available at https://luckyhzt.github.io/x2video (Huang et al., 9 Oct 2025).

The principal intended use case is as a benchmark and training corpus for intrinsic-guided neural video rendering models, especially video diffusion models requiring accurate material, geometry, and illumination supervision. It is the foundational dataset for X2Video, which leverages the five ground-truth channels to support controllable, temporally consistent photorealistic video synthesis, with global and local editing capabilities for color, material, geometry, and lighting.

6. Significance and Context in Video Generation Research

InteriorVideo is, as of the cited publication, unique in offering frame-synchronous, physically grounded render passes across an extensive scale of indoor scenes, supporting robust generalization in data-driven neural rendering regimes. The richness of its intrinsic channel annotations makes it amenable to research targeting disentangled and interpretable video generation, as well as techniques requiring accurate supervision for material and lighting transfer, domain adaptation, or multimodal controllable generation (Huang et al., 9 Oct 2025). Compared to existing datasets lacking explicit per-frame ground-truth for material and light transport, InteriorVideo provides unprecedented fine-grained annotation density for developing and evaluating next-generation neural rendering methods in controlled indoor settings.