---
title: 'Occ-ScanNet: Indoor 3D Occupancy Benchmark'
url: https://www.emergentmind.com/topics/occ-scannet
type: topic
---

# Occ-ScanNet: Indoor 3D Occupancy Benchmark

Occ-ScanNet is a large-scale benchmark dataset for semantic 3D occupancy prediction in indoor environments. Constructed from the ScanNet RGB-D reconstruction corpus, Occ-ScanNet provides per-frame dense voxel grids with fine-grained semantic labels, serving as a foundation for training and evaluating monocular occupancy prediction architectures. The benchmark supports both local (single-frame, frustum-centric) and embodied (sequential, global) occupancy tasks and drives advances in online scene mapping, embodied perception, and semantic spatial reasoning.

## 1. Dataset Construction and Annotation Protocol

Occ-ScanNet derives its data from the ScanNet indoor dataset, which consists of RGB videos, depth maps, camera intrinsics, and trajectory information for over 650 real-world indoor scenes, including living rooms, bedrooms, kitchens, offices, and corridors [2407.11730]. For Occ-ScanNet, frames are sampled such that each view is represented as a canonical voxel grid of dimensions $60 \times 60 \times 36$, corresponding to a camera frustum of $4.8 \,\mathrm{m} \times 4.8\, \mathrm{m} \times 2.88\,\mathrm{m}$ at 8 cm resolution [2504.14604][2605.25059].

Ground-truth annotation is obtained by assigning each voxel the semantic label of the nearest mesh surface point from the ScanNet-fused 3D reconstruction. Only voxels positioned in front of the camera are labeled; frames are filtered if camera parameters are invalid, the frustum is not fully contained in the reconstructed mesh, or the labeled voxel population is insufficiently diverse. This yields dense per-frame occupancy grids where each voxel is one-hot encoded for one of 12 semantic categories: ceiling, floor, wall, window, chair, bed, sofa, table, TVs, furniture, generic objects, and free space [2407.11730][2504.14604][2603.14076].

## 2. Benchmark Structure and Data Splits

Occ-ScanNet releases two principal versions:

- **Full Dataset**: Training set of 45,755 frames and validation set of 19,764 frames. Roughly 100 frames are sampled per scene before filtering, drawn from $\approx655$ distinct ScanNet scenes.
- **Occ-ScanNet-Mini**: Subset for rapid prototyping, with 4,639 train and 2,007 validation frames [2407.11730][2605.25059].

Splits are scene-disjoint and maintain diversity across space types. Each frame provides:

- RGB image ($480 \times 640 \times 3$)
- Camera intrinsics and pose
- Dense semantic voxel grid ($60 \times 60 \times 36$), aligned to the current frustum
- (Optional: depth map)

Compared to earlier volumetric datasets such as NYUv2 (1,449 labeled frames, 13 labels), Occ-ScanNet is approximately 40$\times$ larger and offers greater variety in room structure, depth range (up to 10 m), and semantic granularity [2407.11730].

## 3. Occupancy Representation, Tasks, and Evaluation Metrics

### Occupancy Representation

For each frame, occupancy is defined over a cubic 3D voxel grid corresponding to the camera frustum, with each voxel assigned a discrete semantic class or marked as free space. In the local task, the grid encodes only the region visible to a single RGB frame. In the embodied/global setting, per-frame occupancy maps are integrated incrementally into a global map (covering sequential frames), using known or estimated camera poses to align and merge voxel grids [2605.25059][2504.14604].

### Core Tasks

- **Local semantic occupancy prediction:** Infer per-voxel semantic occupancy within the current camera frustum using a single RGB image and known camera parameters.
- **Embodied/global occupancy prediction:** Sequentially aggregate per-frame predictions to build a global semantic occupancy map as an agent moves through a trajectory [2605.25059][2603.14076].

### Evaluation Metrics

Standard metrics are defined as:

- **Per-class Intersection-over-Union (IoU):** For each class $c$,
  $$
  \mathrm{IoU}_c = \frac{\mathrm{TP}_c}{\mathrm{TP}_c + \mathrm{FP}_c + \mathrm{FN}_c}
  $$
  where $\mathrm{TP}_c$, $\mathrm{FP}_c$, and $\mathrm{FN}_c$ denote true positives, false positives, and false negatives for class $c$, respectively.

- **Mean IoU (mIoU):**
  $$
  \mathrm{mIoU} = \frac{1}{C}\sum_{c=0}^{C-1} \mathrm{IoU}_c
  $$
  with $C=12$ (semantic classes) [2504.14604][2605.25059].

- **Scene completion IoU (SC-IoU):**
  $$
  \mathrm{SC\text{-}IoU} = \frac{\lvert V_{\mathrm{pred}} \cap V_{\mathrm{gt}} \rvert}{\lvert V_{\mathrm{pred}} \cup V_{\mathrm{gt}} \rvert}
  $$
  where $V_{\mathrm{pred}}$ and $V_{\mathrm{gt}}$ are predicted and ground-truth occupied voxel sets [2603.14076].

- **Accuracy (Acc):**
  $$
  \mathrm{Acc} = \frac{1}{N}\sum_{i=1}^N \mathbb{1}(y_i = \hat y_i)
  $$
  with $y_i$ and $\hat y_i$ as ground-truth and predicted labels, respectively [2407.11730].

## 4. Methodological Advances Driven by Occ-ScanNet

Occ-ScanNet catalyzes research in monocular occupancy prediction by providing a high-resolution, semantically rich, and spatially well-aligned evaluation platform. Algorithms are required to estimate fine spatial geometry, model class imbalance (free space vs. objects), and resolve small-object and occlusion challenges.

### Representative Architectures

- **ISO [2407.11730]:** Introduces a Dual Feature Line of Sight Projection module to fuse 2D depth distributions and image features into the 3D voxel space. Designed to exploit pre-trained depth predictors for improved lifting of monocular features.
- **VEOcc [2605.25059]:** Proposes a recursive voxel-centric paradigm, eliminating scene-size priors for open-ended global mapping. Introduces Cross-Temporal Logit Aggregation (TLA), Reliability-Aware Confidence Modulation (RCM), and Confidence-Driven Incremental State Update (CSU) for robust online semantic fusion.
- **RoboOcc [2504.14604]:** Combines opacity-guided self-encoding and geometry-aware cross-encoding to address semantic ambiguity in overlapping Gaussians.
- **SGR-OCC [2603.14076]:** Employs a Soft-Gating Feature Lifter for depth-uncertainty-aware voxel lifting, Dynamic Ray-Constrained Anchor Refinement for sub-voxel geometric accuracy, and semantic-adaptive geometric regularization.

### Ablation and Qualitative Results

Leading methods demonstrate superior boundary preservation, improved recovery of fine structures such as windows, chair legs, and occluded objects, and resilience to noisy monocular depth [2605.25059][2603.14076]. SGR-OCC, for example, shows a +5.30 percentage point gain in SC-IoU over naive lifting due to soft-gating, and VEOcc outperforms prior SplatSSC on mIoU by +3.66 in local settings, with absolute gains up to nearly +10 on the Occ-ScanNet-Mini [2605.25059][2603.14076].

## 5. Comparative Performance and Benchmark Statistics

### Scale and Diversity

| Dataset               | Frames (train/val) | Voxel grid (per frame) | Classes |
|-----------------------|--------------------|------------------------|---------|
| Occ-ScanNet           | 45,755 / 19,764    | 60×60×36               | 12      |
| Occ-ScanNet-Mini      | 4,639 / 2,007      | 60×60×36               | 12      |
| NYUv2                 | 795 / 654          | 60×36×60 (downsampled) | 13      |

Occ-ScanNet comprises approximately 8.5 billion voxels overall, broadening scene diversity and depth coverage compared to previous indoor volumetric datasets [2407.11730].

### Leaderboard (Local Task, Validation Split)

| Method         | Input | IoU (%) | mIoU (%) | SC-IoU (%) |
|----------------|-------|---------|----------|------------|
| SGR-OCC        | RGB   | —       | 49.89    | 58.55      |
| VEOcc          | RGB   | 64.55   | 55.49    | —          |
| RoboOcc        | RGB   | 56.48   | 44.78    | —          |
| EmbodiedOcc++  | RGB   | —       | 46.20    | 54.90      |
| EmbodiedOcc    | RGB   | 53.95   | 42.90    | 53.55      |
| ISO            | RGB   | 42.16   | 24.61    | 42.16      |
| MonoScene      | RGB   | 41.60   | 19.84    | 41.60      |

This table samples results observed in the literature, with top performances achieved by VEOcc (mIoU=55.49), SGR-OCC (mIoU=49.89), and RoboOcc (mIoU=44.78) [2605.25059][2603.14076][2504.14604].

## 6. Distinctions from Related Benchmarks and Limitations

Occ-ScanNet differs from ScanNet by supplying spatially aligned dense occupancy labels rather than meshes or sparse point clouds and being formatted expressly for voxel-based monocular occupancy prediction. In contrast to EmbodiedOcc-ScanNet, which targets global spatiotemporal fusion, Occ-ScanNet’s primary focus is on the single-view, frustum-local occupancy estimation, though it also supports sequential (embodied) tasks [2504.14604][2605.25059].

Unique challenges stem from class imbalance, high sparsity of occupied voxels, limited frustum extent, clutter, and occlusion. The inclusion of small, fine-grained object classes (e.g., windows, TVs, and miscellaneous objects) requires methods to maintain high geometric precision [2504.14604].

## 7. Impact and Research Trajectory

Occ-ScanNet has established itself as a dominant benchmark for indoor semantic occupancy, underpinning numerous advances in monocular online mapping, voxel-centric representation learning, and embodied AI for spatial understanding. State-of-the-art methods such as VEOcc and SGR-OCC directly attribute their algorithmic development and validation to the availability of high-fidelity frame-aligned occupancy supervision presented in Occ-ScanNet [2605.25059][2603.14076]. A plausible implication is that future benchmarks may further expand embodied, interactive, and sensor-fusion tasks seeded by the Occ-ScanNet evaluation design and protocol.

Source: https://www.emergentmind.com/topics/occ-scannet