---
title: 'SOPE: 6D Pose, Size & Shape Dataset'
url: https://www.emergentmind.com/topics/sope-dataset
type: topic
---

# SOPE: 6D Pose, Size & Shape Dataset

SOPE (Synthetic Object Pose Estimation) is a large-scale synthetic dataset purpose-built to advance category-level 6D object pose, size, and dense shape estimation. It provides extensive high-fidelity RGB-D renderings, unified geometric annotations, and diversified semantic coverage across a wide range of everyday object categories. SOPE has become foundational in evaluating and pretraining deep models for open-set 6D understanding, especially in robotics and computer vision contexts where generalization beyond closed sets of templates or CAD models is central [2510.11687][2406.04316].

## 1. Purpose and Design Scope

SOPE was created to overcome the limitations of prior corpora that were constrained to few categories, template-based annotation, or lacked unified dense shape supervision. It is designed to:

- Enable large-scale, simulation-only training for category-level joint 6D pose, metric size, and dense 3D shape estimation.
- Cover a broad set of 149 object categories (e.g., dishes, utensils, tools, bottles, toys, electronics), structured to enable category-agnostic generalization.
- Provide every sample with all target labels—6D pose, size, and point-based ground-truth dense shape.

A primary motivation was to provide sufficient coverage for downstream zero-shot and cross-domain generalization tasks, as well as to facilitate synthetic-to-real transfer using paired synthetic and real benchmarks [2510.11687][2406.04316].

## 2. Dataset Composition and Statistics

| Property              | Value/Description             | Notes                                                         |
|-----------------------|------------------------------|---------------------------------------------------------------|
| Number of Categories  | 149                          | Broadest among category-level 6D datasets [2510.11687]        |
| Number of Instances   | 4,162 (Omni6DPose), 5,000    | Instance count varies slightly by paper [2406.04316][2510.11687] |
| Number of Frames      | 475,000 (Omni6DPose SOPE)    | Each frame: RGB-D, pose labels for all instances present      |
| Number of Annotations | 5,000,000+                   | One 6D annotation per object per frame                        |
| Per-category Coverage | ≥25,000 annotations/category | Over-sampling for transparent and specular categories         |
| Intended Split        | Train only                   | Evaluation occurs on real (ROPE) or mixed datasets            |

Most variant counts and summary statistics are directly specified in the Omni6DPose paper [2406.04316]. The SOPE dataset as described in [2510.11687] comprises 5,000 instances spanning all 149 categories, yielding an average of approximately 33 instances per category.

## 3. Rendering and Data Generation Pipeline

SOPE images are generated entirely in simulation, using physically plausible, randomized and context-rich pipelines:

- **RGB Images (Context-Aware Mixed Reality)**: Scene generation places each 3D object into a random background derived from large-scale indoor panoramas (Matterport3D, ScanNet++, IKEA), with ray-traced lighting and material property randomization. This guarantees physically accurate shadows, occlusions, and material/illumination effects by true light transport simulation—not raster overlays.
  
- **Depth Simulation**: Depth is rendered using structured-light/active stereo simulation, closely modeling the RealSense D415 sensor characteristics. IR dot patterns are rendered, disparities are evaluated via stereo matching, and zero-mean Gaussian noise (with variance as a quadratic function of depth) is introduced to mimic real sensor behavior.
  
- **Domain and Appearance Randomization**: Object materials, colors, light source temperature/intensity, distractor occlusion, and camera poses are heavily randomized. Camera viewpoints are sampled on a hemisphere (elevation: [0°,60°], azimuth: [0°,360°], distance: [0.5,1.5] m). Between 1 and 3 distractor objects are placed in front of the target, ensuring robust occlusion statistics.

- **Point Cloud Extraction**: From each simulated depth image, points corresponding to the target object are back-projected via camera intrinsics and sampled (1,024 points per partial view; 2,048 for full shapes when available) [2510.11687].

## 4. Annotation Schema and File Formats

SOPE supplies a unified multi-task annotation suite per sample:

- **6D Pose**: Rigid transform $p = (R, t) \in SE(3)$, where $R$ is a 3×3 rotation and $t \in \mathbb{R}^3$ is the canonical→camera translation. Format supports rotation matrices or equivalent quaternions; all annotations are given in the camera frame (z forward, y up).
    - **Symmetry-Aware Loss**: During training, all object symmetries are enumerated in a set $\mathcal{G}_R$, and the final loss is the minimum smooth L1 error over this set (per [2510.11687]).
    - **NOCS Alignment**: Object coordinates follow the NOCS canonicalization protocol.
    - **Example Annotation (JSON-like)**:
      ```json
      {
        "image_id": 12345,
        "object_id": "mug_007",
        "category": "mug",
        "R": [[0.98, -0.15, 0.12], [0.14, 0.99, -0.03], [-0.13, 0.05, 0.99]],
        "t": [0.23, -0.04, 0.88],
        "K": [[615,0,320],[0,615,240],[0,0,1]]
      }
      ``` 
      [2406.04316]

- **Object Size**: $s \in \mathbb{R}^3$, indicating the object's full extent along the x, y, z axes (in camera frame).

- **Dense Shape Supervision**: 
    - **Complete Shape**: Ground-truth point cloud $P_{\text{dense}}^{gt} \in \mathbb{R}^{2048\times 3}$.
    - **Partial Shape**: Input view-specific point cloud $P_{\text{partial}} \in \mathbb{R}^{1024\times 3}$, sampled from observed surface.
    - These facilitate joint learning of pose, size, and voxel/point-based 3D reconstruction [2510.11687].

- **Data Modalities**: Each scene/frame provides an RGB image, a channel-aligned simulated depth map, and per-object mask and annotation files. The camera intrinsics for all samples replicate a RealSense D415: $f_x = f_y \approx 615\, \text{px}$, $c_x \approx 320\, \text{px}$, $c_y \approx 240\, \text{px}$ [2406.04316].

## 5. Benchmarking, Metrics, and Comparative Analysis

SOPE's unified design enables benchmarking on single- and multi-task 6D understanding benchmarks. Key statistics (on "seen" categories with [2510.11687]'s method):

- **AUC (3D IoU at 25/50/75%):** 56.4 / 39.8 / 12.7
- **Mean Rotation Error:** 16°
- **Mean Translation Error:** 0.99 cm

These metrics exceed those of previous category-level methods, with marked improvements attributed to dataset scale, instance diversity, and unified dense shape supervision. Evaluation protocols for cross-domain testing are defined by training solely on SOPE and assessing direct transfer to real-world datasets, such as ROPE [2406.04316].

Compared to prior datasets:
- **Category Coverage:** 149 in SOPE vs. 12 (PhoCaL), 30 (T-LESS), ∼100 (HouseCat6D).
- **Instance Scale:** 4,162–5,000 models vs. a few hundred in earlier corpora.
- **Material Diversity:** ∼20% transparent, ∼10% specular, remainder diffuse.
- **Occlusion:** ∼30% of frames have >25% occlusion by distractors.

| Dataset      | Categories | Instances | Dense Shape | Context/Lighting | Occlusion            |
|--------------|------------|-----------|-------------|------------------|----------------------|
| SOPE         | 149        | 4,162–5k  | Yes (2048)  | Mixed-reality    | Distractors, 30%+    |
| PhoCaL       | 12         | -         | Unknown     | Simple           | Low                  |
| T-LESS       | 30         | -         | Unknown     | Factory/close-up | Low                  |
| HouseCat6D   | ~100       | -         | Unknown     | Simple           | Low                  |

## 6. Usage, Preprocessing, and Recommendations

SOPE is distributed for research under the CC BY-NC 4.0 license. Official release includes download links, intrinsics files, and code for scene export/import. Researchers should:

- Convert raw depth PNGs to metric point clouds using the provided camera matrix.
- Apply per-frame segmentation masks to isolate object pixels.
- Perform farthest point sampling (1,024 points) for partial clouds and resize RGB crops (e.g., to 224×224 for GenPose++ pipelines).
- Train models exclusively on SOPE synthetic data, with evaluation performed on cross-domain real benchmarks such as ROPE.

SOPE’s scalability, alignment with real-world object categories, and multi-modal annotation enable rigorous benchmarking for sim-to-real transfer, zero-shot recognition, and open-set 6D pose challenges [2510.11687][2406.04316].

## 7. Significance and Limitations

SOPE represents a milestone in dataset construction for category-level 6D understanding. Its primary innovations include scale, breadth of semantic coverage, unified annotation (pose, size, complete shape per instance), and physically plausible simulation pipelines. These properties facilitate the development and benchmarking of category-agnostic 6D pose estimators, especially those requiring dense shape signals for high-fidelity manipulation, grasp planning, and embodied AI.

Limitations include lack of native real sensor noise modeling in all variants (basic “simulated depth” only in [2510.11687]), absence of a predefined held-out test split (as SOPE is “train-only” by construction in [2406.04316]), and no explicit occlusion-severity annotations in every release. A plausible implication is that precise performance under extreme occlusion or rare material types should be cross-validated on real benchmarks (e.g., ROPE, HANDAL).

Overall, SOPE provides the scale, diversity, and annotation density needed to drive forward universal and category-agnostic 6D perception research, acting as the recommended pretraining corpus for next-generation pose, size, and shape estimation frameworks [2510.11687][2406.04316].

Source: https://www.emergentmind.com/topics/sope-dataset