---
title: 3D Semantic Occupancy Graphs
url: https://www.emergentmind.com/topics/3d-semantic-occupancy-graphs
type: topic
---

# 3D Semantic Occupancy Graphs

A 3D semantic occupancy graph is a scene representation in which spatial regions are discretized volumetrically or hierarchically, annotated with both occupancy and category labels, and encoded in a data structure combining the topological properties of a graph with rich node and edge semantics. These graphs are central to contemporary research in embodied perception, autonomous driving, and scene understanding, enabling precise per-voxel or per-object reasoning while supporting downstream tasks such as navigation, path planning, and retrieval. Recent methodological advances have transitioned from dense voxel grids to adaptive, sparse, and graph-based data structures, incorporating deep learning, probabilistic modeling, and transformers for high fidelity and computational efficiency.

## 1. Foundations and Definitions

3D semantic occupancy graphs assign to each element of a spatial partition (voxel, Gaussian, octree cell, or instance) both an occupancy probability and a probability distribution over semantic classes. These elements become graph nodes, and are connected by edges reflecting geometric adjacency, semantic affinity, or spatial relationship. Node features typically include position, geometry, semantic representation, and class probabilities; edges may encode distance, direction, or learned relations. Compared to classical point clouds or voxel grids, this representation enables expressive modeling of both free and occupied space, boundary details, and object relationships. Notably, occupancy graphs decouple storage efficiency from spatial resolution by leveraging adaptive or sparse structures, facilitating real-time use in large-scale or open-vocabulary settings [2411.16253, 2506.14825, 2404.09502, 1707.07388].

## 2. Structural Representations and Node Construction

The construction of 3D semantic occupancy graphs differs by application and data modality. Four dominant paradigms are observed in recent literature:

- **Dense Voxel Grids (+CRF):** Grids centered at the agent are indexed in 3D, with per-voxel occupancy and class label probabilities updated incrementally using image priors, and organized into a graph for conditional random field (CRF) optimization [1707.07388].
- **Sparse Voxel Graphs:** Sparse COO representations retain only nonzero-occupancy voxels as nodes, each with position, features, occupancy, and semantic probabilities; adjacency is defined via spatial proximity (e.g., 6- or 26-connectivity), and edge weights depend on geometric and occupancy features [2404.09502].
- **Gaussian Splatting Graphs:** Learnable Gaussians parameterize local spatial extents, positions, and semantic features. Two graphs are built: a geometric graph via spatially adaptive radii (scaled by covariance), and a semantic graph based on top-M cosine similarities in feature space; graph transformer layers refine the set [2506.14825].
- **Adaptive Octree-Graphs:** Object instances are represented as root-centered octrees adaptively subdivided to match geometric complexity. Each octree acts as a node, storing a single semantic descriptor derived from fused vision-language features; edges encode spatial adjacency and directional relations [2411.16253].

The table below summarizes core node types:

| Approach              | Node Definition              | Node Features                                        |
|-----------------------|-----------------------------|------------------------------------------------------|
| Voxel-CRF [1707.07388] | Voxel (x, y, z)             | Log-odds, class probs, color                         |
| SparseOcc [2404.09502] | Active voxel (pᵢ)           | Feature fᵢ, occupancy oᵢ, semantic probs sᵢ           |
| GraphGSOcc [2506.14825]| Gaussian (μᵢ, Σᵢ)            | Position, extent, opacity, semantic vector, hᶠᵢ       |
| Octree-Graph [2411.16253]| Adaptive octree per object | Octree, center, semantic feature (IFA-aggregated)    |

## 3. Edge Construction and Graph Topology

Edge definition shapes the expressive power and sparsity of the resulting graph:

- **Spatial Adjacency:** In voxel approaches, connect each node to immediate neighbors (Chebyshev or Euclidean distance ≤ 1) for local region continuity [2404.09502, 1707.07388].
- **Geometric Proximity:** In Gaussian graphs, nodes are connected if Euclidean distance is below radius rᵢ adaptively set by covariance, optionally KNN-constrained [2506.14825].
- **Semantic Similarity:** For fine semantic relationships, connect nodes if their semantic feature vectors yield top-M cosine similarity [2506.14825].
- **Object Relation:** In instance- or octree-based graphs, edges link nodes whose centers lie within a threshold radius, and store distance, direction, and qualitative spatial relations (e.g., left, behind) [2411.16253].

These multi-criteria edge sets enable both local contextual reasoning and long-range semantic unification, as leveraged in dual-graph transformer architectures.

## 4. Graph-Based Neural Architectures

Semantic occupancy graphs serve as substrates for advanced neural network models, particularly transformer-based and CRF-based architectures:

- **Graph Transformers with Dual Graphs:** GraphGSOcc employs layers that alternately attend over geometric and semantic graphs, fusing their updates adaptively. Coarse-grained attention in lower layers captures object-level topology; finer-grained attention in higher layers sharpens boundaries [2506.14825].
    - Dynamic-static decoupling optimizes context propagation separately for movable and static objects, with independent projection weights and adaptive fusion per node.
- **Sparse Transformers:** SparseOcc decodes only nonzero voxels with a sparse transformer head, using trilinear feature interpolation and efficient attention masking, avoiding computation on empty space [2404.09502].
- **High-order Graphical Models:** CRF-based approaches define unary, pairwise, and high-order clique potentials over the voxel graph, with efficient mean-field inference and pairwise terms reflecting both feature and geometric compatibility [1707.07388].
- **Instance-level Aggregation:** Octree-Graph aggregates multi-view vision-language features at the instance level using self-attention and clustering, assigning this descriptor to the corresponding adaptive octree node [2411.16253].

A plausible implication is that graph-based neural modules can exploit rich geometric and semantic structure without incurring dense computation, facilitating higher fidelity at lower memory and compute budgets.

## 5. Training, Inference, and Decoding

Supervision and inference in 3D semantic occupancy graphs proceed in a manner closely tied to the underlying representation:

- **Gaussian Splatting:** After graph-transformed embedding refinement, occupancy probabilities for each class at a voxel are obtained by splatting the Gaussian density weighted by semantic logits:
  $$
  p(v,c) = \sum_{i=1}^N w_i(v) \cdot \mathrm{softmax}_c(W_o h_i)
  $$
  where $w_i(v) \propto \exp\big[-\frac{1}{2}(v-\mu_i)^\top \Sigma_i^{-1}(v-\mu_i)\big]$ [2506.14825].
- **Sparse Voxel Decoding:** SparseOcc scatters mask logits onto active voxel coordinates, aggregates via learned multi-scale interpolation, and classifies via a sparse transformer [2404.09502].
- **CRF Graphs:** Unary terms from 2D CNN segmentation guide label propagation; pairwise/high-order CRF terms penalize spatial, appearance, and instance-inconsistent assignments. Efficient mean-field updates implement iterative refinement [1707.07388].
- **Octree Path Queries:** Occupancy is queried per octree node or recursively within leaves. Semantic retrieval leverages learned descriptors and spatial relations [2411.16253].

Loss functions commonly include standard cross-entropy per voxel or node, with optional terms for geometric regularization, temporal consistency, and covariance penalty.

## 6. Empirical Performance and Benchmarks

Recent advancements demonstrate state-of-the-art accuracy and substantial efficiency gains by leveraging 3D semantic occupancy graphs:

- **GraphGSOcc:** On SurroundOcc-nuScenes (17 classes, voxel size 0.4 m³), achieves mIoU$=24.10\%$ (mean IoU$=35.21\%$), +1.97% mIoU over GaussianWorld, memory usage of 6.1 GB (–13.7%) [2506.14825].
- **SparseOcc:** On nuScenes-Occupancy, increases mIoU to $14.1\%$ (+1.3% over dense baseline) and reduces FLOPs by $74.9\%$. On SemanticKITTI, similar accuracy at half the computation [2404.09502].
- **Octree-Graph:** On ScanNet, boosts zero-shot 3D semantic segmentation mIoU by up to +17.1% and path-planning success at 0.25 m endpoints to 97%. Storage is reduced to <0.1 MB per object, versus >6 MB for point clouds [2411.16253].
- **Voxel-CRF:** On KITTI, hierarchical CRF over the voxel graph improves mean IoU by 15% over prior 3D systems, with mean-field updates feasible in near-real time on $2.5\times10^5$-voxel maps [1707.07388].

Ablative studies confirm the value of dual graph attention, multi-scale transformer hierarchies, and high-order CRF terms for both accuracy gains and robustness.

## 7. Applications and Significance

3D semantic occupancy graphs are integral to multiple downstream applications:

- **Autonomous Driving:** Provide explicit modeling of free and occupied space, essential for navigation, collision avoidance, and trajectory planning [2506.14825, 2404.09502].
- **Robotic Navigation:** Support efficient incremental mapping, semantic path planning, and dynamic/static object decoupling [1707.07388].
- **Open-vocabulary Scene Understanding:** Enable zero-shot semantic retrieval, object-centric queries, and language–vision integration in complex environments [2411.16253].
- **Graph-based Reasoning:** Facilitate efficient neural and probabilistic operations via GNNs, transformer heads, or CRFs, leveraging the structured graph substrate for high-level reasoning and generalization.

The recurring trend is a shift from dense, monolithic grids to lightweight, adaptive, and semantically expressive graph-based representations—balancing memory, accuracy, and versatility for a broad range of scene understanding tasks.

Source: https://www.emergentmind.com/topics/3d-semantic-occupancy-graphs