---
title: Humanoid Occupancy in 3D Robotics
url: https://www.emergentmind.com/topics/humanoid-occupancy
type: topic
---

# Humanoid Occupancy in 3D Robotics

Searching arXiv for recent papers on humanoid occupancy and closely related occupancy perception benchmarks.
Humanoid occupancy denotes a family of occupancy-based representations, datasets, and perception systems tailored to humanoid robots, in which three-dimensional space is discretized or implicitly parameterized so that each spatial element encodes occupancy and, in many formulations, semantic class information. In recent work, the term specifically refers to a generalized multimodal occupancy perception system that integrates hardware and software components, data acquisition devices, sensor layout, and a dedicated annotation pipeline for humanoid robots [2507.20217]. Adjacent literature broadens the concept in two directions: panoramic and stereo surround occupancy for embodied platforms [2511.03571], [2606.22971], and predictive self-modeling in which a humanoid maps joint configurations to three-dimensional body occupancy [2606.13222]. The resulting field spans scene understanding for navigation and manipulation, human-aware occupancy in pedestrian-rich environments [2511.16949], and bodily self-representation, all under the shared premise that occupancy is a unifying intermediate for downstream decision-making.

## 1. Formalization and conceptual scope

In the explicit grid-based formulation used by Humanoid Occupancy, $\mathbb{R}^3$ is discretized into a voxel grid of size $X\times Y\times Z$, with occupancy probability
$$
o_{ijk}^{t}=P\bigl(\mathrm{O}(x_i,y_j,z_k,t)=1\bigr)\in[0,1]
$$
and semantic-label distribution
$$
s_{ijk,c}^{t}=P\bigl(Y(x_i,y_j,z_k,t)=c\bigr),\quad c\in\{1,\dots,C\}.
$$
A discrete occupancy-semantic tensor is then written as
$$
\mathbf{G}^{t}=\bigl\{\bigl(o_{ijk}^{t},\arg\max_c s_{ijk,c}^{t}\bigr)\bigr\}_{i,j,k},
$$
so that occupancy status and semantic identity are represented jointly rather than as separate maps [2507.20217].

This representation has close analogues in related datasets. OneOcc predicts a voxelized semantic occupancy grid $\mathbf{S}\in\{0,\ldots,C\}^{X\times Y\times Z}$ with per-voxel logits $\mathbf{Z}\in\mathbb{R}^{X\times Y\times Z\times C}$ and supports both Cartesian and cylindrical-polar discretizations to match panoramic sensing geometry [2511.03571]. MobileOcc models occupancy as a function $f:\mathbb{R}^3\to[0,1]^C$ with $C=10$ for nine semantic classes plus free space, while also storing pedestrian instance identities within the grid [2511.16949]. Humanoid-OmniOcc uses a robot-centric grid of $[44\times384\times384]$ voxels at $\Delta=0.04\,\mathrm{m}/\text{voxel}$ and predicts binary occupancy $\hat O\in\{0,1\}^{X\times Y\times Z}$, with an extended semantic version containing 15 channels [2606.22971].

A distinct but closely related line replaces discrete semantic scene voxels with an implicit body-occupancy field. In the self-other distinction framework of Chen et al., the target is a kinematics-free predictive self-model $f_\theta:\mathbb{R}^{29}\to\{O\}$, implemented through an implicit network that maps a query point $x$, ray direction $d$, and full robot state $S$ to density $\sigma$ and visibility $v$ via $(\sigma,v)=F_\phi(x,d,S)$ [2606.13222]. This suggests that humanoid occupancy is not restricted to exteroceptive scene completion; it also encompasses endogenous body-space modeling when occupancy is defined as “where the robot body is” under configuration $q$.

Historically, humanoid use of occupancy voxel grids predates these large-scale benchmarks. Wada et al. discretized workspace around shelf bins at resolution $r=0.005\,\mathrm{m}$, projected per-pixel object probabilities into a 3D voxel grid, fused them with log-odds updates, and extracted connected components for bin-picking; after 5 views, their occupancy-fusion method reported voxel precision $=0.90$, recall $=0.87$, and an 82% picking success rate [2001.05406]. That earlier formulation was manipulation-centric and object-specific, whereas later humanoid occupancy systems generalize to full-scene semantic perception.

## 2. Sensor configurations and dataset regimes

A central theme in humanoid occupancy research is that sensor layout cannot be inherited directly from autonomous driving. Humanoid Occupancy formulates sensor placement as an optimization
$$
S^*=\arg\max_S \bigl[C(S)-\lambda E(S)\bigr],
$$
where $C(S)$ measures coverage and $E(S)$ penalizes overlap and self-occlusion. The reported solution uses six cameras with $FOV_h=118^\circ\times FOV_v=92^\circ$ plus one $360^\circ$ LiDAR on a stabilizing neck mount, yielding $\ge 95\%$ horizontal coverage with minimal self-occlusion [2507.20217]. The associated data collection platform is a wearable head-rig with identical 6 RGB cameras and a 40-line $360^\circ$ LiDAR, a collector height of $\sim160$ cm, and a neck stabilizer to suppress shake.

Other systems adopt different sensing trade-offs while targeting humanoid or humanoid-adjacent embodiments.

| Resource | Sensor configuration | Data regime |
|---|---|---|
| Humanoid Occupancy [2507.20217] | 6 RGB cameras + 40-line $360^\circ$ LiDAR | Home / Industrial / Outdoor clips |
| OneOcc [2511.03571] | Single panoramic camera | QuadOcc and Human360Occ benchmarks |
| Humanoid-OmniOcc [2606.22971] | Four stereo rigs at $0^\circ$, $90^\circ$, $180^\circ$, $270^\circ$ yaw | 15 simulated indoor scenes + 5 real environments |
| MobileOcc [2511.16949] | Front stereo camera + dense Ouster-style LiDAR | Outdoor pedestrian-rich campus trajectories |

OneOcc is explicitly designed for legged and humanoid robots with a single panoramic camera, emphasizing gait-introduced body jitter and $360^\circ$ continuity [2511.03571]. Humanoid-OmniOcc instead follows a surround-stereo design derived from the exact sensor specifications of a Unitree G1 humanoid head: four stereo rigs with baseline $b=6$ cm, focal length $f=596.81$ px, horizontal FoV $\approx106^\circ$, vertical FoV $\approx86^\circ$, and rectified image size $1280\times1080$ with FoV $\approx93^\circ\times83^\circ$ after rectification [2606.22971]. Its Real2Sim2Real paradigm is defined by real sensor specifications driving physically accurate simulation, simulation generating annotated training data, and models trained in simulation being directly evaluated on real-world captures.

MobileOcc addresses a different but related deployment regime: mobile robots navigating densely pedestrian-populated, near-field outdoor scenes. It is built on the UT Campus Object Dataset, which provides time-synchronized front stereo RGB and dense LiDAR at 10 Hz; for occupancy annotation the streams are downsampled to 5 Hz, producing 116,511 frames, of which 37,622 contain at least one pedestrian [2511.16949]. Although not a humanoid dataset, its human-aware occupancy formulation is directly relevant to humanoid robots operating in shared pedestrian spaces.

## 3. Annotation pipelines and ground-truth generation

The defining difficulty in humanoid occupancy is not only voxel prediction but also voxel supervision. Humanoid Occupancy generates scene labels by combining dynamic-object handling with static-scene aggregation. Dynamic objects receive 3D bounding boxes for “ordinary” pedestrians, cyclists, and vehicles, alongside point-wise segmentation inside boxes for “special-pose” pedestrians. Static scenes are built by multi-frame LiDAR stitching,
$$
P^{\rm static}=\bigcup_{\tau=-K}^{K}T_{t\to t+\tau}\bigl(P_{t+\tau}\setminus P^{\rm dynamic}_{t+\tau}\bigr),
$$
followed by voxelization of the merged cloud and majority-vote semantic assignment per voxel [2507.20217]. This pipeline yields occupancy-semantic supervision over three scene types—Home, Industrial, and Outdoor—with 200 frames $\times$ (180 train + 20 val) clips and 8–13 semantic classes per scene.

Humanoid-OmniOcc uses a different annotation regime built around simulation-first labeling and LiDAR-based real-world verification. In simulation, each frame’s mesh is voxelized at $\Delta=1$ cm and then back-projected into the cameras for depth consistency, assigning label $0$ for free, $1$ for occupied, and $255$ for unknown according to
$$
\mathrm{Label}(v)=
\begin{cases}
0\;(\mathrm{free}), & d_v < d_{\mathrm{img}} - \Delta_v,\\
1\;(\mathrm{occupied}), & |d_v - d_{\mathrm{img}}|\le 2\,\Delta_v,\\
255\;(\mathrm{unknown}), & d_v > d_{\mathrm{img}} + \Delta_v.
\end{cases}
$$
In real environments, LiDAR point clouds are fused, denoised, voxelized into the same grid, and processed with a Bresenham ray-tracer to mark free, occupied, and unknown space within each camera’s FoV, after which semantic labels are manually assigned [2606.22971].

MobileOcc’s annotation pipeline is the most elaborate for human occupancy. Static occupancy is derived by removing dynamic pedestrian points from LiDAR returns inside 3D pedestrian detections, fusing the remaining points over time into a global OctoMap, and assigning final static semantic labels through per-class 2D semantic counts projected onto 3D points with a max-vote rule [2511.16949]. Human occupancy is then superimposed through a dedicated mesh optimization framework: an initial CLIFF estimate of SMPL parameters $\theta=\{\theta_{\rm glob},\theta_{\rm body}\}$, $\beta$, and $t_{\rm cam}$ is refined by visibility filtering, rigid ICP alignment against per-instance LiDAR points, and joint non-rigid optimization of $(\beta,\theta,t_{\rm cam})$ under a multi-term loss
$$
L_{\rm total}=L_J+\lambda_{3D}L_{3D}+\lambda_\theta L_\theta+\lambda_a L_a+\lambda_\beta L_\beta+\lambda_{\rm occ}L_{\theta,\rm occ}.
$$
The static map and refined SMPL meshes are finally voxelized at $0.02$ m resolution and transformed into a shared robot-local grid [2511.16949]. This is a human-aware annotation pipeline rather than a purely geometric occupancy pipeline, because it explicitly models deformable pedestrian geometry.

An important methodological implication is that humanoid occupancy labels are increasingly produced by hybrid procedures: geometric carving for free space, semantic voting for static structure, and model-based fitting for humans or articulated bodies. The literature suggests that annotation fidelity is becoming as decisive as network design.

## 4. Model architectures and fusion strategies

Humanoid occupancy models differ primarily in how they align heterogeneous sensors and lift 2D or point-set evidence into 3D. In Humanoid Occupancy, the camera branch uses a shared ResNet50 + FPN backbone, producing $\{C_2,C_3,C_4,C_5\}$ at strides $\{4,8,16,32\}$, while the LiDAR branch uses PointPillars to generate a BEV feature map $F_L$ [2507.20217]. Fusion is performed by deformable cross-attention, with LiDAR BEV features treated as queries and camera features as keys and values:
$$
F^{\rm fuse}(q)=\sum_{m=1}^{n_c}\sum_{l=1}^{L}
\mathrm{Softmax}\Bigl(\frac{Q_qK_{m,l}^T}{\sqrt{d}}\Bigr)\,V_{m,l}.
$$
Temporal integration follows a BEVDet4D-style design in which past BEV features are warped into the current frame and concatenated before a BEV encoder refines them. The final head reshapes channel features into height bins and predicts occupancy with a sigmoid branch and semantics with a softmax branch.

OneOcc addresses a different failure mode: panoramic sensing on legged or humanoid platforms with severe geometric discontinuities and gait jitter. Its architecture combines four modules [2511.03571]. Dual-Projection Equirectangular–Radial fusion maintains parallel encoders for the raw annular image and its equirectangular unfolding, preserving both native panoramic geometry and convolution-friendly continuity. Bi-Grid Voxelization constructs both Cartesian and cylindrical-polar 3D volumes and injects polar context into the Cartesian stream through precomputed index mappings. The Hierarchical AMoE-3D decoder is a depthwise-separable 3D U-Net that uses dual-path volumetric saliency and gradient-energy-driven mixture-of-experts routing. Gait Displacement Compensation predicts a small per-scale 2D warp $\Delta_s=(d_x,d_y)$ from pooled image features and shifts sampling coordinates during voxel lifting, thereby correcting feature-level motion misalignment without extra sensors.

Humanoid-OmniOcc’s HS model relies on stereo-guided depth priors to improve 2D-to-3D lifting [2606.22971]. A shared 2D backbone extracts left and right image features for each stereo rig, a disparity cost volume is built by feature correlation, and this cost volume is converted into a depth-aligned volume that yields a depth posterior
$$
P_i(u,v,z)=\sigma\bigl(-S_i^{\rm depth}(u,v,z)\bigr),\qquad
Z_i^l(u,v)=\sum_{z=1}^K P_i(u,v,z)\,z.
$$
Only the left-view features are then lifted into the robot-centric voxel grid through trilinear splatting weighted by the depth posterior. A lightweight 3D decoder maps the lifted tensor to occupancy logits, supervised by binary cross-entropy together with focal, geometric, lovász, and depth bin-wise terms.

A separate architectural branch appears in self-body occupancy modeling. Chen et al. use a part-aware proprioceptive encoder that partitions the 29 joint angles into torso, left and right arms, and left and right legs, encodes each group by small MLPs, and concatenates the results into a 256-D posture code [2606.13222]. Query points and ray directions receive sinusoidal positional encoding, and an implicit MLP with 6–8 hidden layers and two output heads predicts density and visibility. Training is driven first by proprioceptive–visual correspondence through an InfoNCE objective and then by silhouette reconstruction using volumetric rendering. This is not a scene-semantic architecture, but it is an occupancy architecture in the strict sense.

## 5. Benchmarks, metrics, and quantitative findings

Evaluation protocols in humanoid occupancy are centered on per-voxel IoU and mIoU, but individual systems introduce additional metrics that reflect specific deployment goals. Humanoid Occupancy reports mIoU over all voxels and classes, together with rayIoU computed along sampled LiDAR beams [2507.20217]. On its benchmark range $X,Y\in[-10,10]$ m and $Z\in[-1.5,0.9]$ m with voxel size $0.1$ m, single-frame camera-plus-LiDAR prediction achieves mIoU $=52.79\%$ and rayIoU $=60.49\%$, while the two-frame variant reaches mIoU $=55.73\%$ and rayIoU $=61.32\%$. Under identical training settings, BEVDet (camera only) gives mIoU $\approx47.9\%$, FB-Occ (camera only) $\approx47.4\%$, BEVFusion (camera + LiDAR, 1 frame) $\approx53.98\%$, and HumanoidOcc (camera + LiDAR, 1 frame) $\approx52.79\%$ with 40.5 M parameters versus 60.6 M. The reported two-frame model is described as best in both accuracy and efficiency.

Humanoid-OmniOcc evaluates voxel IoU, mean IoU, precision, and recall on both held-out simulation and real-world captures [2606.22971]. On the simulated test set, HS reports IoU $=29.67$ and mIoU $=11.69$, compared with FB-Occ at $(28.59,\,5.11)$, FlashOcc at $(18.05,\,1.71)$, SurroundOcc at $(24.71,\,6.86)$, and GaussianFormer at $(26.15,\,5.84)$. In real-world evaluation, the best monocular baseline, SurroundOcc, reports IoU $=20.35$ and mIoU $=8.89$, whereas HS reports IoU $=35.45$ and mIoU $=19.26$. The ablation on the stereo backbone shows that replacing the default FoundationStereo with LightStereo-S, COEX, or IGEV reduces real-world mIoU from 19.26 to as low as 6–9.

OneOcc evaluates on QuadOcc and Human360Occ using standard per-class IoU, mIoU, precision, and recall on non-empty voxels [2511.03571]. On QuadOcc, OneOcc achieves mIoU $=20.56\%$, precision $=66.69\%$, and recall $=64.74\%$, compared with LMSCNet at mIoU $=18.44\%$ and MonoScene at $19.19\%$. On Human360Occ, the within-city split yields mIoU $=37.29\%$ versus MonoScene at $33.46\%$, while the cross-city split yields $32.23\%$ versus $24.15\%$. These gains are attributed in the paper to the combined effect of dual projections, bi-grid reasoning, the AMoE-3D decoder, and gait compensation.

MobileOcc introduces a broader benchmark suite that includes dense semantic occupancy, panoptic occupancy, pedestrian detection, and pedestrian velocity prediction [2511.16949]. At $0.2$ m grid resolution, FlashOcc achieves the highest mIoU at approximately $32.5\%$, ahead of VoxFormer at $31.9\%$ and Panoptic-FlashOcc at $31.0\%$. For pedestrian IoU, FlashOcc reports $5.36\%$, VoxFormer $5.20\%$, and Panoptic-FlashOcc $3.66\%$. In panoptic evaluation, Panoptic-FlashOcc gives $PQ=19.9\%$, $PQ^\dagger=28.1\%$, and $AP^{Ped}=45.5\%$, while BEVDet4D detection-only yields $AP^{Ped}=41.7\%$. For velocity prediction, BEVDet4D reports $AVE$-$D=0.36$ m/s, and Panoptic-FlashOcc-vel reports $AVE$-$T=0.97$ m/s, $AVE$-$D=0.39$ m/s, and $AVE$-$O=0.67$ m/s with mIoU $=26.0\%$. The human mesh optimization stage is also evaluated independently: on 3DPW with synthetic Ouster LiDAR, “Ours (Ouster-64)” achieves MPJPE $=43.9$ mm and PA-MPJPE $=38.5$ mm without root alignment, while “Ours (Ouster-128)” reaches MPJPE $=39.1$ mm and PA-MPJPE $=35.1$ mm.

For self-body occupancy, Chen et al. evaluate IoU, MSE, MAE, and Chamfer Distance between predicted and ground-truth point clouds [2606.13222]. With oracle masks the self-model reaches IoU $\approx0.81$, MSE $\approx7\times10^{-3}$, MAE $\approx0.02$, and CD $\approx3.2$ mm; with their pseudo-ground-truth masks at 99.5% accuracy, performance remains near IoU $\approx0.80$, MSE $\approx8\times10^{-3}$, MAE $\approx0.025$, and CD $\approx3.4$ mm. At 80% mask accuracy, IoU drops to $\approx0.76$ and CD rises to $\approx4.2$ mm, while at 50% accuracy the model degrades substantially.

## 6. Applications, misconceptions, and open problems

The most direct downstream uses of humanoid occupancy are collision-aware locomotion, task-space manipulation, and map-centric navigation. Humanoid Occupancy states that its grid representation supports navigation and path planning with A* and D*, locomotion and footstep planning on uneven terrain, manipulation and grasping with semantic voxels such as “chair,” “table,” and “objects,” and teleoperation or mixed reality through dense occupancy reconstructions [2507.20217]. Humanoid-OmniOcc similarly identifies collision-aware locomotion via DWA and RRT*, arm-reach manipulation planning, and incremental map fusion for long-range navigation, revisiting, and loop-closing as target applications [2606.22971]. MobileOcc frames its contribution explicitly in terms of safer, more human-centered mobile robot navigation in pedestrian-rich near-field environments [2511.16949].

A recurring misconception is that humanoid occupancy is simply autonomous-driving semantic scene completion with a different robot body. The literature rejects that equivalence in several ways. Humanoid-OmniOcc argues that existing occupancy datasets are predominantly designed for autonomous driving with forward-facing cameras, far-field geometry, and static road priors, which limits applicability to embodied humanoid perception [2606.22971]. OneOcc addresses the specific panoramic continuity and gait-induced jitter of legged and humanoid embodiments [2511.03571]. Humanoid Occupancy emphasizes kinematic interference and self-occlusion in sensor layout [2507.20217]. These are not minor implementation differences; they alter sensing geometry, annotation design, and architectural choices.

Another misconception is that occupancy is only an external world model. The self-other distinction literature shows that a humanoid can learn self-other distinction from proprioceptive-visual correspondence and then train a predictive self-model that maps joint configurations to three-dimensional body occupancy, supporting target reaching, collision-aware motion planning, and human-to-robot motion retargeting [2606.13222]. This suggests that humanoid occupancy has bifurcated into two complementary regimes: exteroceptive scene occupancy and proprioceptively conditioned self-occupancy.

The open problems are equally consistent across papers. Humanoid Occupancy reports limited dataset scale and diversity, pose drift in temporal warping beyond two frames, and residual blind spots near hips and arms [2507.20217]. Humanoid-OmniOcc notes that dynamic agents are not annotated and that material reflectance mismatch can degrade depth priors in specular regions [2606.22971]. MobileOcc observes that vision-only near-field pedestrian occupancy remains weak, with pedestrian IoU below 10%, and that fine-grained motion disambiguation such as forward versus backward walking remains difficult [2511.16949]. In the self-modeling setting, performance depends strongly on pseudo-mask quality, and removing the visibility branch reduces IoU by about 5 points [2606.13222]. A plausible implication is that future humanoid occupancy systems will require simultaneous progress in multimodal sensing, temporally stable calibration, human-instance modeling, and scalable annotation.

Across these works, humanoid occupancy emerges as a general-purpose spatial intermediate rather than a single benchmark task. Its core promise lies in converting heterogeneous sensory streams and body-state signals into dense, robot-centric occupancy structure that is usable by planning, control, and interaction modules. The present literature shows that this promise is technically viable, but also that robustness to dynamic humans, embodiment-specific occlusions, and sim-to-real appearance gaps remains an active research frontier.

Source: https://www.emergentmind.com/topics/humanoid-occupancy