HierOctFusion: Octree 3D Shape Diffusion

• HierOctFusion is a part-aware multi-scale octree diffusion model that leverages semantic part hierarchies to refine both global structure and fine-grained local details.
• It employs a two-stage denoising process using octree latents, VAE encoding, and SDF decoding with Marching Cubes for efficient and detailed mesh reconstruction.
• Empirical evaluations report modest FID improvements over OctFusion, indicating enhanced representation of complex parts such as airplane engines and car tires.

Searching arXiv for HierOctFusion and closely related octree-based 3D shape generation papers to ground the article in current literature. arXiv search query: "HierOctFusion octree diffusion 3D shape generation OctFusion LAS-Diffusion" HierOctFusion is a part-aware multi-scale octree diffusion model for 3D shape generation that treats objects not as purely holistic volumes but as sparse structures organized by a semantic part–whole hierarchy. It combines octree-based hierarchical generation with cross-attention conditioning so that part-level information can influence both global structure and local refinement. The model operates on octree latents, decodes them through a signed distance field (SDF) representation, and reconstructs meshes with Marching Cubes. Its stated objective is to improve fine-grained shape quality and efficiency relative to prior octree-based diffusion methods while preserving the sparsity advantages of layered octree generation (Gao et al., 14 Aug 2025).

1. Problem setting and conceptual motivation

HierOctFusion is motivated by two limitations attributed to prior octree diffusion models. First, existing methods typically model 3D objects as holistic entities, which ignores semantic part hierarchies and limits generalization. Second, holistic high-resolution modeling is computationally expensive, even though real-world 3D objects are sparse and hierarchical, with fine detail concentrated in relatively few spatial regions. The method therefore adopts a layered, coarse-to-fine octree strategy and injects part-level cues into the denoising process so that generation better matches the local-to-global organization of object structure (Gao et al., 14 Aug 2025).

Within this formulation, sparsity is not merely a memory-saving device; it is treated as a structural prior. Octrees recursively partition space and refine only where needed, making them well suited to objects whose geometry is unevenly distributed. HierOctFusion extends this premise by asserting that refinement should also be semantically guided. This suggests that octree depth and semantic structure are treated as coupled hierarchies rather than as independent aspects of the generative process.

The paper frames this design as especially relevant for fine-grained local regions. Holistic generation can distribute capacity uniformly across space, whereas part-aware conditioning can steer capacity toward semantically important regions, such as airplane engines or car tires. A plausible implication is that the method aims to improve both local fidelity and out-of-distribution compositional generalization by aligning geometric refinement with semantic decomposition.

2. Octree representation and two-stage multi-scale denoising

HierOctFusion begins from meshes sampled into dense point clouds and voxelized into octrees via recursive spatial partitioning. Each leaf node is associated with a latent feature. A pre-trained VAE encodes each leaf node, and the diffusion model refines these latents through denoising. At decoding time, leaf latents are mapped by an MLP to an SDF, and the SDF is converted to a mesh via Marching Cubes (Gao et al., 14 Aug 2025).

The denoising pipeline is explicitly multi-scale and consists of two sequential stages sharing a single octree-aware U-Net backbone. The coarse stage, denoted $F_1$, starts from noisy octrees at depth 4 and denoises to depth 6, with emphasis on global structure and coarse geometry. The fine stage, denoted $F_2$, starts from noisy depth-6 octrees and denoises to depth 8, focusing on fine-grained refinement and deeper octree subdivisions. The overall process is written as

$x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$

where $d_1$ and $d_2$ denote the maximum depths of the coarse and fine scales respectively. The final depth is capped at 8 to balance fidelity and efficiency.

This decomposition gives the model a clear division of labor. Coarser levels organize the overall shape layout, while finer levels address local semantic detail. Because only deeper regions are refined where necessary, the method preserves the sparse computational pattern typical of octree-based generation. The paper explicitly treats this layered refinement as more compatible with the structure of real objects than holistic high-resolution modeling.

The diffusion process follows the canonical Gaussian forward noising formulation

$$q(x_t|x_{t-1}) = \mathcal{N}\!\big(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t \mathbf{I}\big),$$

with closed-form noisy sample

$$x_t = \sqrt{\alpha_t}x_0 + \sqrt{1-\alpha_t}\,\epsilon,$$

where $\alpha_t = \prod_{i=0}^t(1-\beta_i)$ and $\epsilon \sim \mathcal{N}(0,\mathbf{I})$. Training regresses $x_0$ from $F_2$0 using

$F_2$1

The exact noise schedule is not specified beyond $F_2$2 being hyperparameters, and no classifier-free guidance or auxiliary guidance equations are reported.

3. Architecture and part–whole hierarchy message passing

The core network is a single octree-aware U-Net adapted to hierarchical 3D generation. It includes residual blocks, self-attention, downsampling, and upsampling in the coarse scale, together with octree-specific adaptations. In the fine scale, graph residual blocks are employed to better exploit local connectivity in the octree. The architectural distinction between the two stages is important: the coarse stage emphasizes global organization, whereas the fine stage intensifies local neighborhood reasoning (Gao et al., 14 Aug 2025).

HierOctFusion’s defining mechanism is part–whole hierarchy message passing implemented through cross-attention conditioning at each depth. The paper does not provide explicit bottom-up or top-down recursion equations, but it conceptually describes semantic guidance as flowing from parts to the whole during coarse generation and from whole to parts during fine refinement. This is operationalized by inserting cross-attention modules after each self-attention layer in $F_2$3 and after each graph residual block in $F_2$4.

The cross-attention module fuses octree-node features with part embeddings. Octree features serve as queries, and part features serve as keys and values:

$F_2$5

where $F_2$6 are octree features projected to queries, $F_2$7 and $F_2$8 are part features projected to keys and values, and $F_2$9 is the key/query dimensionality. The updated representation overwrites or enriches octree node features. Applied per octree node, this mechanism retrieves relevant part features and updates local geometry-aware latents.

The placement of these modules encodes the model’s interpretation of hierarchy. In the coarse stage, part-aware signals are injected early, after self-attention, to influence global form. In the fine stage, they are inserted after graph residual blocks to strengthen local semantic guidance where detailed part refinement occurs. The paper’s ablation results support this placement strategy: the largest gain arises when the part-aware module is present in the fine scale, and the best result is obtained when it is used in both scales.

4. Semantic conditioning and ShapeNet-Seg construction

Part-level conditioning is derived from ShapeNet-Seg, a dataset constructed for the method using a pre-trained and fine-tuned DGCNN segmentation backbone. The pipeline is specified as meshes to dense point clouds of approximately 200k points, downsampling to 2,048 points via farthest point sampling, DGCNN segmentation, and per-point label assignment. Category-specific part definitions are fixed at four parts each for airplane, car, and chair, and three parts for table (Gao et al., 14 Aug 2025).

During training, the DGCNN encoder extracts part-level feature descriptors for the segmented parts. These embeddings are then fed into the cross-attention blocks. In the paper’s interpretation, early injection in $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$0 helps establish category priors and overall shape layout, whereas deep injection in $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$1 sharpens local geometry in semantically complex regions. The examples given include airplane engines and car tires.

The base data source is ShapeNet, described as approximately 50k meshes across 55 categories, while evaluation follows OctFusion and is restricted to four classes: airplane at approximately 4,000 instances, car at approximately 7,500, chair at approximately 6,800, and table at approximately 8,400. The paper states that ShapeNet-Seg is released and provides extensive part labels for rigorous training and evaluation.

Several implementation details remain unspecified. The paper does not provide the optimizer, learning rate, batch size, training time, or regularization settings. It also does not specify auxiliary losses such as occupancy or hierarchical consistency. These omissions limit exact reproducibility, although the structural training setup is described at the level of the U-Net backbone, VAE leaf encoding, cross-attention placement, MLP SDF decoding, and Marching Cubes reconstruction.

5. Empirical evaluation and ablation findings

The primary reported metric is shading-image-based FID, used as in prior work to measure visual fidelity and diversity of rendered shading images. The paper also presents qualitative nearest-neighbor retrieval by Chamfer distance to show that generated samples differ from the closest training shapes. This is offered as evidence against memorization rather than as a full geometric evaluation suite (Gao et al., 14 Aug 2025).

A direct comparison with OctFusion shows consistent but numerically modest FID improvements across all four evaluated categories.

Category	OctFusion	HierOctFusion
Airplane	24.29	23.84
Car	78.00	77.13
Chair	16.15	16.07
Table	17.19	17.09

Relative to OctFusion, the paper reports improvements of $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$2 for airplane, $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$3 for car, $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$4 for chair, and $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$5 for table, corresponding approximately to $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$6, $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$7, $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$8, and $x_0 = \mathcal{F}_2(\mathcal{F}_1(x_T; d_1); d_2),$9 respectively. It also includes broader comparisons against IM-GAN, SDF-StyleGAN, Wavelet-Diffusion, and LAS-Diffusion, with HierOctFusion achieving the lowest reported FID in each of the four evaluated categories.

The ablation study isolates the effect of part–whole hierarchy modules on the chair category. The OctFusion-like baseline yields FID 16.15. Adding cross-attention only in the coarse scale gives 16.14; only in the fine scale gives 16.10; and in both scales gives 16.07. The paper concludes that most of the gain comes from placing the part-aware module in the fine scale, where local refinement occurs, while adding it in the coarse stage alone yields only marginal improvement.

Qualitatively, the reported improvements are concentrated in fine-grained parts. The paper specifically highlights better intake and exhaust shape and mounting details on airplane engines, improved tire treads on cars, and crisper part boundaries in shading images relative to OctFusion and LAS-Diffusion. It also states that nearest-neighbor retrieval by Chamfer distance shows generated samples differing from the closest training shapes, which the authors interpret as indicating avoidance of memorization.

6. Efficiency profile, limitations, and terminological scope

HierOctFusion preserves the same octree node counts as OctFusion at both scales, which the paper uses to argue that its part-aware modules do not alter sparsity patterns or subdivision behavior. On an NVIDIA 3090 with batch size 1, the reported memory and inference profile is as follows (Gao et al., 14 Aug 2025).

Method	Memory	Inference
LAS-Diffusion	1.06G	66.1ms
XCube	12.76G	135.3ms
OctFusion	0.69G	48.2ms
HierOctFusion	0.83G	54.5ms

The paper reports coarse and fine node counts of 4,096 and 11,634 for both OctFusion and HierOctFusion. Relative to OctFusion, HierOctFusion incurs an overhead of $d_1$0G memory, approximately $d_1$1, and $d_1$2ms inference time, approximately $d_1$3. This overhead is attributed to lightweight cross-attention blocks. Compared to LAS-Diffusion and XCube, HierOctFusion is described as having substantially lower active node counts and faster inference with less memory.

The stated limitations are largely tied to conditioning quality, evaluation scope, and representational bounds. The approach depends on DGCNN-derived part labels, so mis-segmentations or ambiguous structures may propagate errors. The octree depth cap at 8 may constrain extremely thin or ultra-high-frequency details. Evaluation is centered on shading-image FID, while geometric metrics beyond nearest-neighbor visualization are not reported. The experiments are restricted to four ShapeNet categories, so broader category coverage and open-world semantics remain untested. Future directions suggested in the paper include full scene generation, interactive 3D object editing, and further optimization for scalability, semantic control, and robustness to segmentation errors.

The name should also be kept distinct from several unrelated lines of work. It is not the fusion-rule framework for hierarchical tilings studied in "Fusion: A general framework for hierarchical tilings" (Frank, 2013), not the hybrid voxel-octree incremental reconstruction system HVOFusion (Liu et al., 2024), and not the cross-modal OCT/OCTA diagnostic fusion framework for diabetic retinopathy (Badhon et al., 4 Jul 2026). In current arXiv usage, HierOctFusion specifically denotes the part-aware multi-scale octree diffusion model for 3D shape generation introduced in 2025 (Gao et al., 14 Aug 2025).