Voxel Diffusion Module (VDM)

• Voxel Diffusion Module (VDM) is a family of techniques that apply diffusion or propagation mechanisms directly on voxelized 3D data to generate, enhance, or complete spatial structures.
• It leverages diverse methodologies such as sparse 3D convolutions, discrete categorical diffusion, and latent diffusion to refine voxel features across various applications.
• Applications include point-cloud detection, scene generation, indoor arrangement, and medical image restoration, each employing explicit conditioning and geometric preservation.

Voxel Diffusion Module (VDM) is a context-dependent term used in recent arXiv literature for modules that operate in explicit voxel space and apply either diffusion-model machinery or diffusion-like propagation to generate, refine, complete, or enhance 3D data. In some papers, VDM is a denoising diffusion model over voxel occupancies or voxel tokens; in others, it is a sparse-convolutional feature-diffusion front end for voxelized LiDAR; and in still others, it is a graph-based reaction–diffusion simulator on connected voxels. This suggests that VDM is not a single standardized architecture, but a family of voxel-native mechanisms whose semantics depend on representation, task, and surrounding pipeline (Liu et al., 22 Aug 2025, Dahmani et al., 7 Apr 2026, Xiang et al., 8 May 2026, Xu et al., 2 Jun 2026, Mao et al., 16 May 2026, Sumuk, 1 Jan 2026, Jiang et al., 20 Apr 2026, Klai et al., 2024).

1. Scope and nomenclature

Across the literature, the same label is attached to substantially different technical objects. In point-cloud detection, the VDM in "A Unified Voxel Diffusion Module for Point Cloud 3D Object Detection" is a lightweight preprocessing block composed of sparse 3D convolutions, submanifold sparse convolutions, and residual connections; its purpose is to diffuse foreground voxel features and aggregate fine-grained spatial information before serialization into Transformer- or SSM-based backbones (Liu et al., 22 Aug 2025). In "SEM-ROVER: Semantic Voxel-Guided Diffusion for Large-Scale Driving Scene Generation," the voxel diffusion component is the core 3D generator that synthesizes an E-Voxfield grid from semantic voxel conditioning before deferred rendering (Dahmani et al., 7 Apr 2026). In "DVD: Discrete Voxel Diffusion for 3D Generation and Editing," it is a discrete diffusion prior for sparse voxel scaffolds in an SLat pipeline (Xiang et al., 8 May 2026). In "PatchScene: Patch-based Voxel Diffusion for Large-Scale Scene Completion," it is the core generative component that completes local overlapping voxel patches and then fuses them spatially and temporally (Xu et al., 2 Jun 2026). In "VoxScene: Anchor-Conditioned Voxel Diffusion for Indoor Scene Arrangement," it is a latent diffusion model over compressed object-centric voxel occupancies conditioned on anchors and context (Mao et al., 16 May 2026).

The term is also used, or mapped, in medical and restoration settings. "Mask-Conditioned Voxel Diffusion for Joint Geometry and Color Inpainting" defines a mask-conditioned 3D diffusion U-Net that inpaints damaged occupancy and RGB voxels at fixed $32^3$ resolution (Sumuk, 1 Jan 2026). "Structure-Adaptive Sparse Diffusion in Voxel Space for 3D Medical Image Enhancement" uses a sparse voxel-space diffusion core with Structure-aware Trajectory Modulation for denoising and super-resolution in CT, PET, and MRI (Jiang et al., 20 Apr 2026). In "VGDM: Vision-Guided Diffusion Model for Brain Tumor Detection and Segmentation," the paper does not explicitly name a separate submodule "VDM," but the methodology identifies a voxel-wise iterative correction mechanism that refines segmentation predictions through transformer-guided denoising (Behnam, 2 Oct 2025).

Context	VDM role	Voxel substrate
Point-cloud detection	feature diffusion before serialization	sparse voxel feature maps
Scene generation and completion	direct 3D generation or denoising	occupancy grids, patches, or E-Voxfield tokens
Indoor arrangement	anchor-conditioned latent synthesis	object-centric local voxel blocks
Medical restoration and enhancement	mask-conditioned or sparse denoising	occupancy/color grids or 3D image volumes
Porous-media simulation	physical diffusion–transformation	graph of connected voxels

A useful consequence of this terminological spread is that any discussion of a VDM must specify whether “diffusion” refers to probabilistic denoising, categorical diffusion, sparse convolutional feature spreading, or physical transport on a voxel graph.

2. Representational substrates and conditioning interfaces

A defining property of VDMs is their commitment to explicit voxel-space structure, but the voxel object itself differs markedly across papers. PatchScene represents a dense occupancy scene as $\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$, extracts overlapping patches $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$, and conditions each patch denoiser on the noisy patch, the timestep, and a learnable patch position encoding $\mathbf{p}_k$ (Xu et al., 2 Jun 2026). DVD uses a binary occupancy scaffold $\mathbf{X}\in\{0,1\}^{N\times N\times N}$ as stage 1 of an SLat pipeline, where active voxels define the spatial scaffold later populated by continuous latents (Xiang et al., 8 May 2026). VoxScene uses a global instance grid $G_o \in \mathbb{N}^{D \times W \times H}$, derives a local object-centric grid $G_l^i$, and remaps labels into a relative ternary state $\tau \in \{0,1,2\}$ for free space, contextual objects, and target object (Mao et al., 16 May 2026).

SEM-ROVER adopts a more elaborate voxel substrate. Each occupied voxel stores a fixed number $n$ of surface samples with relative 3D coordinates and RGB color,

$$V_i = \{(x_i^j, y_i^j, z_i^j), (r_i^j,g_i^j,b_i^j)\}_{j \in [1,n]},$$

and is flattened into $\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$0; in the final setup, $\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$1, so each token is 120-dimensional. Conditioning is provided by semantic labels $\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$2, voxel-center locations, and 3D positional encodings, while generation is performed over local sets of adjacent occupied voxels rather than over the full scene (Dahmani et al., 7 Apr 2026).

Medical VDMs attach conditioning directly to reconstruction structure. The mask-conditioned inpainting model constructs a 5-channel tensor

$\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$3

where masked occupancy and masked RGB are zeroed inside the predicted damage mask $\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$4 (Sumuk, 1 Jan 2026). The sparse medical-enhancement framework instead conditions on degraded input $\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$5 and learns a sparse denoising trajectory over overlapping 3D patches, while STM derives local structural descriptors from volumetric, coronal, and sagittal branches (Jiang et al., 20 Apr 2026). VGDM conditions on skull-stripped, intensity-normalized, isotropically resampled BraTS2020 MRI volumes with T1, T1ce, T2, and FLAIR modalities, and uses a lightweight decoder to map refined features back to voxel-wise tumor/background probabilities (Behnam, 2 Oct 2025).

These design choices indicate that “voxel” in VDM does not imply a single tensor convention. It may mean binary occupancy, occupancy-plus-color, semantic local neighborhoods, sparse feature maps, or compact per-voxel surface token sets. The conditioning interface is equally heterogeneous: semantics, masks, degraded inputs, anchors, previously generated context, pending anchors, prior frames, sparse point clouds, and 3D positions all appear as first-class controls in different systems.

3. Diffusion formulations and denoising strategies

Many VDMs use DDPM-style Gaussian corruption, but they instantiate different prediction targets and reverse-process parameterizations. VGDM follows the standard forward process

$\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$6

and learns the reverse transition

$\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$7

with a vision transformer or Swin Transformer backbone providing $\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$8 and $\mathbf{X}_0 \in \{0,1\}^{H \times W \times D}$9. The practical interpretation in the paper is iterative voxel-level refinement of segmentation predictions rather than unconditional image generation (Behnam, 2 Oct 2025). PatchScene uses the same Gaussian corruption family patch-wise, but predicts the clean patch $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$0 and reconstructs the noise estimate from the closed-form forward relation before taking a standard reverse DDPM step (Xu et al., 2 Jun 2026).

Other VDMs depart from noise prediction. The sparse medical-enhancement framework predicts $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$1 directly on the data manifold, yet supervises training in velocity space using

$$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$2

and restricts both training and inference to a uniformly subsampled sparse schedule $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$3 with $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$4 in experiments (Jiang et al., 20 Apr 2026). VoxScene likewise adopts velocity parameterization in latent space after VQ-VAE compression, with a 3D U-Net predicting

$$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$5

from noisy latent $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$6, timestep $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$7, and rasterized context $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$8 (Mao et al., 16 May 2026).

DVD represents the most explicit divergence from continuous Gaussian diffusion. It models each voxel token as a categorical variable with $$\mathbf{x}_0^{(k)} \in \{0,1\}^{h \times w \times d}$$9 states, empty or occupied, and uses a uniform state discrete diffusion model with

$\mathbf{p}_k$0

The reverse process predicts a categorical distribution over clean states $\mathbf{p}_k$1, samples ancestrally over a time grid $\mathbf{p}_k$2, and exposes predictive entropy as an uncertainty signal for voxel-wise ambiguity and shape-level assessment (Xiang et al., 8 May 2026).

A further distinction appears in physical simulation. The voxel graph-based approach for soil decomposition formulates diffusion on a graph of 6-connected pore voxels via

$\mathbf{p}_k$3

where $\mathbf{p}_k$4 is the graph Laplacian and $\mathbf{p}_k$5 is the vector of dissolved organic matter masses. It combines this graph diffusion with local biological transformations, and compares explicit, implicit, synchronous, and asynchronous numerical schemes (Klai et al., 2024). This use of “diffusion” is physically literal rather than generative; it broadens the semantic range of VDM beyond score-based or denoising models.

4. Generative synthesis, completion, and scene arrangement

In large-scale 3D generation, VDMs are often the principal scene generator rather than an auxiliary post-processor. SEM-ROVER generates the 3D scene directly in voxel space from a coarse semantic voxel layout, using a semantic-conditioned diffusion model over local E-Voxfield neighborhoods with 3D positional encodings. The denoiser is a 1D Diffusion Transformer with masked attention over voxel tokens; in the supplementary architecture it has 12 transformer layers, 8 attention heads, hidden dimension 1024, and input token dimension 120. Attention is masked to a 3-meter neighborhood, and large scenes are synthesized by progressive spatial outpainting with overlapping regions and the RePaint scheduler, keeping each denoising step constant-cost for $\mathbf{p}_k$6 to $\mathbf{p}_k$7 voxels. The resulting system generates scenes spanning over $\mathbf{p}_k$8 m$\mathbf{p}_k$9 and tens of thousands of voxels, with reported memory and runtime of 8 GB VRAM and about 20 minutes per scene (Dahmani et al., 7 Apr 2026).

PatchScene addresses the same scale problem with a different decomposition. Rather than diffuse a global latent or a dense full-scene grid, it denoises overlapping local occupancy patches and then fuses them. Spatial coherence is handled by stochastic coupling of local and global noise estimates in overlap regions,

$\mathbf{X}\in\{0,1\}^{N\times N\times N}$0

with $\mathbf{X}\in\{0,1\}^{N\times N\times N}$1, while temporal coherence is obtained by ICP-based alignment and adaptive fusion across adjacent frames. The method additionally imposes Annular-Flow outward generation over concentric LiDAR rings, beginning from dense near-range observations and propagating outward to sparse regions. The paper reports that a model trained on 20 m LiDAR ranges generalizes effectively to 50 m scenes without retraining, and that 10 denoising timesteps provide the preferred trade-off between quality, boundary coherence, and efficiency (Xu et al., 2 Jun 2026).

DVD occupies a different point in the design space: it is a first-stage scaffold prior for 3D generation and editing. Because occupancy is modeled natively as a binary categorical variable, it avoids continuous-to-discrete thresholding, provides explicit voxel-wise probabilities, and exposes predictive entropy for uncertainty estimation. The same discrete sampler is extended to editing and inpainting through block-structured perturbation fine-tuning, where corruption is applied inside randomly sampled axis-aligned hypercubes at multiple scales so that masked completion can be achieved in a single sampling trajectory without RePaint-style iterative resampling (Xiang et al., 8 May 2026).

VoxScene applies voxel diffusion to indoor arrangement rather than scene completion. It sequentially synthesizes local object-centric voxel occupancies conditioned on already generated objects, the current target anchor, and pending anchors. To make direct voxel diffusion tractable, a 3D fully convolutional VQ-VAE compresses local blocks by a factor of 4; diffusion then proceeds in latent space under anchor-conditioned context. A central geometric constraint is explicit voxel exclusivity, $\mathbf{X}\in\{0,1\}^{N\times N\times N}$2, enforced during voxelization and preserved at write-back by allowing the target object to occupy only remaining free space in the global grid. The generated occupancies are then used as geometric queries for CAD retrieval through Soft Chamfer Score matching, linking voxel diffusion to downstream asset instantiation (Mao et al., 16 May 2026).

5. Medical reconstruction, segmentation, and perception-oriented variants

In medical imaging, VDMs are frequently cast as conditional refiners rather than free-form generators. VGDM treats diffusion as iterative denoising of segmentation predictions on BraTS2020 MRI. The denoiser is transformer-driven rather than U-Net-based: noisy inputs are divided into non-overlapping patches, linearly projected, and processed by transformer encoder layers with multi-head self-attention, with a Swin Transformer backbone adopted for efficiency and hierarchical features. After denoising, a lightweight decoder outputs tumor/background probability maps. Training uses a composite loss that combines BCE, Dice, and BoundaryLoss, and the paper reports BraTS2020 results of Dice 95.7 and HD95 4.3 mm for VGDM, compared with 90.8 and 8.1 mm for U-Net, 92.6 and 6.8 mm for TransBTS, and 93.4 and 5.9 mm for DMCIE (Behnam, 2 Oct 2025).

The mask-conditioned inpainting framework separates damage localization from reconstruction. A 2D U-Net predicts slice-wise damage masks, which are stacked, combined with logical OR, and post-processed by 3D morphological closing to obtain $\mathbf{X}\in\{0,1\}^{N\times N\times N}$3. A 3D diffusion U-Net with four resolution levels then reconstructs occupancy and color directly on a $\mathbf{X}\in\{0,1\}^{N\times N\times N}$4 voxel grid from masked occupancy, masked color, and the predicted mask. The model uses $\mathbf{X}\in\{0,1\}^{N\times N\times N}$5 timesteps with a linear beta schedule from $\mathbf{X}\in\{0,1\}^{N\times N\times N}$6 to $\mathbf{X}\in\{0,1\}^{N\times N\times N}$7, and the total objective combines diffusion noise-prediction loss, occupancy BCE, masked color $\mathbf{X}\in\{0,1\}^{N\times N\times N}$8, perceptual regularization, and a weak color prior, with stated weights 1.0 for occupancy BCE, 20.0 for color $\mathbf{X}\in\{0,1\}^{N\times N\times N}$9, and 0.1 for perceptual and prior terms (Sumuk, 1 Jan 2026).

The sparse medical-enhancement framework is motivated by the claim that 3D denoising and super-resolution are conditional reconstruction problems with strong anatomical priors already present in degraded input. On that basis, it trains and samples only on a compact set of uniformly subsampled timesteps, predicts clean data directly in voxel space, and uses Structure-aware Trajectory Modulation to recalibrate time embeddings at each residual block based on local anatomy. The paper reports up to $G_o \in \mathbb{N}^{D \times W \times H}$0 training acceleration and state-of-the-art or second-best results across PSNR, SSIM, MS-SSIM, and HFEN on four datasets spanning CT, PET, and MRI (Jiang et al., 20 Apr 2026).

Outside medical imaging, perception-oriented VDMs reinterpret “diffusion” as spatial feature spreading. The point-cloud detection VDM is inserted before voxel sequence serialization and alternates SubM3D layers, SRB-3D modules, and two SPConv-3D layers. The output feature map is reduced to one-fourth of the input resolution, simultaneously enriching local context and lowering token count for the downstream sequence model. Embedded into DSVT and LION, the method reports 74.7 mAPH (L2) on Waymo, 72.9 NDS on nuScenes, 42.3 mAP on Argoverse 2, and 67.6 mAP on ONCE; an ablation on Waymo attributes a 1.3 mAPH (L2) gain to the diffusion component alone (Liu et al., 22 Aug 2025).

A closely related voxel branch appears in PVNet, although the paper does not explicitly name it VDM. There the voxel-side module voxelizes the synthesized point cloud into a $G_o \in \mathbb{N}^{D \times W \times H}$1 grid, initializes voxel features with a two-layer MLP using voxelized point features, noisy point coordinates, and timestep information, and then refines incomplete voxel features by a small-stride 3D convolution block followed by three hierarchical Multi-Path ResBlocks and a lightweight 2D U-Net borrowed from LMSCNet. The completed voxel features are fused with point features through a point-voxel interaction module; the reported ablation improves F-Score from 1.966 without point-voxel interaction to 8.021 with it, and the full voxel completion plus U-Net setting yields CD 0.425 and F-Score 8.021 (Cheng et al., 23 Aug 2025).

6. Benefits, misconceptions, and limitations

A recurrent misconception is to treat every VDM as a denoising diffusion probabilistic model. The literature does not support that simplification. In point-cloud detection, VDM denotes sparse 3D convolutional feature diffusion rather than probabilistic sampling (Liu et al., 22 Aug 2025). In the soil-decomposition simulator, diffusion is Fickian transport on a graph of connected voxels governed by a graph Laplacian and coupled to local microbial kinetics; it achieves comparable results to LBM-based simulations with one-fourth of the computing time, while remaining slower than Mosaic but more accurate and calibration-free (Klai et al., 2024). Even within probabilistic models, some VDMs are continuous Gaussian DDPMs, some are discrete categorical diffusions, and some are latent diffusion systems over VQ-VAE codes (Xiang et al., 8 May 2026, Mao et al., 16 May 2026).

Despite this heterogeneity, several common design patterns recur. One is locality for tractability: SEM-ROVER diffuses over bounded local voxel neighborhoods, PatchScene over local overlapping patches, VoxScene over local object-centric blocks, and the sparse medical-enhancement framework over a compact set of timesteps rather than a dense trajectory (Dahmani et al., 7 Apr 2026, Xu et al., 2 Jun 2026, Mao et al., 16 May 2026, Jiang et al., 20 Apr 2026). A second is explicit conditioning: semantic labels and 3D positions in SEM-ROVER, patch position encodings and prior frames in PatchScene, anchors and pending objects in VoxScene, masks in volumetric inpainting, and degraded inputs plus structure-aware modulation in medical enhancement (Dahmani et al., 7 Apr 2026, Xu et al., 2 Jun 2026, Mao et al., 16 May 2026, Sumuk, 1 Jan 2026, Jiang et al., 20 Apr 2026). A third is explicit preservation of geometric structure, whether through discrete occupancy in DVD, voxel exclusivity in VoxScene, or observed-region preservation in mask-conditioned inpainting (Xiang et al., 8 May 2026, Mao et al., 16 May 2026, Sumuk, 1 Jan 2026).

The limitations are equally domain-specific. SEM-ROVER’s scalability depends on progressive spatial outpainting because the diffusion model is local rather than whole-scene (Dahmani et al., 7 Apr 2026). PatchScene notes that too few denoising steps leave visible patch boundaries, whereas too many can introduce excessive stochasticity (Xu et al., 2 Jun 2026). DVD states that generation from scratch still needs hundreds of NFEs and is not yet ideal for real-time applications, while also restricting itself to binary occupancy rather than richer voxel attributes (Xiang et al., 8 May 2026). The mask-conditioned inpainting framework operates at fixed $G_o \in \mathbb{N}^{D \times W \times H}$2 resolution (Sumuk, 1 Jan 2026). The sparse medical-enhancement method assumes that degraded inputs already contain strong anatomical priors, which is central to its sparse-schedule argument (Jiang et al., 20 Apr 2026).

Taken together, these systems show that VDM is best treated as a research category rather than a canonical module definition. The unifying idea is not a single architecture, but the decision to perform diffusion, denoising, propagation, or refinement directly in voxelized 3D structure, with the exact mathematical mechanism determined by the application regime.