---
title: 3D Wavelet Diffusion Models
url: https://www.emergentmind.com/topics/3d-wavelet-diffusion-model-wdm3d
type: topic
---

# 3D Wavelet Diffusion Models

Searching arXiv for recent and foundational papers on 3D wavelet diffusion models to ground the article in current literature.
3D Wavelet Diffusion Model (WDM3D) denotes a family of diffusion-based generative and restoration models that integrate 3D wavelet representations into the modeling of volumetric data. Across the literature, the term encompasses at least two closely related constructions: direct diffusion on 3D wavelet coefficients, and controlled volumetric diffusion whose reverse dynamics are guided by 3D wavelet priors. In both forms, the central idea is to exploit the frequency-separated, invertible, and spatially compact structure of a 3D Discrete Wavelet Transform (DWT) in order to improve memory efficiency, multi-scale fidelity, and volumetric coherence for tasks such as 3D shape generation, medical image synthesis, denoising, inpainting, and outpainting [2402.19043] [2209.08725] [2601.07093].

## 1. Concept and nomenclature

The expression WDM3D is not attached to a single canonical architecture. In some works it is the formal method name, as in “WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis,” where diffusion is performed directly in wavelet space on full 3D medical volumes [2402.19043]. In other works it is an accurate descriptive label rather than the authors’ official acronym. “Neural Wavelet-domain Diffusion for 3D Shape Generation” presents a diffusion generator on wavelet-domain implicit 3D representations and is explicitly characterized as aligning with the notion of a 3D Wavelet Diffusion Model [2302.00190]. “UDiFF” likewise does not use the term WDM3D, but it is described as precisely instantiating a 3D wavelet diffusion model for unsigned distance fields [2404.06851].

The term also has important boundaries. “3D Wavelet Latent Diffusion Model” performs diffusion in a learned latent space and uses wavelets inside the encoder–decoder through a Wavelet Residual Module; it is therefore closely related in spirit but not equivalent to wavelet-domain diffusion conducted directly over 3D coefficients [2507.11557]. Similarly, “WaveletGaussian” applies 2D wavelet-domain diffusion to per-view renders inside a 3D Gaussian Splatting pipeline rather than diffusing over a 3D wavelet field [2509.19073]. “3D-WMoCo” uses slice-wise 2D Haar DWT and fused orthogonal 2D score priors for 3D MRI motion correction; it is described as a practical 3D wavelet diffusion framework, but its wavelet updates are pseudo-3D rather than a full volumetric 3D DWT formulation [2511.02256].

A persistent source of confusion is therefore the distinction between **wavelet-domain diffusion**, **wavelet-conditioned diffusion**, and **wavelet-enhanced latent diffusion**. The literature treats all three as wavelet-informed volumetric generative modeling, but only the first category strictly performs the Markov noising and denoising process on 3D wavelet coefficients.

## 2. Mathematical formulation and wavelet representations

The defining transform in WDM3D is a separable 3D DWT. For a 3D volume $x \in \mathbb{R}^{H \times W \times D}$, single-level separable analysis with low-pass and high-pass filters along each axis yields eight subbands,
$$
X_{abc} = \mathrm{down}_2\big((x \ast f_a^{(x)} \ast f_b^{(y)} \ast f_c^{(z)})\big), \quad a,b,c \in \{L,H\},
$$
corresponding to $LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH$. In wavelet-domain medical synthesis, these subbands are stacked as channels, giving a tensor in $\mathbb{R}^{8 \times (H/2) \times (W/2) \times (D/2)}$, and exact reconstruction is obtained by the inverse DWT [2402.19043]. Single-level decompositions are common because they halve each spatial dimension while preserving exact invertibility and keeping the denoiser aligned with standard UNet resolutions [2411.17203] [2601.07093].

Direct WDM3D usually defines diffusion in wavelet space. If $y_0 = W(x_0)$ denotes the wavelet coefficients of a real volume, a standard forward process is
$$
q(y_t \mid y_{t-1}) = \mathcal{N}(y_t; \sqrt{1-\beta_t}\,y_{t-1}, \beta_t I),
$$
with closed form
$$
y_t = \sqrt{\bar{\alpha}_t}\,y_0 + \sqrt{1-\bar{\alpha}_t}\,\varepsilon,\quad \varepsilon \sim \mathcal{N}(0,I).
$$
Within this general template, parameterization differs by task. The medical-image WDM of Friedrich et al. uses $x_0$-prediction in wavelet space and reports that $\varepsilon$-prediction produced checkerboard artifacts, whereas the earlier 3D shape models and PET denoising framework adopt the standard noise-prediction objective [2402.19043] [2209.08725] [2601.07093]. cWDM also uses $x_0$-prediction on 3D wavelet coefficients for conditional cross-modality MR synthesis [2411.17203]. POWDR formulates pathology-preserving outpainting with an $x_0$-prediction loss computed over the eight wavelet subbands [2601.09044].

A second major formulation factorizes frequency content into coarse and detail components. In the 3D shape lineage, a compact pair of coefficient volumes is used: a coarse coefficient volume $C^J$ is synthesized by diffusion, and a deterministic predictor produces the corresponding detail volume $D^J$, after which inverse wavelet synthesis reconstructs the truncated signed distance field [2209.08725]. UDiFF generalizes this principle to unsigned distance fields, learning a biorthogonal 3D wavelet transform and performing diffusion only on coarse coefficient volumes while regressing fine coefficient volumes with a separate 3D U-Net [2404.06851].

A third formulation injects wavelets as explicit priors rather than diffusing over all subbands. WCC-Net computes a single-level separable 3D DWT of the low-dose PET input, emphasizes low-frequency bands such as $LLL$, embeds them with a lightweight 3D transposed convolution, and injects them into a frozen pretrained 3D DDPM through zero-initialized $1 \times 1 \times 1$ convolutions at encoder skip connections [2601.07093]. This suggests a broader operational definition of WDM3D: volumetric diffusion whose reverse process is explicitly structured by 3D wavelet priors.

## 3. Architectural patterns

Three architectural patterns recur across WDM3D systems.

The first is **joint subband denoising**. Here the eight subbands are stacked as channels and processed by a 3D UNet at half spatial resolution. This is the design used in high-resolution medical image synthesis and in conditional cross-modality synthesis, where the denoiser directly predicts either clean coefficients or the diffusion target for all subbands jointly [2402.19043] [2411.17203]. POWDR follows the same pattern for conditioned outpainting: the network input concatenates eight noised target channels with eight conditioning channels derived from the masked real pathology, and the inverse DWT reconstructs the final MRI volume [2601.09044].

The second is **coarse-to-fine generation**. The 2022 and 2023 implicit-shape models use a diffusion-based generator for coarse coefficients and a separate detail predictor for fine structure, both implemented with modified 3D UNets. This decomposition reflects the observation that coarse wavelet coefficients capture global geometry and topology, whereas the detail volume restores fine structures and clean surfaces [2209.08725] [2302.00190]. UDiFF preserves the same division but replaces fixed filters with learnable decomposition and inversion filters, yielding an “optimal wavelet transformation” for UDFs and reconstructing the field through the learned inverse transform [2404.06851].

The third is **wavelet-conditioned control**. WCC-Net freezes a pretrained 3D diffusion backbone and trains only a lightweight control branch that processes selected wavelet priors, by default the low-frequency $LLL$ subband. The control features are injected additively into frozen skip features through zero-initialized convolutions so that the network begins as the original pretrained DDPM and gradually learns conditioning without destabilizing the generative prior [2601.07093]. This design is closely related to ControlNet, but the control signal is a 3D wavelet prior rather than an edge map or segmentation mask.

Related systems expand the design space further. 3D-WLDM uses wavelet-enhanced encoding and decoding, structure–modality disentanglement, and Dual Skip Connection Attention, but the diffusion itself is latent-space diffusion rather than wavelet-space diffusion [2507.11557]. 3D-WMoCo combines a mean-reverting SDE, slice-wise wavelet diffusion, fused orthogonal 2D score priors, and WTConv blocks, showing that wavelet acceleration can also be embedded into pseudo-3D score-based restoration [2511.02256].

## 4. Application domains

WDM3D has developed across several application domains, with notably different representations and evaluation protocols.

| Domain | Representative formulation | Example paper |
|---|---|---|
| 3D shape generation | Diffusion on coarse wavelet coefficients plus detail prediction for TSDF/UDF reconstruction | [2209.08725], [2404.06851] |
| High-resolution medical synthesis | Direct diffusion on 3D wavelet coefficients of volumetric MRI or CT | [2402.19043], [2411.17203] |
| Medical restoration | Wavelet-conditioned or wavelet-domain denoising, inpainting, and outpainting | [2601.07093], [2507.13146], [2601.09044] |
| Hybrid spatio-temporal generation | 3D wavelet volume encoding fused with other latent factors before diffusion | [2402.13729] |

In 3D shape modeling, wavelet-domain diffusion first appeared in implicit representations built from truncated signed distance functions sampled on a $256^3$ grid, with diffusion applied to coarse wavelet coefficients and Marching Cubes used after inverse reconstruction [2209.08725]. UDiFF extended this line to unsigned distance fields for open-surface generation, emphasizing that UDFs avoid the inside–outside ambiguity that constrains signed-distance and occupancy models on non-watertight geometries [2404.06851].

In volumetric medical synthesis, WDM established wavelet-space diffusion as a practical alternative to patch-wise, slice-wise, cascaded, or latent-autoencoder approaches. It synthesizes full 3D medical images at $128 \times 128 \times 128$ and $256 \times 256 \times 256$ resolution, while cWDM adapts the same principle to paired cross-modality MR synthesis by concatenating the DWTs of three conditioning modalities with the noisy target coefficients [2402.19043] [2411.17203]. fastWDM3D specializes the formulation to healthy tissue inpainting, using the wavelet coefficients of the voided image, the mask, and the noisy target as a 24-channel input [2507.13146].

In restoration and enhancement, WDM3D has become a mechanism for imposing structure while preserving volumetric continuity. WCC-Net uses low-frequency 3D wavelet priors to guide low-dose PET denoising; POWDR uses wavelet-domain conditioning to preserve real pathology while synthesizing surrounding anatomy in 3D MRI; both are explicitly motivated by the need to retain clinically relevant structures while controlling the generative process [2601.07093] [2601.09044]. A plausible implication is that wavelet-domain conditioning is especially attractive in medical settings where geometric fidelity is more important than unconstrained sample diversity.

The broader influence of WDM3D is also visible in hybrid video modeling. HVDM introduces a hybrid autoencoder in which a 3D wavelet volume latent encodes low-pass and high-pass spatio-temporal subbands, these features are fused with 2D triplane context by cross-attention, and diffusion is then performed on the fused hybrid latent [2402.13729]. Although this is not a pure WDM3D in the strict sense, it demonstrates the portability of 3D wavelet representations beyond static volumes.

## 5. Empirical characteristics and reported performance

A central empirical claim of WDM3D is that wavelet-space operation makes high-resolution 3D diffusion tractable without abandoning full-volume modeling. In the original medical-image WDM, unconditional generation on BraTS at $128^3$ achieved $\mathrm{FID}\times 10^3 = 0.154$, $\mathrm{MS\text{-}SSIM} = 0.888$, and inference memory of $2.55$ GB; at $256^3$, WDM reported $\mathrm{FID}\times 10^3 = 0.379$, $\mathrm{MS\text{-}SSIM} = 0.890$, and $7.27$ GB, and was described as the only diffusion-based method the authors could train on a single $40$ GB GPU at that resolution [2402.19043].

Conditional full-volume synthesis exhibits the same scaling advantage. cWDM processes entire $155 \times 240 \times 240$ MR volumes on a single A100 40GB GPU and reports, on validation data, $\mathrm{MSE}=1.65\times 10^{-3}$, $\mathrm{PSNR}=29.74$, and $\mathrm{SSIM}=0.956$ for missing T1, with a “Random” missing-modality setting of $\mathrm{MSE}=1.82\times 10^{-3}$, $\mathrm{PSNR}=28.72$, and $\mathrm{SSIM}=0.946$ [2411.17203]. fastWDM3D pushes efficiency further: on the BraTS inpainting test set it reports $\mathrm{SSIM}=0.8571$, $\mathrm{MSE}=0.0079$, and $\mathrm{PSNR}=22.26$ using only two time steps, with a runtime of $1.81$ s per image and a speedup stated as up to $800 \times$ relative to other DDPM-based healthy tissue inpainting systems [2507.13146].

In controlled denoising, WCC-Net reports consistent gains over CNN-, GAN-, and diffusion-based baselines. On the internal $1/20$-dose PET test set it achieves $43.594 \pm 1.404$ dB PSNR, $0.984 \pm 0.005$ SSIM, GMSD $0.011 \pm 0.003$, and NMAE $0.111 \pm 0.014$, improving over a strong 3D DDPM baseline by $+1.21$ dB PSNR and $+0.008$ SSIM while reducing GMSD by $-0.003$ and NMAE by $-0.006$ [2601.07093]. In pathology-preserving outpainting, POWDR reports synthetic-versus-real FID of $0.0042$, inside-lesion MS-SSIM of $0.9998 \pm 0.0003$, inside-lesion LPIPS of $0.0030 \pm 0.0017$, and a tumor-segmentation Dice increase from $0.6992 \pm 0.3009$ to $0.7137 \pm 0.2868$ when adding $50$ synthetic cases [2601.09044].

Shape-generation results show that the wavelet-domain design is not restricted to medical imaging. The 2022 wavelet-domain diffusion model reports, for ShapeNet chairs, COV-CD $58.19$, COV-EMD $55.46$, MMD-CD $11.70$, MMD-EMD $14.31$, 1-NNA-CD $61.47$, and 1-NNA-EMD $61.62$, while UDiFF reports on DeepFashion3D COV $69.62$ (CD), $67.72$ (EMD), MMD $11.60$ (CD$\times10^3$), $14.01$ (EMD$\times10^2$), and 1-NNA $81.83$ (CD), $82.14$ (EMD) [2209.08725] [2404.06851].

These results do not imply a uniform performance advantage for every waveletized diffusion design. They do, however, consistently support two narrower claims that recur across the literature: wavelet-space diffusion reduces the dominant 3D spatial footprint, and frequency-separated conditioning often improves structural fidelity.

## 6. Limitations, distinctions, and open directions

The literature identifies wavelet choice and decomposition depth as recurrent sensitivities. The medical-image WDM uses single-level orthonormal Haar wavelets and notes that different families such as Daubechies or biorthogonal filters may alter the frequency partition and affect synthesis quality [2402.19043]. UDiFF responds by learning biorthogonal analysis and synthesis filters, arguing that fixed manually chosen wavelets incur larger information loss near the zero-level set of UDFs [2404.06851]. WCC-Net likewise identifies dependence on the chosen wavelet family and level as a limitation, since a fixed single-level Haar decomposition may not capture multi-scale nuances of certain anatomies [2601.07093].

Another limitation concerns the treatment of fine structure. The TSDF and UDF models explicitly note that extremely fine structures, very thin shells, or small high-frequency artifacts remain challenging when diffusion is restricted to coarse coefficients and details are predicted separately [2209.08725] [2404.06851]. In pathology-preserving outpainting, conditioning can collapse diversity outside the preserved region unless random connected masks are introduced during training; without this strategy, repeated samples for the same lesion become nearly identical outside the mask [2601.09044].

A further conceptual limitation is terminological. Wavelet-informed 3D diffusion systems are often grouped together under the label WDM3D, but the underlying diffusion space may be fundamentally different. 3D-WLDM is latent diffusion with wavelet-enhanced encoding and decoding rather than diffusion over wavelet coefficients [2507.11557]. 3D-WMoCo accelerates 3D restoration by slice-wise 2D Haar DWT and pseudo-3D score fusion, not by a volumetric 3D DWT denoiser [2511.02256]. WaveletGaussian is a per-view 2D wavelet repair model embedded in a 3D reconstruction loop, not a fully volumetric WDM3D [2509.19073]. For that reason, the term is best understood as a family resemblance category rather than a single architectural standard.

Open directions in the literature are correspondingly diverse. They include learned or adaptive wavelet bases, multi-level or joint multi-scale denoising, direct diffusion on selected detail bands, acceleration beyond iterative DDPM sampling, and stronger conditioning mechanisms for structure preservation, modality disentanglement, or controllable editing [2404.06851] [2601.07093] [2507.13146]. This suggests that the most durable contribution of WDM3D is not a fixed recipe, but a design principle: volumetric diffusion can be made more tractable and more structurally precise by moving either the representation, the conditioning signal, or both into a 3D wavelet basis.

Source: https://www.emergentmind.com/topics/3d-wavelet-diffusion-model-wdm3d