3D Wavelet Latent Diffusion Model

• 3D-WLDM integrates wavelet representations with latent diffusion, enabling efficient multiresolution processing of high-dimensional 3D data.
• It employs fixed analytic wavelet transforms or learned autoencoder latents to compress data while preserving fine structural details.
• This approach reduces memory requirements and enhances scalability, proving effective in tasks like MR-to-CT translation and pathology-preserving outpainting.

3D Wavelet Latent Diffusion Model (3D-WLDM) denotes a family of 3D generative models that move diffusion away from raw voxel or image space and into a wavelet-structured representation. In strict usage, the term refers to models that perform diffusion in a learned latent space while explicitly incorporating wavelet-aware encoding or residual wavelet structure, as in whole-body MR-to-CT translation and WaLa; in broader usage, it also encompasses 3D wavelet-domain diffusion systems that denoise fixed discrete wavelet coefficients directly, even when no learned autoencoder latent is present (Zheng et al., 14 Jul 2025, Sanghi et al., 2024, Friedrich et al., 2024).

1. Terminology and conceptual scope

A central distinction in the literature is between wavelet-domain diffusion and wavelet latent diffusion. WDM, for example, is explicitly described as a diffusion model operating on a fixed wavelet-domain representation of a 3D medical image, not a latent diffusion model in the strict learned-latent sense: no encoder is learned, no decoder is learned, the representation is invertible via exact IDWT, and the compression comes from deterministic Haar DWT rather than a pretrained autoencoder (Friedrich et al., 2024). POWDR inherits the same conceptual boundary: it is highly relevant to a 3D-WLDM agenda, but it remains a 3D conditional wavelet diffusion model rather than a canonical latent diffusion model because it applies diffusion directly to a fixed wavelet-domain representation of the full 3D volume (Tan et al., 14 Jan 2026).

By contrast, the whole-body MR-to-CT model explicitly named 3D Wavelet Latent Diffusion Model performs modality translation in a learned latent space and augments the encoder-decoder with a Wavelet Residual Module, while WaLa first converts a high-resolution 3D shape into a wavelet representation, then compresses that representation into a very small latent grid using a VQ-VAE, and finally trains diffusion in that compact latent space (Zheng et al., 14 Jul 2025, Sanghi et al., 2024). These systems satisfy the stronger sense of “wavelet latent diffusion,” because diffusion operates in a learned latent rather than directly on analytic coefficients.

A common misconception is therefore that any diffusion over wavelet coefficients is automatically a latent diffusion model. The literature repeatedly rejects that equivalence. The 3D shape papers on neural wavelet-domain diffusion, for instance, diffuse coarse wavelet coefficients and use auxiliary latent mechanisms for inversion or manipulation, but diffusion itself does not run in an encoder latent; similarly, UDiFF is described as a close variant of 3D-WLDM only if “latent” is interpreted broadly as a compact hidden representation rather than a VAE-style latent space (Hu et al., 2023, Zhou et al., 2024). In the opposite direction, canonical 3D latent diffusion can be entirely non-wavelet: 3D-LDM diffuses a 256-dimensional auto-decoder latent vector for neural implicit SDFs, with no wavelet decomposition at all (Nam et al., 2022).

2. Representation spaces and multiresolution structure

The simplest 3D wavelet representation used in this literature is the single-level 3D Haar discrete wavelet transform. For a 3D volume $y_0 \in \mathbb{R}^{H \times W \times D}$, POWDR writes

$$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$

with

$$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$

WDM uses the same basic construction: one low-frequency approximation sub-band and seven high-frequency detail sub-bands, concatenated along the channel dimension, so diffusion operates on an 8-channel tensor at half spatial resolution in every axis (Tan et al., 14 Jan 2026, Friedrich et al., 2024). This construction is attractive because halving each spatial dimension reduces the 3D spatial volume by a factor of $2^3=8$, while retaining explicit detail channels.

Wavelet latent models introduce an additional learned compression stage on top of multiresolution structure. In the whole-body MR-to-CT 3D-WLDM, the encoder maps MR and CT volumes to learned latent tensors $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$, and each latent is partitioned as

$Z = [S, M],$

where the first half of the channels encode structural information and the second half encode modality-specific information (Zheng et al., 14 Jul 2025). WaLa adopts a different route: a $256^3$ signed distance field is first converted to a diffusible wavelet tree $W_n \in \mathbb{R}^{46^3 \times 64}$, then encoded by a convolutional VQ-VAE to a latent grid

$Z \in \mathbb{R}^{12^3 \times 4},$

yielding 6912 latent variables and a reported $2427\times$ compression ratio relative to the original $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$0 field (Sanghi et al., 2024).

A third regime appears in wavelet-coefficient 3D shape generation. The 2022 and 2023 neural wavelet-domain diffusion papers represent a $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$1 TSDF by recursive multiscale coefficients $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$2, choose $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$3, and keep only $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$4 as a compact coarse/detail pair (Hui et al., 2022, Hu et al., 2023). UDiFF similarly decomposes a $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$5 UDF into coarse and fine coefficients,

$$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$6

but learns the decomposition and inversion filters themselves, describing them as a learnable biorthogonal wavelet transform adapted to UDF statistics (Zhou et al., 2024).

This variety of representations shows that “3D-WLDM” does not refer to a single tensor layout. In existing work it may mean analytic DWT coefficients, a learned latent built from wavelet-processed features, or a learned latent over a wavelet tree. The unifying property is not a fixed architecture but the use of wavelet-structured multiresolution geometry to make full 3D diffusion tractable.

3. Diffusion objectives, denoisers, and conditioning

The diffusion objective is not uniform across the literature. WDM performs diffusion entirely in wavelet space and predicts denoised coefficients $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$7 rather than noise: $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$8 The paper states that predicting the noise to remove produced checkerboard artifacts, so direct $$x_0 = DWT(y_0) = \{LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH\},$$9-prediction was retained (Friedrich et al., 2024). POWDR uses the same wavelet-space reconstruction objective,

$$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$0

and is therefore most naturally interpreted as an $$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$1-prediction conditional DDPM in wavelet space (Tan et al., 14 Jan 2026).

By contrast, the named 3D-WLDM for MR-to-CT synthesis adopts latent noise prediction: $$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$2 with latent forward noising

$$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$3

Here the noisy CT latent is denoised under MR latent conditioning, and the structural component of the MR latent is meant to anchor anatomy during reverse diffusion (Zheng et al., 14 Jul 2025). WaLa also departs from standard $$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$4-prediction: its denoiser is trained to recover the clean latent directly,

$$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$5

and it uses a cosine noise schedule (Sanghi et al., 2024).

Other systems remain closer to standard DDPM practice. UDiFF minimizes

$$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$6

for coarse wavelet coefficients, with CLIP-based cross-attention for text or image conditioning (Zhou et al., 2024). HVDM similarly applies standard DDPM $$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$7-prediction in a fused latent volume

$$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$8

using a 3D U-Net denoiser after a hybrid autoencoder combines a 2D projected latent branch with a 3D wavelet-volume branch (Kim et al., 2024).

Conditioning mechanisms are likewise heterogeneous. POWDR concatenates noisy target wavelet coefficients with DWT-transformed conditioning coefficients from a masked lesion-bearing image, producing a 16-channel input to a 3D ResUNet with attention (Tan et al., 14 Jan 2026). The whole-body MR-to-CT 3D-WLDM conditions on MR latent features and inserts Dual Skip Connection Attention into the diffusion U-Net; its Structure Emphasis Module attends from shallow diffusion features to $$x_0 \in \mathbb{R}^{8 \times \frac{H}{2}\times \frac{W}{2}\times \frac{D}{2}}.$$9, while its Modality Filtering Module subtracts modality-specific interference associated with $2^3=8$0 (Zheng et al., 14 Jul 2025). SC-Diff, though non-wavelet, provides a relevant comparison: it combines CLIP cross-attention for optional RGB conditioning with a ControlNet-style 3D conditioning branch for partial TSDF observations (Galvis et al., 2024).

4. Medical imaging formulations

Medical imaging is the domain in which 3D wavelet diffusion has been developed most explicitly as a response to cubic memory growth. WDM targets unconditional 3D medical volume generation for BraTS T1-weighted MRI and LIDC-IDRI lung CT at $2^3=8$1 and $2^3=8$2 resolution, and frames wavelet decomposition as a simple alternative to patch-wise, slice-wise, cascaded, or learned-latent 3D generation. It reports that WDM can be trained on a single 40 GB NVIDIA A100, requires 2.55 GB inference memory at $2^3=8$3 and 7.27 GB at $2^3=8$4, and is the only diffusion-based method in its comparison that could be trained at $2^3=8$5 on a single 40 GB GPU (Friedrich et al., 2024).

POWDR moves from unconditional synthesis to pathology-preserving outpainting. Rather than generating lesions from scratch, it retains a real pathology-bearing region and synthesizes only the surrounding anatomy. The condition is formed as $2^3=8$6, where $2^3=8$7 is a pathology mask or a random connected mask, then transformed to wavelet space and concatenated with the noisy sample at every denoising step (Tan et al., 14 Jan 2026). Its most distinctive contribution is the random connected mask strategy, introduced because training only on pathology masks produced near-deterministic completions: repeated sampling with tumor-mask conditioning gave pairwise cosine similarity $2^3=8$8 and KL divergence $2^3=8$9, while adding random connected masks reduced cosine similarity to $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$0 and increased KL divergence to $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$1 (Tan et al., 14 Jan 2026). The method also reports downstream gains: using nnU-Net for whole-tumor segmentation, the baseline Dice $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$2 increases to $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$3 when 50 synthetic cases are added (Tan et al., 14 Jan 2026).

The explicitly named 3D Wavelet Latent Diffusion Model for Whole-Body MR-to-CT Modality Translation adopts a stricter 3D-WLDM formulation. Its encoder-decoder backbone is VAE-style, wavelet processing is inserted through the Wavelet Residual Module, and the latent is disentangled into structural and modality-specific components. During diffusion, the noisy CT latent and MR latent are concatenated and passed through a U-Net with Dual Skip Connection Attention, whose SEM branch emphasizes structure from $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$4 and whose MFM branch suppresses MRI-specific interference from $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$5 (Zheng et al., 14 Jul 2025). Quantitatively, the model reports PSNR $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$6, SSIM $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$7, MAE $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$8, and NCC $$Z_{MR}, Z_{CT} \in \mathbb{R}^{c \times d \times h \times w}$$9 against baselines including DDPM, DDIM, StableDM-2D, ControlNet, StableDM-3D, and CycleGAN, and its full ablation stack improves PSNR by 2.31 dB, SSIM by 0.23, and MAE by 28.08 over the vanilla latent diffusion baseline (Zheng et al., 14 Jul 2025). The same study reports improved downstream TotalSegmentator Dice, with full-model values of 81.4 ± 9.6 for spleen, 75.9 ± 21.8 for kidney, 89.7 ± 5.4 for liver, 71.1 ± 10.5 for stomach, and 80.1 ± 17.0 for vertebrae (Zheng et al., 14 Jul 2025).

Across these medical systems, the practical motivation is consistent: wavelet structure is used to preserve fine anatomy, reduce memory pressure, and avoid the artifacts or downstream restrictions attributed to patch-wise or slice-wise generation. The divergence lies in whether the wavelet representation is analytic and invertible, as in WDM and POWDR, or embedded inside a learned latent autoencoder, as in the whole-body 3D-WLDM.

5. Shape, geometry, and video analogues

In 3D shape generation, the earliest wavelet-diffusion line models geometry as a truncated signed distance field and performs diffusion on compact multiscale wavelet coefficients rather than raw voxels. “Neural Wavelet-domain Diffusion for 3D Shape Generation” normalizes each shape, samples a $Z = [S, M],$0 TSDF, truncates to $Z = [S, M],$1, and uses a compact scale-$Z = [S, M],$2 pair $Z = [S, M],$3; on ShapeNet chairs, reconstructing from only $Z = [S, M],$4 and $Z = [S, M],$5 changes TSDF magnitudes by only 2.8% while removing 97% of the coefficients (Hui et al., 2022). Diffusion is applied only to the coarse coefficients, and a deterministic detail predictor restores fine structure; generation takes about 6 seconds per shape on one RTX 3090 (Hui et al., 2022). The 2023 extension adds an encoder $Z = [S, M],$6 for inversion and manipulation, but diffusion still operates on coarse wavelet coefficients rather than on the encoder latent, so it remains wavelet-domain diffusion with an auxiliary latent conditioner rather than a standard latent diffusion model (Hu et al., 2023).

UDiFF generalizes wavelet-domain diffusion from SDFs to unsigned distance fields so that open surfaces can be modeled. It learns both the decomposition filter $Z = [S, M],$7 and inversion filter $Z = [S, M],$8, diffuses only over coarse coefficients, predicts fine coefficients with a second U-Net, and reconstructs UDFs before meshing with DCUDF rather than marching cubes (Zhou et al., 2024). Its strongest evidence for learned wavelet optimization is the reconstruction ablation on DeepFashion3D: mesh recovery from Haar gives 264.8, Biorthogonal3-3 gives 46.04, Biorthogonal6-8 gives 42.92, learnable decomposition only gives 36.12, learnable inversion only gives 32.15, and learning both gives 28.51 in L2 Chamfer Distance scaled by $Z = [S, M],$9 (Zhou et al., 2024).

HVDM shows how wavelet ideas migrate into latent video diffusion. It uses a deterministic hybrid autoencoder that decomposes video into a 2D projected latent branch and a 3D wavelet-volume branch, with the latter applying a true spatio-temporal 3D DWT over time, height, and width and separately encoding $256^3$0 and the concatenated high-frequency sub-bands (Kim et al., 2024). Diffusion is then trained on the fused latent rather than on raw wavelet coefficients. In UCF-101 reconstruction, the wavelet frequency-matching loss improves R-FVD from 8.08 to 5.35, LPIPS from 0.041 to 0.038, PSNR from 33.67 to 34.00, and SSIM from 0.892 to 0.915 (Kim et al., 2024). This is not a pure wavelet latent diffusion model, but it demonstrates that explicit low/high-frequency handling remains useful even when diffusion acts on a hybrid latent.

WaLa is the clearest large-scale 3D-WLDM instantiation in geometry generation. It starts from a $256^3$1 SDF, converts it to a wavelet tree $256^3$2, compresses that representation to $256^3$3, and trains conditional or unconditional diffusion with models of roughly one billion parameters (Sanghi et al., 2024). The method reports a 2427x compression ratio, conditioned inference in 2 to 4 seconds, and strong performance across point-cloud, voxel, single-view, and multi-view settings (Sanghi et al., 2024). On GSO, for example, its representation comparison reports IoU 0.9780 with only 6912 variables, compared with 0.9956 for the full wavelet tree at 1,129,528 variables and 1.0 for the original $256^3$4 SDF at 16,777,216 variables (Sanghi et al., 2024). WaLa is therefore not merely wavelet-domain diffusion; it is diffusion over a learned latent that is itself built on a wavelet representation.

6. Relation to non-wavelet 3D latent diffusion and recurrent limitations

Wavelet-based 3D latent diffusion is best understood against neighboring 3D latent diffusion systems that do not use wavelets. 3D-LDM trains diffusion over a 256-dimensional DeepSDF-style latent vector and conditions on images or text via CLIP embeddings, but the latent has no explicit spatial locality or frequency separation (Nam et al., 2022). SC-Diff compresses $256^3$5 TSDFs to a VQ-VAE latent

$256^3$6

then performs conditional latent diffusion for shape completion, showing that latent-space diffusion uses 30.72% less GPU memory than a non-latent alternative while enabling $256^3$7 completion (Galvis et al., 2024). LaGeM builds a hierarchical latent set representation for native 3D geometry and performs cascaded diffusion over latent levels $256^3$8, but explicitly does not use wavelets; its hierarchy is multiresolution only in a learned latent-set sense (Zhang et al., 2024). “AutoDecoding Latent 3D Diffusion Models” similarly diffuses a dense volumetric latent $256^3$9 learned from 2D-supervised autodecoding, again with no wavelet structure (Ntavelis et al., 2023).

Relative to those systems, wavelet-based models repeatedly claim four advantages. First, they provide explicit multiscale separation between coarse and high-frequency detail rather than relying on a monolithic latent to preserve sharp geometry. Second, in fixed-transform variants such as WDM and POWDR, they provide exact invertibility without requiring a difficult 3D autoencoder pretraining stage (Friedrich et al., 2024, Tan et al., 14 Jan 2026). Third, in learned latent variants such as WaLa and the whole-body 3D-WLDM, they offer a route to combine latent compression with wavelet-aware preservation of fine structure (Sanghi et al., 2024, Zheng et al., 14 Jul 2025). Fourth, they often make full 3D generation feasible without patch-wise or slice-wise decomposition, which is a recurring concern in medical imaging (Friedrich et al., 2024).

The limitations are equally recurrent. Terminology remains unstable: several important “3D-WLDM-like” systems are not latent diffusion models in the strict modern sense, because diffusion acts on analytic wavelet coefficients rather than on a learned latent (Friedrich et al., 2024, Tan et al., 14 Jan 2026). Learned wavelet formulations can also be underspecified: UDiFF does not fully document the exact neural parameterization of $W_n \in \mathbb{R}^{46^3 \times 64}$0 and $W_n \in \mathbb{R}^{46^3 \times 64}$1, including filter structure and constraints (Zhou et al., 2024). Some conditional settings collapse without additional training strategies, as POWDR explicitly shows (Tan et al., 14 Jan 2026). Diffusion speed remains a bottleneck in unconditional or long-chain settings, and geometry pipelines still often depend on marching cubes or DCUDF for surface extraction (Sanghi et al., 2024, Zhou et al., 2024). The whole-body MR-to-CT 3D-WLDM also depends on accurate MR-CT alignment during training, a limitation the authors identify directly (Zheng et al., 14 Jul 2025).

A plausible implication is that future 3D-WLDMs will continue to hybridize these two traditions: the exact multiresolution structure of wavelet-domain modeling and the scalability of learned latent diffusion. The current literature already spans fixed Haar DWT, learned biorthogonal transforms, wavelet-aware residual autoencoders, hybrid projected-plus-wavelet video latents, and VQ-VAE compression of wavelet trees. The common thread is not a single canonical design, but the use of wavelet structure as a principled mechanism for making high-resolution 3D diffusion both computationally feasible and detail-aware.