---
title: Denoising Diffusion Codebook Models
url: https://www.emergentmind.com/topics/denoising-diffusion-codebook-models-ddcms
type: topic
---

# Denoising Diffusion Codebook Models

Denoising Diffusion Codebook Models (DDCMs) are a class of generative and compression frameworks that discretize the stochasticity of denoising diffusion models using finite, reproducible codebooks of Gaussian noise vectors. DDCMs simultaneously enable lossless mapping between bitstreams and high-quality images, with applications in perceptual image compression, compressed sample generation, and conditional inverse problems. Recent advances include highly efficient variants (Turbo-DDCM), generalizations across diffusion families (gDDCM), and integration with conditional and region-prioritized sampling.

## 1. The DDCM Framework: Discretizing Diffusion Sampling

Denoising Diffusion Probabilistic Models (DDPMs) establish a Markov chain \( x_0 \to x_1 \to \dots \to x_T \) where each step injects additive Gaussian noise: 
\[
q(x_t\mid x_{t-1}) = \mathcal N \left( x_t ; \sqrt{\alpha_t}x_{t-1}, (1-\alpha_t)\mathbf I \right)
\]
The reverse process is learned via a neural denoiser or score model, with the canonical form:
\[
x_{t-1} = \mu_\theta(x_t, t) + \sigma_t z_t, \quad z_t\sim\mathcal N(0,I)
\]
DDCMs replace the standard Gaussian samples \( z_t \) with elements from a finite codebook \( C_t = [c_t^{(1)}, \dots, c_t^{(K)}] \), \( c_t^{(k)}\sim\mathcal N(0,I) \), chosen deterministically according to the current reverse path and potentially a target reconstruction \( x_0 \) [2502.01189, 2511.06424]. The update thus becomes:
\[
x_{t-1} = \mu_\theta(x_t, t) + \sigma_t c_t^{(k_t)}, \quad k_t \in \{1,\dots, K\}
\]
The full trajectory is losslessly specified by the index sequence \( \{k_2,\dots,k_T\} \), yielding a constant-bits-per-pixel (BPP) discrete encoding:
\[
\mathrm{BPP}_{\mathrm{DDCM}} = \frac{(T-1) \lceil \log_2 K \rceil}{\#\mathrm{pixels}}
\]
The encoder and decoder share the same codebooks and neural denoiser, ensuring reproducibility from the compressed stream.

## 2. Noise Selection, Conditional Objectives, and Algorithmic Details

Unlike random codeword selection, DDCM achieves compression or conditional generation by greedily aligning codebook entries to desired reconstruction objectives. For image compression, the encoder chooses \( k_t = \arg\max_{k} \langle c_t^{(k)}, r_t \rangle \) at each step, where \( r_t = x_0 - \hat{x}_{0|t}(x_t) \), and \( \hat{x}_{0|t} \) is the current MMSE denoiser prediction [2502.01189]. This pushes the reverse process to reconstruct \( x_0 \) with minimal residual error, subject to the codebook discretization. For conditional tasks (e.g., restoration with condition \( y \)), the objective generalizes:
\[
k_t = \arg\min_{k} \mathcal{L}(y, x_t, c_t^{(k)})
\]
where \( \mathcal{L} \) may encode posterior likelihood, classifier-free guidance, or arbitrary perceptual metrics.

The DDCM sampling and compression scheme does not require modification or retraining of the underlying diffusion backbone—the selection operates entirely at inference time using upstream network predictions [2502.01189, 2511.06424]. A plausible implication is that DDCM is model-agnostic, provided access to noise prediction or score-based outputs.

## 3. Turbo-DDCM: Runtime Optimization and Multi-Atom Selection

DDCM with single-codeword selection is computationally intensive due to the large number of denoising steps (typically 500–1000). Turbo-DDCM accelerates encoding and decoding by allowing the selection of multiple codewords (atoms) per denoising step, transforming the codeword combination problem into a constrained least-squares objective [2511.06424]:
\[
\min_{w \in \mathbb{R}^K} \| C_t w - r_t \|_2^2  \quad \text{s.t. } \|w\|_0 = M, \; w_i \in V \cup \{0\}, \; V=\{ \pm 1 \}
\]
A hard-thresholding solution computes the \( M \) largest absolute codebook correlations and forms a normalized sum:
- Select atom indices \( S \) corresponding to top \( M \) entries.
- Set signs \( w_S = \mathrm{sign}(\alpha_S) \).
- Output \( z_t = \mathrm{normalize}(C_t w) \).

This approach maintains residual reduction comparable to matching pursuit but at only \( \Theta(K d) \) operations per step, independent of \( M \), and drastically reduces the required number of denoising steps (e.g., \( T \) drops from 1000 to \( \approx 20 \)), yielding observed runtime improvements of over \( 40\times \) relative to DDCM [2511.06424]. Bitstream representation encodes the lexicographic index of the unordered \( M \)-subset, plus coefficient quantization, efficiently packing information with minimal overhead.

## 4. Extensions: gDDCM, Conditional, and Priority-Aware Variants

Generalized DDCM (gDDCM) extends the paradigm to alternative diffusion forms, including score-based SDEs, consistency models, and flow-matching (rectified flow), exploiting the shared marginal structure \( x_t = s(t)x_0 + \Sigma(t)\epsilon \) [2511.13387]. Noise tokenization uses nearest-neighbor codebook assignment in noise space, with variants adjusting the stage at which codebook injection occurs (pure vs. two-stage). 

Specialized Turbo-DDCM variants further enhance flexibility [2511.06424]:
- **Priority-aware Turbo-DDCM** modifies the residual by spatial weights, enhancing ROI quality without increasing overall BPP.
- **Distortion-controlled Turbo-DDCM** predicts PSNR from JPEG size proxies to select BPP, reducing PSNR variance by >40%.

DDCM selection objectives extend naturally to conditional and guided scenarios, yielding compressed posterior samples for image restoration, classifier guidance, and perception-optimized reconstructions—all while emitting lossless discrete code streams [2502.01189].

## 5. Computational Complexity, Scaling, and Practical Advantages

Classic DDCM with matching pursuit refinement exhibits \( \Theta(M 2^C K d) \) per-step complexity and scaling with \( T=500\text{--}1000 \) neural queries. Turbo-DDCM reduces the sampling to \( T\approx 20 \) with \( \Theta(K d) \) per step. For typical image compression benchmarks, this yields per-image runtime of 1.5 seconds (A40 GPU), compared to 65 seconds for DDCM and 4 seconds for DiffC [2511.06424]. The Turbo approach achieves near-constant runtime across a wide range of BPPs. No custom kernels or hardware-specific acceleration is required.

## 6. Empirical Results and Applications

Across standard test sets (Kodak24, DIV2K, CIFAR-10, LSUN Bedroom), DDCM and Turbo-DDCM deliver competitive or state-of-the-art Pareto rate–distortion–perception trade-offs [2502.01189, 2511.06424, 2511.13387]. Representative findings:
- At \( \mathrm{BPP} \approx 0.05 \), Turbo-DDCM achieves PSNR ≈ 24 dB, LPIPS ≈ 0.12, FID ≈ 9.5 within 1.5 seconds/image.
- DDCM-encoded samples match the diversity/FID of full-precision DDPMs even with codebooks of size \( K = 64 \).
- Perceptual compression with DDCM produces visually faithful reconstructions at bitrates down to 0.02 BPP, outperforming JPEG/BPG at ultra-low rates.
- Adaptive variants recover ROIs with superior clarity and target PSNRs with substantially lower prediction error.

gDDCM yields further improvements, outperforming original DDCM across all quality metrics and unifying token-based compression across diffusion and flow-matching paradigms [2511.13387].

## 7. Limitations, Open Problems, and Outlook

While DDCM-based methods enable efficient, high-fidelity compressed generation, three principal limitations are noted [2511.06424, 2511.13387]:
- Turbo-DDCM still requires tens of backbone inferences per image; a single-step, truly zero-latency compression method remains an open problem.
- At high BPPs, compression performance is ultimately bounded by the expressiveness and distortion of the upstream latent diffusion encoders.
- Theoretical understanding of optimal codebook size, codeword sparsity, and token-allocation schedules in the continuous-to-discrete regime is incomplete and constitutes an active area of research.

Future directions include end-to-end learning of codebook schedules, extension to multimodal and video domains, integration into standardized codecs, and deeper analysis of the relationship between discrete tokenization and score-based posterior sampling.

---

**Key references:**  
- Ohayon et al., "Compressed Image Generation with Denoising Diffusion Codebook Models" [2502.01189]  
- Ohayon et al., "Turbo-DDCM: Fast and Flexible Zero-Shot Diffusion-Based Image Compression" [2511.06424]  
- Ohayon et al., "Generalized Denoising Diffusion Codebook Models (gDDCM): Tokenizing images using a pre-trained diffusion model" [2511.13387]

Source: https://www.emergentmind.com/topics/denoising-diffusion-codebook-models-ddcms