Diffusion-Based Image Compression

• Diffusion-based image compression is a generative coding paradigm that uses denoising diffusion probabilistic models to reconstruct high-fidelity images from compressed data.
• It employs adaptive encoding, quantization, and entropy coding to achieve precise rate–distortion–perception tradeoffs and progressive image refinement.
• Recent advances include zero-shot posterior sampling, transformer-based decoders, and region-adaptive techniques that significantly enhance perceptual quality at low bitrates.

Diffusion-based image compression (DBIC) denotes a generative coding paradigm in which a diffusion model, typically a denoising diffusion probabilistic model (DDPM), is leveraged as a powerful prior to reconstruct perceptually faithful images from highly compressed representations. DBIC frameworks depart from classical deterministic codecs by explicitly exploiting the expressive capacity and inverse problem-solving properties of diffusion models, allowing reconstructions with high realism even at extremely low bitrates. The field encompasses a broad array of methodologies, including zero-shot posterior-sampling schemes, entropy-coded progressive diffusion chains, adaptive latent coding, and rapid transformer-based architectures. This article surveys the theoretical foundation, principal families of DBIC algorithms, rate-distortion-perception tradeoffs, methodological advancements for efficient and flexible coding, and open research problems in the domain.

1. Diffusion-Based Compression: Fundamental Principles

DBIC rests on the probabilistic modeling capacity of diffusion generative models, which learn expressive priors $p(x)$ capable of capturing the true data distribution via iterative noise corruption and denoising. The typical pipeline operates as follows:

• Encoding: The input image $x\in\mathbb{R}^D$ is mapped to a lower-dimensional latent (via analysis transform, VAE, or learned embedding), or linear measurements $y = Hx$ are acquired.
• Quantization & Entropy Coding: Latent representations or measurements are quantized, then entropy-coded (via hyperpriors, range encoders).
• Decoding: The generative prior (diffusion model) is used to reconstruct $x$ from the compressed representation, either by conditional iterative sampling, ODE-based refinement, or a one-step denoising inversion.

Crucially, the generative model may perform zero-shot posterior sampling conditioned on measurements $y$—as in posterior sampling-based transform coding (Elata et al., 2024)—or decode via progressive transmission and universal quantization (Yang et al., 2024). This modeling allows the decoder to sample from $p(x|y)$, guaranteeing high-perceptual reconstructions even when the transmitted information is extremely sparse.

Rate-distortion-perception theory underpins DBIC evaluation, quantifying the fundamental tradeoff between bit cost ($R$), distortion ($D$; e.g., PSNR), and perceptual realism ($P$; e.g., FID) (Yang et al., 26 Jan 2026).

2. Methodological Taxonomy of DBIC Architectures

Several dominant families of DBIC methods have crystallized:

• Posterior Sampling–Based Compression (PSC): Constructs adaptive transform bases $H$ via zero-shot posterior sampling, selecting rows to greedily maximize information gain and utilizing pre-trained diffusion models. PSC operates with no additional training and is highly rate-flexible, as rate/distortion tradeoffs are controlled at inference by the number of measurements transmitted. The decoder reconstructs $x\in\mathbb{R}^D$0 from $x\in\mathbb{R}^D$1 without explicit side-information, using the same seed and quantization protocol (Elata et al., 2024).
• Universally Quantized Progressive Diffusion (UQDM): Replaces Gaussian forward processes with uniform-noise channels, allowing end-to-end universal quantization during transmission. The negative Evidence Lower Bound (ELBO) corresponds to the bit-cost, and partial bitstreams yield progressively refined reconstructions, ensuring a single model covers the ultra-low to lossless regime without retraining (Yang et al., 2024, Relic et al., 3 Apr 2025).
• Entropy-Coded Latent Diffusion and Transformers: Modern approaches (e.g., DiT-IC) replace U-Net backbones with transformers capable of one-step denoising in highly downsampled latent spaces (e.g., $x\in\mathbb{R}^D$2), achieving up to $x\in\mathbb{R}^D$3 faster decoding at comparable or better perceptual fidelity. Variance-guided reconstruction flows and self-distillation enforce consistent latent geometry (Shi et al., 13 Mar 2026).
• Zero-Shot Codebook-based and Turbo schemes: Denoising Diffusion Codebook Models (DDCM), including Turbo-DDCM, seek compressed codes by encoding selections from large noise vector codebooks per step. Turbo variants compress by linear combinations, drastically reducing required diffusion steps (from $x\in\mathbb{R}^D$41000 to 20), supporting ROI or distortion-based control, while maintaining flexible bitrate adaptation (Vaisman et al., 9 Nov 2025).
• Region- and Content-Adaptive and Semantic Guidance: Models such as CADC (Sheng et al., 25 Feb 2026) and region-adaptive codecs (Relic et al., 1 Apr 2026) adjust quantization or diffusion schedules spatially, focusing capacity on salient or complex image regions. Semantic guidance, through hyperprior-derived embedding or textual cues, further aligns generative priors to invented content under severe compression.

The table below summarizes representative methods:

Method	Key Compression Approach	Rate Control
PSC	Adaptive posterior transform + DDPM	Inference (N rows)
UQDM	Universal quantization in diffusion	Stepwise (progressive)
DiT-IC	One-step latent transformer	VAE + latent entropy
Turbo-DDCM	Greedy codebook combination	Codebook sparsity
Region-adapt.	Spatially-varying noise/quant.	ROI, per-pixel maps

3. Mathematical Formalism and Posterior Sampling Algorithms

Mathematically, DBIC formalizes lossy compression as a statistical inverse problem under a generative prior: $x\in\mathbb{R}^D$5 where $x\in\mathbb{R}^D$6 (linear measurements), $x\in\mathbb{R}^D$7 is a (possibly scalar) quantizer, and $x\in\mathbb{R}^D$8 is the diffusion model prior. PSC algorithms greedily select the next measurement direction $x\in\mathbb{R}^D$9 as the principal eigenvector of the current posterior covariance, iteratively constructing $y = Hx$0 to maximize information gain (Elata et al., 2024).

Zero-shot posterior samplers (e.g., DDRM, SNIPS) simulate samples $y = Hx$1 without retraining. A singular value decomposition (SVD) over posterior samples identifies orthogonal measurement directions, adaptive to the uncertainty structure in $y = Hx$2.

In progressive schemes such as UQDM, the negative ELBO term aligns with compression cost via a chain of uniform-noise channels. Universal quantization with public dither ensures continuous-valued latents can be efficiently entropy-coded with negligible distortion gaps, and every step’s bits incrementally refine reconstruction (Yang et al., 2024, Relic et al., 3 Apr 2025).

4. Entropy Coding, Quantization, and Rate Control

Quantization strategies in DBIC are shaped by the specific architecture:

• PSC: Simple float-to-float mapping (e.g., float32$y = Hx$3float8) or advanced scalar quantizers are deployed, exploiting the approximate whitening properties of orthonormal $y = Hx$4. The resulting symbols are compressed with range encoders (ANS-based), and bitrates are empirically measured as BPP (Elata et al., 2024).
• Latent Diffusion/Transformers: Quantization occurs in highly downsampled latent domains (e.g., $y = Hx$5 for DiT-IC), using context models (hyperprior + autoregressive) to predict per-block entropy. Adaptive quantization, including uncertainty-guided scaling (spatial SNR maps) or content-aware entropy modeling, facilitates flexible bitrate control (Shi et al., 13 Mar 2026, Sheng et al., 25 Feb 2026).
• Progressive and Codebook-based: Transmission is often realized through universal quantization and entropy coding, or in codebook methods, by mapping index sets or sparse combinations into bitstreams with near-optimal encoding schemes (Yang et al., 2024, Vaisman et al., 9 Nov 2025).

Rate flexibility is a distinguishing feature in DBIC: both progressive diffusion (by truncating bitstreams or decoding after variable steps) and adaptive schemes (selecting transform/quantization parameters) enable arbitrarily fine-grained tradeoffs between rate and perceptual fidelity without retraining (Elata et al., 2024, Yang et al., 2024).

5. Rate–Distortion–Perception Performance and Empirical Analysis

Performance of DBIC schemes is quantified along the rate–distortion–perception surface:

• Distortion: Measured via PSNR, MS-SSIM, or $y = Hx$6 error.
• Perceptual Quality: Evaluated with FID, LPIPS, or DISTS.
• Flexibility: Measured by attainable bitrates, inference-time controllability, and potential for partial decoding.

Empirical results demonstrate that modern DBIC codecs (PSC, UQDM, DiT-IC, StableCodec) outperform legacy codecs (JPEG, JPEG2000, BPG), and are competitive with or superior to state-of-the-art learned codecs (HiFiC, ELIC, PerCo), especially at low BPP. For example, PSC combined with pseudoinverse-guided diffusion achieves the minimal FID in BPP < 0.1 (Elata et al., 2024). UQDM produces continuous improvements as bitrate rises, in contrast to neural codecs that plateau (Yang et al., 2024). DiT-IC achieves up to 30$y = Hx$7 speedup over U-Net-based codecs while improving BD-rate on LPIPS and DISTS (Shi et al., 13 Mar 2026). Region-adaptive codecs set new lows for LPIPS and user preference in ROI-masked settings (Relic et al., 1 Apr 2026).

Decoding times and computational footprint vary dramatically: classical multi-step DDPM decoders are slow (often minutes per megapixel), whereas one-step diffusion (DiT-IC, StableCodec, OSDiff) and two-step refinement (DiffCR) approaches match the latency of transform coding or neural codecs (Shi et al., 13 Mar 2026, Zhang et al., 27 Jun 2025, Xia et al., 15 Jan 2026, Jia et al., 2 Feb 2026).

6. Flexibility, Adaptivity, and Practical Considerations

Recent efforts focus on augmenting the adaptivity and usability of DBIC:

• Adaptive Transform and Quantization: Content- or uncertainty-driven schedules allow spatially variable allocation of bitrate and generative effort, efficiently capturing salient structure while reducing rate in unimportant regions (Sheng et al., 25 Feb 2026, Relic et al., 1 Apr 2026).
• Semantic and Region Control: Cross-attention over hyperpriors, learned semantic embeddings, or side information (text, edge maps, region importance) enables spatial fidelity, ROI targeting, and robust control over generation (Relic et al., 1 Apr 2026, Vaisman et al., 9 Nov 2025).
• One-Step/Few-Step Generation: Distilled diffusion models and transformer backbones permit collapsing the iterative denoising process into a single or few steps, with negligible loss in realism or fidelity and dramatically improved runtime (Zhang et al., 27 Jun 2025, Shi et al., 13 Mar 2026, Jia et al., 2 Feb 2026).
• Zero-Shot and Foundation Model Approaches: Methods like PSC and codebook-based Turbo-DDCM leverage fixed, pre-trained foundation models (e.g., Stable Diffusion, DiT) with new compression logic, enabling rapid deployment across domains without retraining (Elata et al., 2024, Vaisman et al., 9 Nov 2025).
• User Configurable Bitrate/Distortion Targets: By design, progressive and codebook-based methods allow users to specify rate or distortion targets dynamically at inference, supporting a wide range of downstream requirements (Vaisman et al., 9 Nov 2025).

Resource and complexity tradeoffs remain a practical concern: high-quality zero-shot methods can demand substantial compute for posterior sampling or large base model storage, though ongoing architectures (e.g., DiT-IC, OSDiff) show significant improvements.

7. Open Problems and Future Research Directions

Key open challenges and emerging directions in DBIC include:

• Theoretical Rate–Distortion–Perception Characterization: Precise information-theoretic characterization of rate–perception tradeoffs under common randomness, channel simulation protocols, and non-Gaussian/noisy latent spaces (Yang et al., 26 Jan 2026).
• Acceleration of Posterior Sampling and Decoding: Development of general-purpose one-/few-step samplers, possibly by progressive distillation, consistency models, or transformer-based generative flows, to close the gap with real-time requirements (Shi et al., 13 Mar 2026, Xia et al., 15 Jan 2026, Jia et al., 2 Feb 2026).
• Joint Encoder–Generator Optimization: Unified end-to-end training of encoder, quantizer, entropy model, and diffusion prior for globally optimal rate–distortion–perception performance across operating points remains an open problem (Jia et al., 24 Nov 2025, Sheng et al., 25 Feb 2026).
• Region- and Semantics-Driven Coding: Improved models of saliency, attention, and importance-driven scheduling, both for foveated displays and semantic preservation in task-driven pipelines (Relic et al., 1 Apr 2026).
• Scalability and Foundation Models: Generalization to very high resolutions, video, and multimodal content, potentially using fully open, compression-specific foundation models (CoD) (Jia et al., 24 Nov 2025).
• Stochastic Coding and Channel Simulation: Efficient algorithms for channel simulation (beyond uniform-noise approximation), and theoretical advances in practical stochastic coding for perceptual generative codecs (Yang et al., 2024, Relic et al., 3 Apr 2025).

These research avenues suggest DBIC will remain a focal point in the development of next-generation image codecs, with ongoing innovation spanning theory, architectures, and practical deployment.