---
title: Denoising Diffusion Null-space Model (DDNM)
url: https://www.emergentmind.com/topics/denoising-diffusion-null-space-model-ddnm
type: topic
---

# Denoising Diffusion Null-space Model (DDNM)

The Denoising Diffusion Null-space Model (DDNM) is a general and mathematically principled paradigm for generative inference and image restoration, leveraging a pretrained denoising diffusion generative prior and explicit range–null space decomposition to efficiently solve arbitrary linear inverse problems. DDNM extends classical diffusion models as a specific implementation within the broader framework of denoising Markov models, allowing for direct enforcement of measurement consistency while refining the unconstrained components through iterative reverse diffusion steps. This approach achieves both rigorous data consistency and high perceptual image fidelity, making DDNM applicable across imaging, simulation-based inference, and related domains characterized by ill-posedness and partial observability.

## 1. Mathematical Foundations and Score Matching

DDNM operates within the "denoising Markov model" (DMM) framework, generalized via Markov generators acting on general state spaces (e.g., $\mathbb{R}^d$, graphs, manifolds) [2211.03595]. The forward ("noising") process is defined by a Markov generator $L$, while the backward (generative) process uses a generator $K$ that ideally inverts $L$. The reverse process is parameterized via a function $\beta(x, t)$, which generalizes score matching:

$$
\Phi(f) = \frac{Lf}{f} - L(\log f)
$$

Specializing to the classical Euclidean case, the denoising score becomes $s_{(\theta)}(x, t) = \nabla_x \log \beta(x, t)$, and the loss is formulated as

$$
I_{DSM}(\theta) = \frac{1}{2} \int_0^T \mathbb{E}_{q_0, x_t} [\, \| \nabla_x \log q_{t|0}(x_t|x_0) - s_{(\theta)}(x_t, t) \|^2 ]\, dt
$$

More generally, implicit score matching extends this to arbitrary state spaces and operator pairs $(L, K)$, and can be written as

$$
I_{ISM}(\beta) = \int_0^T \mathbb{E}_{q_t} \left[ \frac{\hat{L}^* \beta(x, t)}{\beta(x, t)} + \hat{L} \log \beta(x, t) \right] dt
$$

This generalization allows for score matching in complex and structured spaces far beyond vector-valued images.

## 2. Null-space Decomposition and Consistency Enforcement

A central principle of DDNM is decomposition into range and null space for a given linear observation or degradation operator $A$ or $L_{obs}$. Any point $x$ in the ambient space can be split as

$$
x = A^\dagger A x + (I - A^\dagger A)x
$$

where $A^\dagger$ is the Moore–Penrose pseudoinverse and $(I - A^\dagger A)x$ projects onto the null space of $A$. In DDNM, the observed component $A^\dagger y$ is pinned to enforce exact consistency ($A \hat{x} = y$), while the null space is iteratively refined under the pretrained diffusion prior [2212.00490]. Thus, for each reverse diffusion step:

$$
\hat{x}_{0|t} = A^\dagger y + (I - A^\dagger A) x_{0|t}
$$

This explicit separation guarantees that the restored image always matches the measured data, while unconstrained degrees of freedom benefit from probabilistic synthesis.

## 3. Sampling, Conditioning, and Loss Functions

The generative reverse process is conditioned on the measurements (or observations) and accommodates a null–space correction during each step. The reverse diffusion, parameterized with a conditional score or equivalently $\beta(x, y, t)$, follows:

$$
s_{(\theta)}(x, y, t) \approx \nabla_x \log q_t(x \mid y)
$$

and the extended score matching objective for conditional restoration reads

$$
I_{DSM}(\beta) = \int_0^T \mathbb{E}_{q(x_0, x_t, y)} [\, \Phi\big( q(\cdot|0)/\beta(\cdot, y, \cdot) \big)\, \text{at}\ x_t\, ]\, dt
$$

A key theoretical guarantee is the explicit evidence lower bound (ELBO):

$$
\mathcal{E}^{\infty} = \mathbb{E}_Q \left[ \log p_0(Y_T) \mid Y_0 = x \right] - \int_0^T \mathbb{E}_Q \left[ \frac{\hat{L}^* \beta(Y_s, s)}{\beta(Y_s, s)} + \hat{L} \log \beta(Y_s, s) \mid Y_0 = x \right] ds
$$

DDNM's ELBO is maximized when the learned reverse process exactly induces the correct posterior.

## 4. Practical Applications and Enhancements

DDNM is applied to a wide range of linear restoration tasks without retraining or task-specific network modification [2212.00490]:

- **Super-resolution:** $A$ is downsampling (e.g., bicubic), $A^\dagger$ restores spatial resolution; DDNM supports up to $256\times$, outperforming SOTA zero-shot methods.
- **Colorization:** $A$ reduces RGB to grayscale; DDNM reconstructs chromaticity in the null space.
- **Inpainting:** $A$ is a mask; missing regions are refined while observed pixels are strictly preserved.
- **Compressed sensing:** $A$ is sampling matrix; DDNM recovers missing coefficients while enforcing measurement constraint.
- **Deblurring:** $A$ is convolution kernel; $A^\dagger$ inverts the blur.

The DDNM+ variant [2212.00490] enhances robustness by scaling range–space correction and adjusting variance in the reverse sampling step to accommodate noise propagation, especially in noisy or hard scenarios (e.g., “time-travel” trick).

## 5. Model Generalization, Limitations, and Theoretical Guarantees

The denoising Markov model formalism allows for the application of DDNM machinery to general state spaces—discrete, continuous, manifold-valued—by choosing corresponding generators ($L$) and their adjoints ($K$) [2211.03595]:

- **Discrete grid:** CTMC generators can be designed for image grids; null-space corresponds to missing pixels.
- **Manifold-valued data:** For pose estimation, generators on SO(3) or simplex-valued data are supported.
- **Simulation-based inference:** The framework accommodates hybrid models where only a forward oracle is available and inverting the mapping is ill-posed.

The framework guarantees, under technical regularity conditions (on space, generator, and score function), that the learned conditional score in the null space approximates the true posterior gradient. Limitations arise due to discretization errors, score approximation quality, and potential imbalance between range and null-space parameterization.

## 6. Connections to Related Diffusion Models and Optimization Perspectives

DDNM generalizes and connects to recent diffusion methods for inverse problems, denoising, and sampling:

- **Denoising Diffusion Samplers (DDS):** Uses a reverse KL divergence objective, and many theoretical results on convergence and sampling in unnormalized densities extend to null-space-constrained settings [2302.13834].
- **Optimization perspective:** Interpretation of denoising diffusion as gradient descent steps on a distance-to-manifold function naturally aligns with the structure of DDNM. Null-space constraints are enforceable as projection operations within the iterative sampling process [2306.04848].
- **Score Matching Extensions:** DDNM's training objective can be viewed as a generalized score matching (implicit or conditional), providing a unified view of restoration, posterior inference, and generative synthesis.

## 7. Implementation Considerations and Deployment

A practical DDNM implementation requires:

- Definition of $A$ and $A^\dagger$; explicit range–null space projection is crucial for correct data consistency.
- Adoption of a pretrained diffusion model (e.g., DDPM, score-based diffusion) as the generative prior.
- Modification of each reverse step to replace the range-space part with the measured data, while refining the null-space via the denoiser.
- For noisy data, DDNM+ adjusts the correction and variance schedules to suppress noise amplification.
- DDNM is parameter-free and does not require retraining for new tasks or degradation operators.

The approach is compatible with recent enhancements for arbitrary image sizes (e.g., mask-shift restoration, hierarchical restoration) [2303.00354], deployment in wireless image delivery [2309.15889], and stochastic degradation models with direct consistency guidance [2303.14353].

---

In summary, the Denoising Diffusion Null-space Model embodies a mathematically robust, highly generalizable, and empirically competitive strategy for conditional generative modeling and inverse problem solving. Its explicit null-space methodology enables zero-shot restoration across diverse problems while guaranteeing data fidelity—grounded in the foundational denoising Markov model framework and extended by rigorous score matching objectives, ELBO-based training, and modular deployment in research and practical scenarios.

Source: https://www.emergentmind.com/topics/denoising-diffusion-null-space-model-ddnm