---
title: Residual Null-Space Diffusion SDE
url: https://www.emergentmind.com/topics/residual-null-space-diffusion-sde-rn-sde
type: topic
---

# Residual Null-Space Diffusion SDE

Residual Null-Space Diffusion Stochastic Differential Equations (RN-SDEs) are a class of generative diffusion models designed for the limited-angle computed tomography (LACT) reconstruction problem. LACT is fundamentally ill-posed due to the absence of scanning information at certain projection angles, resulting in severe reconstruction artifacts. The RN-SDE framework combines a mean-reverting stochastic differential equation (SDE) prior with rigorous data consistency enforced via range-null space decomposition (RNSD), supplemented by learned neural components for both the diffusion prior and the pseudo-inverse of the tomographic forward operator. This approach enables accurate recovery of high-fidelity images in settings with strong data incompleteness [2409.13930].

## 1. Mathematical Foundation: SDEs in RN-SDE

The RN-SDE framework is grounded in a mean-reverting SDE formulation:

\[
\mathrm{d}x_t = \theta_t(\mu - x_t)\,\mathrm{d}t + \sigma_t\,\mathrm{d}W_t
\]

where \( x_t \in \mathbb{R}^n \) is the image at time \( t \), \( \mu \) is a low-fidelity prior (e.g., obtained via NafNet or filtered back projection), \( \theta_t > 0 \) modulates mean reversion, and \( \sigma_t \) is the diffusion coefficient. The process is driven by the Wiener process \( W_t \).

The reverse-time SDE (sampling direction) before rectification is expressed as:

\[
\mathrm{d}x_t = [\theta_t(\mu - x_t) - \sigma_t^2 \nabla_x \log p(x_t)]\,\mathrm{d}t + \sigma_t\,\mathrm{d}\widehat W_t
\]

where the term \( \nabla_x\log p(x_t) \) (the score function) is learned via denoising score matching. The final reverse SDE, after rectification using RNSD, is:

\[
\mathrm{d}x_t = [(G_t + \theta_t\,\mathrm{d}t)(\mu - x_t) + H_t(\mu - \hat{x}_{0|t})] + \sigma_t\,\mathrm{d}\widehat W_t
\]

with coefficients \( G_t \) and \( H_t \) determined by the SDE parameters and \( \hat{x}_{0|t} \) given by a consistency-enforcing projection (see Section 2) [2409.13930].

## 2. Range-Null Space Decomposition and the Residual Null-Space

For a linear forward operator \( A \) (e.g., Radon transform) and pseudo-inverse \( A^\dagger \), any image can be decomposed as:

\[
x = A^\dagger A x + (I - A^\dagger A)x
\]

where \( A^\dagger A x \) is the range-space component (measurable by the tomographic system) and \( (I - A^\dagger A)x \) is the null-space component (unmeasured by the system). RN-SDE applies this decomposition to every denoised estimate \( x_{0|t} \), enforcing data consistency by replacing the range-space part with the minimum mean square error (MMSE) solution \( A^\dagger y \), while the null-space is filled by the generative model. The data-consistent projection is:

\[
\hat{x}_{0|t} = A^\dagger y + (I - A^\dagger A)x_{0|t}
\]

Importantly, this operation alters only the range-space, leaving the learned null-space component intact. This facilitates the integration of explicit measurement fidelity with deep prior-driven synthesis [2409.13930].

## 3. RN-SDE Sampling and Reconstruction Pipeline

The RN-SDE reconstruction algorithm proceeds as follows:

1. **Initialization**: Sample \( x_T \sim \mathcal{N}(\mu, \lambda^2 I) \), i.e., Gaussian noise centered at the low-fidelity prior \( \mu \).
2. **Iterative Denoising** (for \( t = T \) to 1):
   - **Prediction**: Compute the "clean" image estimate
     \[
     x_{0|t} = -\frac{G_t}{H_t}x_t + \frac{\sigma_t^2}{H_t}s_\phi(x_t, \mu, t)\,\mathrm{d}t + (1+\frac{G_t}{H_t})\mu
     \]
     where \( s_\phi \) is a conditional score network (Cond-NafNet).
   - **RNSD Rectification**: Enforce measurement consistency via
     \[
     \hat{x}_{0|t} = A^\dagger y + (I - A^\dagger A)x_{0|t}
     \]
     Optionally, corrections may be downscaled or skipped periodically to compensate for inaccuracies in \( \hat{A}^\dagger \).
   - **Sampling Step**: Draw the next state \( x_{t-1} \) from the closed-form transition specified in the rectified reverse SDE, with added Gaussian noise.
3. **Output**: The final estimate \( x_0 \) provides the reconstructed, high-quality CT image.

This algorithm achieves direct enforcement of projection-domain consistency alongside progressive null-space restoration guided by learned diffusion priors [2409.13930].

## 4. Model Training and Loss Functions

The RN-SDE framework incorporates two neural components:

- **Score network** (\( s_\phi(x_t, \mu, t) \)), trained via denoising score matching:
  \[
  \mathbb{E}_{t\sim U[0,T], x_0\sim p_{\rm data}} \left\| \nabla_x \log p(x_t|x_0) - s_\phi(x_t, \mu, t) \right\|^2
  \]
  An equivalent maximum-likelihood objective is derived for practical training.
  
- **Pseudo-inverse network** (\( \hat{A}^\dagger_\eta \)), optimized with a two-term loss:
  \[
  \ell_1 = \mathbb{E}_x \left\| A x - A \hat{A}^\dagger_\eta A x \right\|,\quad
  \ell_2 = \mathbb{E}_y \left\| \hat{A}^\dagger_\eta y - \hat{A}^\dagger_\eta A \hat{A}^\dagger_\eta y \right\|
  \]
  \[
  \ell_{\rm pseudo} = (1-\alpha)\ell_1 + \alpha\ell_2,\,\,\alpha\in[0, 1]
  \]

These objectives enforce accurate learning of both the data-driven null-space prior and a stable, approximately MMSE pseudo-inverse for consistent range-space projection [2409.13930].

## 5. Theoretical Guarantees and Stability Properties

RN-SDEs possess several notable theoretical properties:

- The mean-reverting SDE ensures \( x_t \sim \mathcal{N}(\mu, \lambda^2 I) \) as \( t \to \infty \), anchoring the generative process around the low-fidelity prior.
- Projecting onto the range space with \( A^\dagger A \) guarantees \( \|A \hat{x} - y\| = 0 \) for measured angles at every iteration, precluding drift in the measurement-consistent space.
- The process yields a stable fixed point: once the null-space is filled in by the diffusion-driven prior and the range-space matches the acquisition, the algorithm ceases to alter the result.
- Experimental evidence demonstrates rapid mixing and convergence, with RN-SDE requiring several hundred sampling steps versus thousands in vanilla DDPMs under severe data undersampling conditions. No explicit formal convergence rate is provided [2409.13930].

## 6. Empirical Results and Computational Considerations

Extensive evaluations were performed on ChromSTEM and C4KC-KiTS LACT datasets, contrasting RN-SDE against baseline and state-of-the-art diffusion frameworks. Summary metrics include:

| Dataset      | Angle (°) | RN-SDE PSNR (dB) | DOLCE PSNR (dB) | RN-SDE SSIM | DOLCE SSIM | RN-SDE LPIPS | DOLCE LPIPS |
|--------------|-----------|------------------|-----------------|-------------|------------|--------------|-------------|
| ChromSTEM    | 60        | 43.94            | 42.61           | 0.982       | 0.978      | 0.0069       | 0.0119      |
| ChromSTEM    | 90        | 40.43            | 38.47           | -           | -          | 0.0086       | 0.0140      |
| ChromSTEM    | 120       | 35.62            | 31.36           | -           | -          | -            | -           |

*Dashes indicate metrics not explicitly reported at some angles.*

Runtime and efficiency profiling on a Quadro RTX 8000 GPU revealed:

- DOLCE: ≈4634 s (ChromSTEM, 1000 steps, 53.4 kGMac, 93M params)
- RN-SDE: 66 s (ChromSTEM, 593 steps, 6.02 kGMac, 70M params)
- C4KC-KiTS: DOLCE ≈10,875 s, RN-SDE 106 s

RN-SDE thus attains higher perceptual and fidelity scores with a 50–100× reduction in sampling time relative to prior diffusion-based LACT solvers [2409.13930]. 

## 7. Integration and Impact on Limited-Angle CT

RN-SDE tightly integrates a mean-reverting SDE prior for accelerated mixing, a conditional score network (Cond-NafNet), a neural pseudo-inverse for \( A^\dagger \), and an RNSD-based rectification to guarantee exact data consistency throughout the reconstruction process. The architectural synergy enables both principled theoretical guarantees and empirically validated improvements in speed and accuracy. These results underscore the framework's efficacy in limited-angle CT and related inverse imaging tasks [2409.13930].

Source: https://www.emergentmind.com/topics/residual-null-space-diffusion-sde-rn-sde