LatentUnfold: Unified Blind Image Restoration

• LatentUnfold is a unified framework for blind image restoration that integrates interpretable optimization, nonparametric degradation modeling, and conditional latent diffusion priors.
• It employs a multi-stage architecture featuring a Multi-Granularity Degradation-Aware module, a Degradation-Resistant Latent Diffusion Model, and an Over-Smoothing Correction Transformer.
• Experimental results across benchmarks like SIDD, GoPro, and LOL-v2 demonstrate state-of-the-art performance in PSNR and SSIM, highlighting its robustness and efficacy.

LatentUnfold (formally UnfoldLDM) is a unified deep unfolding network for blind image restoration (BIR), integrating interpretable optimization principles, nonparametric degradation modeling, and conditional latent diffusion priors within a multi-stage architecture. Developed to address the dual limitations of degradation-specific dependency and over-smoothing bias inherent in classical Deep Unfolding Networks (DUNs), LatentUnfold introduces a three-part pipeline: Multi-Granularity Degradation-Aware modeling, Degradation-Resistant Latent Diffusion Priors, and Over-Smoothing Correction Transformers. This plug-and-play structure achieves state-of-the-art results across a wide spectrum of BIR tasks by jointly estimating the unknown degradation and restoring structured, high-frequency image details (He et al., 22 Nov 2025).

1. Blind Restoration Optimization Formulation

The blind restoration model observes a degraded image $y \in \mathbb{R}^{c \times h \times w}$ generated via an unknown linear process:

$y = D x + n$

where $x$ is the latent clean image, $D$ is an unknown degradation matrix, and $n$ is additive noise. To capture structure and reduce complexity, $D$ is factorized as a Kronecker product:

$D = M^T \otimes W$

with $W \in \mathbb{R}^{c \times h \times h}$ and $M \in \mathbb{R}^{c \times w \times w}$. The energy minimization for restoration is

$$L(x) = \frac{1}{2} \|y - D x\|_2^2 + \frac{1}{2} \|y - W x M\|_2^2 + \lambda \phi(x)$$

where $y = D x + n$0 is a learned image prior and $y = D x + n$1 weights regularization. The problem is solved via $y = D x + n$2-stage proximal-gradient unfolding, applying block-coordinate descent over fidelity terms $y = D x + n$3 and $y = D x + n$4 followed by a learned proximal operator. Specifically, at each stage $y = D x + n$5:

• $y = D x + n$6
• $y = D x + n$7
• $y = D x + n$8

2. Multi-Granularity Degradation-Aware (MGDA) Module

MGDA replaces analytic gradients with data-driven surrogates, enabling end-to-end handling of unknown degradations:

• Holistic Degradation: Two Siamese Visual State Space (VSS) networks ($y = D x + n$9, $x$0) estimate $x$1 and its transpose, producing

$x$2

• Structured Decomposition: Neural blocks $x$3 and $x$4 alternately estimate components $x$5 and $x$6, constructed via normalized outputs from concatenated feature maps. The structured fidelity update is

$x$7

An intra-stage consistency loss

$x$8

promotes alignment between holistic and structural branches.

3. Degradation-Resistant Latent Diffusion Model (DR-LDM)

The proximal operator in LatentUnfold is realized by a conditional latent diffusion model designed for degradation invariance:

• Latent Prior Extraction: In Phase I, a Prior Inference (PI) network maps $x$9 to a compact latent prior $D$0.
• Diffusion Forward: For $D$1 steps,

$D$2

with $D$3, $D$4.

• Diffusion Reverse: A denoising network $D$5 predicts noise given noisy latent prior and a conditioning vector $D$6. The recursion is:

$D$7

After T steps, the sampled prior $D$8 is passed to the detail recovery module.

4. Over-Smoothing Correction Transformer (OCFormer)

OCFormer is a U-shaped network that fuses intermediate results and the diffusion posterior to restore high-frequency textures:

• Degradation-Resistant Attention (DRA): Features from $D$9 are enriched by learning self-attention weights through mixed $n$0 and $n$1 depthwise convolutions:

$n$2

• Prior-Guided Detail Recovery (PDR): The prior $n$3 modulates normalized features:

$n$4

$n$5

The final output $n$6 is generated by the U-net decoder.

5. Unified Iterative Restoration Algorithm

The end-to-end unfolding procedure, as summarized in the provided pseudocode, executes $n$7 stages. Each stage alternates between MGDA steps to estimate both holistic and structured degradations, then applies DR-LDM to sample a latent prior, and finally invokes OCFormer for refined reconstruction. This approach is designed as plug-and-play; it can be integrated as a wrapper for existing DUN-based methods.

6. Training Procedures and Loss Functions

The training is phased:

• Phase I: Pretrain PI and OCFormer with

$n$8

where $n$9.

• Phase II: Train DR-LDM and fine-tune the entire framework with

$D$0

$D$1, $D$2 in practice.

7. Experimental Results and Interpretation

Implemented in PyTorch on NVIDIA H200 (K=3 stages, T=3, $D$3), LatentUnfold achieves the following on standard BIR benchmarks:

• Blind denoising: SIDD (PSNR 40.02 dB, SSIM 0.961), DND (40.06 dB, 0.958)
• Blind deblurring: GoPro (34.32 dB, 0.970), HIDE (31.85 dB, 0.948)
• Underwater: UIEB (24.70 dB, 0.947)
• Backlit: BAID (24.97 dB, 0.910)
• Low-light: LOL-v2 real (23.58 dB, 0.886); synthetic (27.92 dB, 0.957)
• Deraining: Five benchmarks, average PSNR ~39.5 dB, SSIM ~0.98

Across all cases, UnfoldLDM establishes new state-of-the-art results for blind restoration (He et al., 22 Nov 2025).

8. Significance of Latent Diffusion Priors in Blind Restoration

Standard DUNs exhibit a low-frequency bias due to the dominance of smooth components in gradient-driven updates, especially under severe or unknown degradations, leading to oversmoothing. The latent diffusion prior in DR-LDM is explicitly trained for degradation invariance, promoting generative recovery of natural high-frequency textures. Conditioning the diffusion prior on MGDA’s estimates prevents reintroduction of degraded patterns, while the bidirectional interplay—cleaner inputs aiding prior learning, and stronger priors enhancing restoration—enables sharp, artifact-free outputs for a wide variety of degradations.

In summary, LatentUnfold (UnfoldLDM) represents an advance in blind image restoration by combining interpretable model-based unfolding, neural degradation modeling, and strong generative priors, implemented in a modular, extensible framework (He et al., 22 Nov 2025).