Diffractive Deep Unfolding (DDU) for Optical Imaging

• Diffractive Deep Unfolding (DDU) is a physics-guided strategy that unrolls iterative inverse solvers to integrate optical forward models with learned optimization parameters.
• It enhances phase retrieval and spectral imaging by jointly learning step sizes, preconditioners, and denoisers while preserving diffraction physics.
• Bridging model-based and deep learning approaches, DDU achieves fast convergence and robust performance in complex optical imaging tasks.

Searching arXiv for the specified DDU papers and closely related context. Diffractive Deep Unfolding (DDU) denotes a class of hybrid, model-driven deep reconstruction methods for diffractive optical imaging in which an iterative inverse solver is unrolled into a finite-depth architecture and coupled explicitly to the optical forward model. In the reported formulations, DDU is used for phase retrieval from coded diffraction patterns and for diffractive snapshot spectral imaging (DSSI), with the common objective of retaining physics consistency while learning algorithmic components such as step sizes, preconditioners, correction modules, denoisers, and initializers (Pinilla et al., 2022, Zhuge et al., 7 Jul 2025). The framework therefore sits between conventional model-based solvers and black-box learning systems: it preserves the diffraction operator in the reconstruction loop, yet optimizes learned parameters end to end.

1. Conceptual basis and scope

In diffractive optical imaging, the measurement process is governed by a structured optical encoder, typically implemented through a diffractive optical element (DOE) or a diffractive point-spread function (PSF). DDU addresses inverse problems in which this encoder induces severe nonlinearity or ill-posedness. In the phase-retrieval setting, the task is to recover a complex-valued object from intensity-only diffraction measurements acquired through a coherent system with a DOE. In the DSSI setting, the task is to reconstruct a hyperspectral image from a 2D RGB measurement produced by multi-channel diffractive encoding (Pinilla et al., 2022, Zhuge et al., 7 Jul 2025).

The reported motivation for DDU is twofold. First, deep unfolding bounds the complexity of iterative reconstruction while retaining the efficacy of model-aware optimization. Second, embedding the forward physics inside the network makes the architecture compatible with diffractive systems whose sensing mechanisms differ from those assumed by generic CNNs or by unfolding methods developed for other compressive imagers. In this sense, DDU is not defined by a single optimizer or network topology; rather, it is defined by the unrolling of a diffractive inverse solver whose layers remain tied to the imaging operator. This suggests a broader methodological category spanning multiple diffractive modalities.

2. Forward models in diffractive imaging

For optical phase retrieval, the canonical measurement model is

$y = |D(\phi)\cdot F x|^2 + \eta,$

where $x\in\mathbb{C}^n$ is the unknown scene, $F\in\mathbb{C}^{n\times n}$ is the discrete Fourier transform, $$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$ is the DOE parameterized by its phase vector $\phi\in\mathbb{R}^n$, and $\eta$ models measurement noise such as Gaussian or Poisson noise. Different diffraction regimes—near, middle, and far—are represented by auxiliary propagation matrices so that the overall sensing matrix is $A_k=D(\phi)\cdot P_k\cdot F$ for regime $k\in\{1,2,3\}$ (Pinilla et al., 2022).

For DSSI, the reported spatial-convolutional forward model is

$$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$

where $I[x,y,\lambda]$ is the unknown hyperspectral image, $x\in\mathbb{C}^n$0 is the 2D RGB measurement, $x\in\mathbb{C}^n$1 is the diffractive PSF for spectral band $x\in\mathbb{C}^n$2, $x\in\mathbb{C}^n$3 is the camera color-channel response, and $x\in\mathbb{C}^n$4 is additive noise. After vectorization,

$x\in\mathbb{C}^n$5

with $x\in\mathbb{C}^n$6. Under circular boundary assumptions and Fourier transforms, the model becomes $x\in\mathbb{C}^n$7, where $x\in\mathbb{C}^n$8 is block-diagonal in $x\in\mathbb{C}^n$9 (Zhuge et al., 7 Jul 2025).

These two forward models clarify the operative meaning of the adjective “diffractive” in DDU. In one case the inverse problem is phaseless and nonlinear because measurements are intensities of coded diffraction patterns; in the other, the inverse problem is linear but highly underdetermined because a spectral data cube is compressed into a 2D RGB observation. The common technical requirement is that the network be constructed around the corresponding optical operator rather than around a modality-agnostic image prior alone.

3. Unfolded optimization architectures

In phase retrieval, one reported starting point is the loss

$F\in\mathbb{C}^{n\times n}$0

optimized by a first-order method such as gradient descent or projected gradient descent. A preconditioned gradient step is written as

$F\in\mathbb{C}^{n\times n}$1

with $F\in\mathbb{C}^{n\times n}$2 and $F\in\mathbb{C}^{n\times n}$3. Unrolling $F\in\mathbb{C}^{n\times n}$4 iterations yields a $F\in\mathbb{C}^{n\times n}$5-layer network in which each layer learns the step-size or preconditioner together with a correction network:

$F\in\mathbb{C}^{n\times n}$6

The gradient is

$F\in\mathbb{C}^{n\times n}$7

The reported architecture separates a forward-physics encoder from iterative decoder layers. The encoder models $F\in\mathbb{C}^{n\times n}$8, while decoder layers implement projected or proximal gradient updates, optionally augmented with learned residual branches. Custom nonlinearities include modulus/phase extraction, shrinkage or truncated gradient, and a proximal sparsity projector selecting top-$F\in\mathbb{C}^{n\times n}$9 coefficients (Pinilla et al., 2022).

In DSSI, DDU unfolds $$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$0 stages of ADMM. Each stage contains a fidelity update, a prior update, and a multiplier update. The fidelity subproblem is solved analytically in the frequency domain:

$$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$1

The prior update is a denoising operation,

$$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$2

with $$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$3 and $$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$4, and the multiplier update is

$$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$5

Here, $$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$6 are learned stage-wise scalars. Because $$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$7 is block-diagonal, each block can be inverted analytically or recursively, yielding an $$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$8 solver per iteration rather than $$D(\phi)=\mathrm{diag}(e^{j\phi_1},\ldots,e^{j\phi_n})$$9 (Zhuge et al., 7 Jul 2025).

Taken together, these formulations show that DDU is compatible with distinct optimization backbones. One instantiation unfolds first-order phase-retrieval updates with learned correction terms; another unfolds ADMM with an exact data-fidelity solver and learned denoisers. A common misconception is therefore to equate DDU with a single update rule. The reported literature instead uses the term for a broader physics-guided unrolling strategy.

4. Learned components and parameter optimization

In the phase-retrieval formulation, the learned parameters are collected as

$\phi\in\mathbb{R}^n$0

and the DOE phase $\phi\in\mathbb{R}^n$1 is optimized jointly with the network by minimizing

$\phi\in\mathbb{R}^n$2

The gradients with respect to $\phi\in\mathbb{R}^n$3 are computed by unrolling all $\phi\in\mathbb{R}^n$4 layers and backpropagating through both the decoder updates and the forward model $\phi\in\mathbb{R}^n$5. Since $\phi\in\mathbb{R}^n$6, the derivative $\phi\in\mathbb{R}^n$7 follows by the chain rule through the complex-valued multiplications and the modulus operation (Pinilla et al., 2022).

In the DSSI formulation, the most distinctive learned component is the initializer. Because reconstruction is described as highly ill-posed, naive initialization by zero, random vectors, $\phi\in\mathbb{R}^n$8, or a single linear layer leads to oscillatory ADMM behavior. DDU therefore introduces a small deep network $\phi\in\mathbb{R}^n$9 to map the RGB measurement to an initial hyperspectral estimate $\eta$0. The reported implementation uses a $\eta$1 convolutional stem, an embedding dimension $\eta$2—for example $\eta$3 for MST++—a cascade of identical blocks such as Transformer blocks in MST++ or RRG blocks in MIRNetV2, and a final $\eta$4 convolution producing $\eta$5. The stage-wise denoisers $\eta$6 have the same block structure as $\eta$7 but independent weights per stage, and both the initial estimate and the final reconstruction are supervised during training (Zhuge et al., 7 Jul 2025).

This division of labor between analytic and learned modules is central to DDU. The learned parts do not replace the optical model; they regularize the iterative solver, improve conditioning, and stabilize the dynamics. In the phase-retrieval setting, that role is served by learned preconditioning and residual correction branches, together with optional proximal constraints. In the DSSI setting, it is served by the learned initializer, stage-wise denoisers, and learned ADMM hyperparameters.

5. Reported empirical behavior

The reported experimental settings span synthetic Gaussian signals, real complex images such as dynamic protozoa video, hyperspectral scenes, 3-D phase objects, and single-cell eukaryote videos with 287 frames in the phase-retrieval literature, as well as ICVL, VNIRSR, and ColorChecker in DSSI (Pinilla et al., 2022, Zhuge et al., 7 Jul 2025).

For phase retrieval, the quantitative comparison at $\eta$8 is summarized below.

Method	Reported setting	Reported result
Reshaped WF (model)	$\eta$9	Success Rate $A_k=D(\phi)\cdot P_k\cdot F$0, Rel. Error $A_k=D(\phi)\cdot P_k\cdot F$1, Recon. SNR $A_k=D(\phi)\cdot P_k\cdot F$2 dB
Truncated WF (sparse)	$A_k=D(\phi)\cdot P_k\cdot F$3	Success Rate $A_k=D(\phi)\cdot P_k\cdot F$4, Rel. Error $A_k=D(\phi)\cdot P_k\cdot F$5, Recon. SNR $A_k=D(\phi)\cdot P_k\cdot F$6 dB
Unfolded (A fixed)	$A_k=D(\phi)\cdot P_k\cdot F$7	Success Rate $A_k=D(\phi)\cdot P_k\cdot F$8, Rel. Error $A_k=D(\phi)\cdot P_k\cdot F$9, Recon. SNR $k\in\{1,2,3\}$0 dB
Unfolded (A+G learned)	$k\in\{1,2,3\}$1	Success Rate $k\in\{1,2,3\}$2, Rel. Error $k\in\{1,2,3\}$3, Recon. SNR $k\in\{1,2,3\}$4 dB

In real-data experiments in the near zone with a single snapshot, DDU is reported to achieve more than $k\in\{1,2,3\}$5 detection rate for 3-D object detection and to reconstruct phase with SNR approximately $k\in\{1,2,3\}$6 dB. The same source states that classical methods such as Reshaped Wirtinger Flow and Truncated Amplitude Flow require hundreds of iterations and fail at $k\in\{1,2,3\}$7, whereas DDU reaches fast convergence at approximately $k\in\{1,2,3\}$8 layers rather than more than $k\in\{1,2,3\}$9 iterations (Pinilla et al., 2022).

For DSSI, the reported 5-stage and 7-stage results are as follows.

Method	Parameters / FLOPs	Reported metrics
MST++ 5stg	$$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$0 M, $$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$1 GFLOPS	PSNR $$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$2 dB, SAM $$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$3, SSIM $$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$4
CSST 5stg	$$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$5 M, $$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$6 GF	PSNR $$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$7, SAM $$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$8
Jeon 9stg	$$J[x,y,c] = \sum_{i=1}^{N_\lambda} I[x,y,i] \otimes p[x,y,i]\cdot \Omega[c,i] + \eta,$$9 M, $I[x,y,\lambda]$0 GF	PSNR $I[x,y,\lambda]$1
DDU–MST++ 5stg	$I[x,y,\lambda]$2 M, $I[x,y,\lambda]$3 GF	PSNR $I[x,y,\lambda]$4, SAM $I[x,y,\lambda]$5, SSIM $I[x,y,\lambda]$6
DDU–MST++ 7stg	$I[x,y,\lambda]$7 M, $I[x,y,\lambda]$8 GFLOPS	PSNR $I[x,y,\lambda]$9, SAM $x\in\mathbb{C}^n$00, SSIM $x\in\mathbb{C}^n$01

The same study reports that on real-world ColorChecker data, DDU–MST++ 7stg achieves the lowest average SAM and the best recovered modulation transfer function, up to approximately $x\in\mathbb{C}^n$02 cycles per pixel. The ablations attribute a gain of $x\in\mathbb{C}^n$03 dB PSNR to the learned initializer relative to linear initialization, $x\in\mathbb{C}^n$04 dB PSNR to ADMM relative to HQS without a multiplier, and a loss of $x\in\mathbb{C}^n$05 dB PSNR when supervising only $x\in\mathbb{C}^n$06 rather than $x\in\mathbb{C}^n$07 (Zhuge et al., 7 Jul 2025).

These results are consistent with the central reported claim of DDU: performance gains arise not from discarding the forward model, but from learning the optimizer around that model. In both domains, the comparisons are framed against two alternatives—classical iterative reconstruction and pure deep reconstruction—and DDU is described as occupying the intermediate regime of model-aware learning.

6. Guarantees, limitations, and areas of use

In the phase-retrieval setting, a reported uniqueness condition for DOE design is

$x\in\mathbb{C}^n$08

which guarantees unique recovery, up to global phase, for all 2-D signals regardless of diffraction zone. The same source states that maximizing the variance of coding elements across snapshots further improves reconstruction quality. The computational complexity per sample is reported as $x\in\mathbb{C}^n$09 multiplies plus the cost of proximal or nonlinear steps, with memory dominated by storing $x\in\mathbb{C}^n$10 copies of learned parameters and intermediate activations for backpropagation. Empirically, convergence to high-accuracy solutions is reported within $x\in\mathbb{C}^n$11–$x\in\mathbb{C}^n$12 layers, and learned preconditioning $x\in\mathbb{C}^n$13 is said to accelerate descent and handle ill-conditioned $x\in\mathbb{C}^n$14 (Pinilla et al., 2022).

In DSSI, the principal efficiency claim derives from the closed-form data-fidelity solver, which exploits the block structure of the Fourier-domain operator. The reported implementation is only $x\in\mathbb{C}^n$15 slower than pure MST++ and much faster than GDM-based DSSI unfolding. At the same time, the limitations are explicit: accurate PSF calibration is required, memory and compute grow linearly with the number of stages, and residual errors persist in ultra-high-frequency spatial detail such as fine text (Zhuge et al., 7 Jul 2025).

Several limitations recur across the two formulations. Training cost is nontrivial because the methods require paired data and end-to-end backpropagation through complex or frequency-domain operations. Model mismatch is a recurrent concern: in phase retrieval, fabricated DOE deviations may necessitate calibration layers or adaptive corrections; in DSSI, inaccurate PSF calibration can degrade performance. Theoretical guarantees are also incomplete. The phase-retrieval paper states that unrolled networks inherit local convergence of first-order methods, but that global optimality in the learned setting remains open (Pinilla et al., 2022).

The reported application areas include compact lensless microscopes, real-time 3-D imaging, robust optical metrology across near-, middle-, and far-field regimes, 3-D object detection in single-shot near-zone experiments, hyperspectral reconstruction, and real-data spectral imaging benchmarks. A plausible implication is that DDU is particularly well matched to inverse problems in which the optics are structured enough to admit an explicit forward model, yet ill-posed enough that purely analytic reconstruction remains fragile without learned priors or learned optimizer parameters.