Data-Driven Holographic Reconstruction

• Data-driven holographic reconstruction is a set of inverse methods that recover complex 3D fields from intensity-only holograms using optical models and learned priors.
• Key algorithmic paradigms include supervised regression, self-supervised physics-based training, and optimization-based inversion to address twin-image artifacts and underdetermined measurements.
• Practical applications span particle tracking and high-throughput microscopy, offering significant speedups and enhanced accuracy over traditional diffraction-based methods.

Searching arXiv for recent and foundational papers on data-driven holographic reconstruction to ground the article. Data-driven holographic reconstruction denotes a class of inverse methods that recover complex fields, particle parameters, or volumetric morphology from holograms by combining optical forward models with learned mappings, learned priors, or data-conditioned optimization. In optical holography, these methods are motivated by the ill-posedness of intensity-only measurements, especially in in-line geometry, where twin-image artifacts and axial ambiguity complicate conventional diffraction-based replay. The literature includes direct supervised regression from a raw hologram to task-specific outputs, self-supervised or physics-based training that enforces propagation consistency, regularized inverse solvers informed by measured data, and compact parametric representations that reduce the number of unknowns (Shimobaba et al., 2018, Birdi et al., 2020, Huang et al., 2022, Brault et al., 2 Jul 2026).

1. Physical formulation and inverse structure

In digital holography, the recorded signal is the intensity of an interference pattern produced by a reference field and an object-scattered field. For in-line particle holography, the measurement model used in a representative reconstruction study is

$$I(x,y)=\left|R(x,y)+\sum_{j=1}^{P}u_j(x,y)\right|^2,$$

with each particle contribution $u_j$ determined by its lateral position, axial position, size, and the wavelength-dependent diffraction kernel (Shimobaba et al., 2018). In more general in-line settings, the object-plane field can be written as $u_0=1+o$, propagated to the sensor by a free-space operator $P_z$, with the sensor recording $I(x,y;z)=|P_z\{u_0\}|^2$ (Zhang et al., 25 Sep 2025).

The forward operator is linear in the complex field under weak or single-scattering assumptions, but the measurement is intensity-only. This produces the central nonlinearity of holographic inversion: phase is not directly recorded, and the unknown object field has more degrees of freedom than a single hologram provides. For a single $M\times N$ complex object plane, there are $2MN$ real unknowns, whereas one hologram provides $MN$ real measurements; in in-line holography this underdetermination gives rise to conjugate ambiguities and the twin-image artifact (Zhang et al., 25 Sep 2025).

A central theoretical result for three-dimensional replay is that numerical back-propagation is the Hermitian transpose of the hologram formation operator, not its inverse. In the notation of the corresponding reconstruction study, replay is

$x_{\mathrm{replay}}=\hat{A}^{\dagger}y,$

so it behaves as a matched filter or backprojection rather than an exact volumetric inverse (Birdi et al., 2020). This is why standard back-propagation produces focus at the correct depth but also residual defocus and nonzero values in planes where no object exists. The distinction between adjoint replay and inverse reconstruction is foundational for data-driven methods: once replay is recognized as an approximation, learned priors and regularized inverse solvers become natural mechanisms for selecting a physically plausible solution.

2. Algorithmic paradigms

One major paradigm is direct supervised regression. In particle holography, a U-Net-style encoder-decoder can take a raw $1{,}024\times1{,}024$ in-line hologram and output three maps: a binary lateral-position map, an axial-position map, and a size map, thereby bypassing repeated diffraction propagation and focus-metric search (Shimobaba et al., 2018). In off-axis digital holographic microscopy, OAH-Net embeds differentiable Fourier-domain sideband separation and background demodulation inside a deep network, then learns amplitude recovery and phase unwrapping end to end; its Fourier Imager Heads are initialized from the physical sideband geometry and fine-tuned with pseudo-ground-truth reconstructions (Liu et al., 2024).

A second paradigm is self-supervised or physics-based deep learning. GedankenNet eliminates labeled and experimental training data by training on synthetic random images and enforcing a physics-consistency loss between measured hologram amplitudes and the amplitudes generated by propagating the network output to the measurement planes (Huang et al., 2022). A later single-shot in-line microscopy method uses phase diversity only during training, not inference: the network is trained so that a reconstruction from one back-propagated hologram must reproduce holograms at both $u_j$0 and $u_j$1, allowing single-shot inference of quantitative transmission afterward (Brault et al., 2 Jul 2026). MorpHoloNet extends this principle from 2D complex-field recovery to single-shot 3D morphology reconstruction by coupling a coordinate-based neural field to a multi-slice angular-spectrum propagation model and optimizing directly against the measured hologram intensity (Kim et al., 2024).

A third paradigm is optimization-based data-driven inversion. In “true 3D reconstruction,” regularized optimization over a volumetric object function uses the measured field and a physics operator $u_j$2, solved with FISTA and either $u_j$3 or total-variation regularization, to drive non-object voxels toward zero (Birdi et al., 2020). Deep DIH replaces explicit handcrafted regularization with the implicit bias of an untrained autoencoder optimized on a single hologram through a forward-model loss (Li et al., 2020). DH-GAN combines an untrained generator, a discriminator operating on holograms, explicit angular-spectrum propagation, and a progressive masking module that smooths reconstructed background regions (Chen et al., 2022). Gaussian Splatting Holography instead reduces dimensionality by parameterizing the object-plane field with a small set of smooth Gaussian atoms, turning phase retrieval into an optimization over a compressed representation (Zhang et al., 25 Sep 2025).

3. Reconstructed quantities and problem formulations

The outputs of data-driven holographic reconstruction vary substantially across tasks. Some methods recover a complex field at one plane; others infer physically meaningful latent variables directly. Particle-focused systems estimate lateral position, axial position, and particle diameter from a single hologram, while microscopy-oriented methods recover amplitude and quantitative phase, and volumetric solvers seek a true 3D object function or refractive-index distribution (Shimobaba et al., 2018, Brault et al., 2 Jul 2026, Kim et al., 2024).

In single-shot particle reconstruction, the output is not a reconstructed z-stack but three dense prediction maps from which particle coordinates and sizes are extracted by thresholding and regional statistics. Labels are encoded as $u_j$4 pixel squares rather than single pixels to improve convergence, and the loss combines $u_j$5 and $u_j$6 terms,

$u_j$7

to reduce false positives in the background (Shimobaba et al., 2018).

In self-supervised in-line microscopy, the object is a complex transmission function $u_j$8. The network is trained so that its prediction, when passed through the calibrated angular-spectrum operator and numerical aperture pupil, reproduces holograms at one or more sensor planes. This enables quantitative phase and amplitude estimation from a single hologram at test time, even though phase diversity was needed during training (Brault et al., 2 Jul 2026).

For true volumetric inversion, the unknown is a 3D complex-valued object $u_j$9, and the reconstruction target is a volume whose voxels are near zero wherever no object exists. This differs conceptually from replay-based sectioning: the optimization does not merely sharpen one slice at a time, but solves a regularized inverse problem constrained by the full forward operator (Birdi et al., 2020). MorpHoloNet further specializes the volumetric target to a 3D phase-shift distribution $u_0=1+o$0, from which a refractive-index map is recovered as

$u_0=1+o$1

This formulation is intended for biological cells with negligible absorption, where morphology and refractive index are coupled through a phase-only multi-slice model (Kim et al., 2024).

4. Reported accuracy and computational characteristics

The literature reports speedups over classical diffraction-based pipelines, but the gains are task-dependent and closely tied to the output definition. Direct particle regression reduces a 256-depth propagation stack to a single network pass; off-axis microscopy networks reduce Fourier-domain filtering and phase unwrapping to millisecond-scale inference; by contrast, coordinate-based volumetric fitting can require per-hologram optimization times measured in minutes (Shimobaba et al., 2018, Liu et al., 2024, Kim et al., 2024).

Approach	Reported reconstruction result	Reported runtime
Particle DNN (Shimobaba et al., 2018)	lateral errors $u_0=1+o$2, axial errors $u_0=1+o$3, size errors $u_0=1+o$4 in synthetic tests	$u_0=1+o$5 per hologram; $u_0=1+o$6 faster than 256-depth angular-spectrum stack
Self-supervised DIHM (Brault et al., 2 Jul 2026)	experimental phase RMSE/RMSE-O: beads $u_0=1+o$7 mrad, E. coli $u_0=1+o$8 mrad, M. luteus $u_0=1+o$9 mrad	$P_z$0 unbatched; $P_z$1 with 16-image batch; $P_z$2 speedup over iterative IPA-2 and GS
OAH-Net (Liu et al., 2024)	phase MAE $P_z$3 whole image, $P_z$4 cell region; amplitude MAE $P_z$5 whole image	$P_z$6
MorpHoloNet (Kim et al., 2024)	synthetic phase MAE $P_z$7 rad; ellipsoid inclination RMSE $P_z$8	$P_z$9 minutes per hologram

These numerical results are heterogeneous because the tasks are heterogeneous. The particle network predicts physically meaningful particle descriptors directly from the raw hologram; the self-supervised DIHM model reconstructs a quantitative transmission function; OAH-Net reconstructs off-axis phase and amplitude images; MorpHoloNet performs per-instance 3D fitting. A direct comparison of their error values is therefore not meaningful. What is meaningful is that data-driven holographic reconstruction spans a continuum from ultra-fast task-specific inference to slower but richer per-instance optimization.

Other reported performance figures reinforce this spread. DH-GAN reports about $I(x,y;z)=|P_z\{u_0\}|^2$0 dB PSNR gain over competitor methods and about $I(x,y;z)=|P_z\{u_0\}|^2$1 reduction in PSNR versus noise increase rate, with an additional $I(x,y;z)=|P_z\{u_0\}|^2$2 dB from progressive masking (Chen et al., 2022). Gaussian Splatting Holography reports average PSNR equal to $I(x,y;z)=|P_z\{u_0\}|^2$3 dB and SSIM equal to $I(x,y;z)=|P_z\{u_0\}|^2$4, rising to a peak signal-to-noise ratio of $I(x,y;z)=|P_z\{u_0\}|^2$5 dB when combined with total variation, with compression up to $I(x,y;z)=|P_z\{u_0\}|^2$6 folds (Zhang et al., 25 Sep 2025).

5. Assumptions, limitations, and recurrent misconceptions

A recurring limitation is dependence on weak-scattering or single-scattering models. Particle reconstruction by direct regression assumes linear superposition of individual particle fields and neglects inter-particle multiple scattering (Shimobaba et al., 2018). True 3D optimization assumes weak scattering in its linear operator $I(x,y;z)=|P_z\{u_0\}|^2$7 (Birdi et al., 2020). The self-supervised single-shot DIHM model assumes a thin object, coherent plane-wave illumination, and a calibrated angular-spectrum operator with numerical-aperture pupil (Brault et al., 2 Jul 2026). MorpHoloNet assumes pure-phase biological samples with negligible absorption and scalar paraxial propagation (Kim et al., 2024). These assumptions are not incidental; they define the inverse problem each method is actually solving.

A second limitation is training-distribution dependence. The particle DNN is trained on synthetic holograms defined by a specific wavelength, axial range, particle-size range, density range, and sensor pitch; its generalization to unseen ranges, refractive indices, or real experimental data is explicitly left for validation (Shimobaba et al., 2018). OAH-Net inherits the biases of its pseudo-ground-truth phase and amplitude reconstructions because supervision comes from an external API rather than from direct physical truth (Liu et al., 2024). GedankenNet is designed precisely to weaken such dependence by training on artificial random images, yet even there the forward model must remain reasonably faithful to the real optics (Huang et al., 2022).

A third issue concerns physical interpretation. One common misconception is that replay or back-propagation produces a “true” 3D reconstruction. The formal analysis of digital holography shows that replay is the adjoint of formation, not an inverse, so it cannot selectively null non-object voxels plane by plane (Birdi et al., 2020). Another misconception is that supervised regression below the nominal axial-resolution bound changes the physical numerical aperture. In particle holography, axial errors around $I(x,y;z)=|P_z\{u_0\}|^2$8–$I(x,y;z)=|P_z\{u_0\}|^2$9 mm were reported to be smaller than the conventional $M\times N$0 mm of the simulated system, but the same study explicitly notes that this reflects supervised regression to simulated ground truth rather than a change in the system NA (Shimobaba et al., 2018).

The field also contains a structural trade-off between data fidelity and prior strength. End-to-end networks can remove twin-image artifacts and accelerate inference, but they may be less transparent than explicit propagation pipelines. Compact parametric methods such as Gaussian splatting reduce phase ambiguities by reducing the number of unknowns, but they can underfit high-frequency structure when too few atoms are used and can become unstable when too many are introduced (Zhang et al., 25 Sep 2025). Untrained and self-supervised methods improve transferability and reduce dataset requirements, but they usually pay for that flexibility with per-instance optimization time or heavier dependence on calibration.

6. Applications and emerging directions

Data-driven holographic reconstruction has already been applied to particle volume analysis, label-free microscopy, high-throughput cellular imaging, and dynamic biological tracking. Direct particle regression targets lateral and axial localization and size estimation from a single hologram (Shimobaba et al., 2018). Self-supervised DIHM reconstructs quantitative transmission for beads and bacteria from one hologram at inference, and reports experimental datasets for beads, E. coli, and M. luteus (Brault et al., 2 Jul 2026). MorpHoloNet extends single-shot reconstruction to 3D morphology, refractive-index mapping, and time-resolved cell dynamics, including red blood cells in viscoelastic flow and free-swimming E. coli (Kim et al., 2024). OAH-Net is explicitly positioned for high-throughput off-axis digital holographic microscopy and downstream real-time analysis (Liu et al., 2024).

Several emerging directions are visible. One is stronger but still generalizable priors. An amplitude-only diffusion prior has been shown to recover both amplitude and phase from diffraction intensities without ground-truth phase data for training, and the reported experiments include zero-shot transfer from simple amplitude data such as polystyrene beads to complex biological tissue and lensless configurations (Kim et al., 16 Sep 2025). Another is compact scene parameterization: Gaussian Splatting Holography replaces a pixelwise field with anisotropic Gaussian atoms, compressing the unknowns and suppressing twin-image backgrounds through representation choice rather than explicit support or positivity constraints (Zhang et al., 25 Sep 2025). A third is hardware-level learning. Diffractive networks demonstrate computer-free, all-optical reconstruction of in-line holograms with passive phase-only transmissive layers trained to suppress twin images at the speed of light propagation, thereby relocating the learned inverse map from digital post-processing into the optical path itself (Rahman et al., 2021).

Future work identified within the literature includes multi-wavelength and polarization extensions, multi-angle or coded-illumination variants, temporal priors and warm starts for dynamic sequences, joint system calibration, and volumetric generalizations of compact primitives such as 3D Gaussian splats (Kim et al., 2024, Kim et al., 16 Sep 2025, Zhang et al., 25 Sep 2025). This suggests that the field is moving toward hybrid systems in which wave-optics operators, learned priors, compact parameterizations, and hardware-aware calibration are not alternatives but interoperable components of the same reconstruction pipeline.