Physics-based self-supervised learning of a deep network for single-shot in-line hologram reconstruction

Abstract: Digital in-line holographic microscopy is a computational imaging method useful for characterizing the refractive properties of a sample, i.e. the phase shift and absorption. This indirect measurement technique captures a diffraction pattern and uses reconstruction algorithms to retrieve the optical properties of the sample. Since only the intensity of the diffracted wave is recorded on the sensor, this inversion is not trivial, and simple backward propagation leads to artifacts known in optics as the ``twin-image''. With advances in deep learning, various algorithms have been developed for the reconstruction of in-line holograms, providing computationally efficient alternatives to iterative algorithms. These algorithms rely either on supervised learning, which requires ground truth knowledge, or physics-based self-supervised algorithms that require additional information, like phase diversity, but require multiple holograms for inference. This paper introduces a new self-supervised physics-based deep learning strategy that leverages phase diversity during training and then reconstructs sample's transmission function from a single in-line hologram during inference. We introduce five datasets of simulated and experimental in-line holograms of beads and bacteria. The proposed method produces accurate quantitative reconstructions similar or even more accurate than those obtained by regularized inversion while reducing the computational time by a factor of 1000.

• The paper introduces a self-supervised deep learning framework that integrates physics constraints to reconstruct holograms from a single shot.
• It leverages phase diversity during training and uses a U-Net with residual connections to suppress twin-image artifacts and achieve real-time performance.
• Quantitative experiments on simulated and experimental data demonstrate state-of-the-art accuracy and an order-of-magnitude speedup compared to iterative methods.

Self-Supervised Physics-Based Deep Learning for Fast Single-Shot In-Line Hologram Reconstruction

Introduction

Digital in-line holographic microscopy (DIHM) offers powerful, label-free optical characterization of transparent specimens, retrieving both phase and amplitude information encoded in a diffraction pattern. The principal computational bottleneck in DIHM arises from the ill-posed phase retrieval problem: detectors only record intensity, resulting in the notorious twin-image artifact upon direct inversion. Traditional iterative phase retrieval methods, while physically sound (e.g., Gerchberg-Saxton, regularized inverse problem approaches), are computationally slow and often experimentally restrictive, especially where multiple holograms or strong object priors are unavailable.

Recent deep learning methods achieve efficient hologram reconstruction but usually depend on supervised training with ground-truth data, which is impractical for most biological or complex real-world samples. Existing self-supervised approaches require multiple holograms per inference or compromise reconstruction fidelity. This paper introduces a novel, self-supervised, physics-constrained deep learning framework, leveraging phase diversity only during training and reconstructing the sample's transmission function from a single-shot hologram at inference, thereby substantially accelerating quantitative holographic imaging and uniquely democratizing DIHM for in-the-wild applications (2607.01922).

Methodology

The proposed approach rests on a unification of rigorous optical modeling and modern self-supervised learning. The method uses pairs of out-of-focus holograms (phase diversity) during training, while at test time it only requires a back-propagated single-shot in-line hologram. The network, based on a U-Net backbone with residual connections, is optimized using a loss combining fidelity to image formation physics and regularization (which is annealed and discarded halfway through training). The architecture is designed to unmix the true object image from its twin and zero-order artifacts directly in the object plane.

The full framework is schematized in Figure 1.

Figure 1: Framework of the proposed self-supervised approach, leveraging dual hologram phase diversity for training but requiring only a single hologram at inference to reconstruct the sample transmission function.

Physics-Based Training Objective

Physics is incorporated in the loss via an explicit propagation model, enforcing that the network output, when forward-propagated, produces intensities matching both training holograms. This double-fidelity constraint, coupled with regularization, prevents convergence to non-physical minima and ensures robust twin-image suppression.

After training, inference reduces to a single feed-forward pass after back-propagation preprocessing, yielding order-of-magnitude computational acceleration compared to iterative solvers.

Experimental Setup

To validate the method experimentally and numerically, five datasets were created and released—two simulated (beads, bacteria) and three acquired using a silicon-oil immersion DIHM system (beads, Escherichia coli (Gram-), Micrococcus luteus (Gram+)). The experimental setup—incorporating laser illumination, precise piezo translation, and telecentric imaging—is detailed in Figure 2.

Figure 2: Schematic of the in-line holography experimental system, including laser illumination, optics, and sample details. Sample z-position is varied for phase diversity in training.

Results

Quantitative and Qualitative Comparison

Reconstruction was assessed on both simulated and real experimental data using multiple quantitative metrics—root mean square error (RMSE) and object-only RMSE (RMSE-O)—for both phase and modulus of the transmission function. The proposed self-supervised method was benchmarked against classical back-propagation, Gerchberg-Saxton (GS), and regularized inverse problem approaches (IPA-1: single hologram, IPA-2: dual hologram), as well as supervised deep networks.

Key findings include:

• Accuracy: On simulated datasets (SIM_BEADS, SIM_BACT), the proposed method achieved phase RMSE and RMSE-O of the same order as supervised deep networks, significantly outperforming both GS and IPA-1, and in challenging scenarios, even exceeding IPA-2, despite using only a single hologram at inference.
• Speed: The deep learning model reduced average inference time by $\mathcal{O}(10^3)$ versus IPA methods (milliseconds per image vs seconds).
• Generalization: When trained and tested on object distributions with mismatched morphology, performance degraded—attributable to implicit learned priors—motivating custom training for each specimen category.

On experimental datasets (beads, E. coli, M. luteus), the method closely approximated the IPA-2 baseline reconstruction, and consistently outperformed GS, back-propagation, and IPA-1 methods on quantitative error metrics and artifact suppression. This is especially evident in reconstructions shown in Figure 3, where the twin-image artifact is robustly eliminated.

Figure 3: Visual comparison of phase reconstructions on real experimental data. The proposed method considerably outperforms classical and existing deep learning approaches in artifact removal and visual fidelity.

Dataset Contribution

All datasets, including challenging real-world bacteria and bead samples, together with code, were made publicly available to encourage reproducibility and further benchmarking within the community.

Implications and Discussion

This work demonstrates, for the first time, highly accurate quantitative phase reconstruction from a single-shot in-line hologram achieved purely through self-supervised physics-constrained deep learning with no reliance on paired ground-truth data at inference. The strong empirical results across simulated and experimental datasets indicate that such approaches can bridge the current practical gap in DIHM between computational efficiency and quantitative accuracy.

Beyond DIHM, the strategy suggests a generalizable template for self-supervised, physics-embedded training in inverse imaging problems—where direct supervision is impractical and traditional iterative optimization is computationally prohibitive.

Key practical implications include:

• Real-time operation: Millisecond-level inference on commodity hardware enables live, quantitative imaging for biological or clinical applications.
• Accessibility: By eliminating the requirement for expensive or custom ground-truth datasets, the method lowers the technical and economic barrier for deploying high-quality DIHM reconstructions.
• Extension to other modalities: The approach may generalize to other phase retrieval problems (e.g., X-ray, terahertz, or electron microscopy) amenable to similar propagation models.

Limitations include the need for phase diversity data in the training phase and some loss of generalization when applying the network to object distributions differing markedly from those seen during training.

Conclusion

This paper establishes a robust, self-supervised, physics-based deep learning paradigm for single-shot in-line hologram reconstruction, achieving state-of-the-art accuracy with unprecedented computational speed. By integrating dual-hologram phase diversity in training, the method overcomes the fundamental limitations of prior approaches without requiring ground-truth annotations or multiple holograms at inference. This framework paves the way for scalable, fast, and artifact-free quantitative phase imaging across diverse applications in microscopy and beyond.