Ptychographic Implicit Neural Representation (PtyINR)

• PtyINR is a computational imaging framework that uses continuous neural fields to replace discrete pixel grids in ptychographic reconstruction.
• The method integrates a physics-based forward model with self-supervised optimization to enforce measurement consistency and recover both object and probe.
• Demonstrating improved robustness and efficiency in X-ray and Fourier ptychography, PtyINR reduces memory usage and accelerates reconstruction speed.

Searching arXiv for the cited papers and closely related work on implicit neural representations in ptychography and Fourier ptychographic microscopy. Ptychographic Implicit Neural Representation (PtyINR) denotes a class of self-supervised, physics-informed reconstruction methods in which a ptychographic inverse problem is solved by optimizing continuous neural fields under measurement-consistency constraints rather than reconstructing a discrete pixel or voxel grid. In the terminology used for X-ray ptychography, PtyINR is a framework that jointly recovers both the object and an unknown probe directly from measured diffraction patterns by parameterizing both as continuous neural representations (Li et al., 4 Sep 2025). In Fourier ptychographic microscopy (FPM), the closely related FPM-INR framework applies the same core idea to volumetric image-stack reconstruction: it retains the FPM forward model, but replaces explicit voxel-wise reconstruction of a discrete $x\times y\times z$ stack with a learned implicit function that maps coordinates in a low-dimensional feature space to the complex optical field (Zhou et al., 2023). Taken together, these works establish PtyINR as a ptychographic methodology centered on continuous coordinate-based fields, end-to-end differentiable optics, and self-supervised optimization.

1. Concept and scope

PtyINR is defined by the conjunction of three elements: a ptychographic forward model, an implicit neural representation, and self-supervised optimization through measurement consistency. In X-ray ptychography, the method is introduced as a self-supervised framework for joint object and unknown-probe recovery, with both quantities represented as continuous coordinate-based neural functions (Li et al., 4 Sep 2025). In FPM, the same conceptual structure appears in a volumetric setting, where the object is represented by a compact continuous neural field supervised only by the optical forward model and the measured raw intensities (Zhou et al., 2023).

This places PtyINR between two established reconstruction paradigms. On one side are conventional iterative solvers, which typically use discrete pixel-wise or voxel-wise variables; on the other are supervised deep models that learn a direct inverse mapping from data to reconstruction. PtyINR differs from both. It is not a pre-trained black-box predictor, because optimization is performed per sample using only the current measurements and the known physics. It is also not a conventional grid-based inverse solver, because the object—and in some formulations the probe as well—is stored as a continuous neural function rather than as a dense array (Li et al., 4 Sep 2025).

A plausible implication is that “PtyINR” is best understood not as a single architecture, but as a reconstruction principle for ptychographic problems in which coordinate-based neural parameterizations substitute for explicit image or volume tensors while the measurement operator remains physically exact or nearly so.

2. Inverse-problem formulation and forward models

In ptychography, the central difficulty is that only intensity is measured, while phase is missing. In X-ray ptychography with an unknown probe, the measured data depend multiplicatively on both object and probe, which creates a nonlinear blind phase-retrieval problem. The static forward model described for the $j$-th scan position is

$$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$

with a fly-scan extension given by

$$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$

(Li et al., 4 Sep 2025). The multiplicative coupling means that errors in one estimate contaminate the other, which is one reason unknown-probe recovery is difficult for standard methods.

In FPM, the corresponding measurement model is formulated in Fourier space under angled LED illumination: $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$ where each illumination angle reveals a different shifted patch of the object spectrum, and the pupil function includes the defocus-related quadratic phase term (Zhou et al., 2023). In the $z$-stack setting, this must be applied across many depths, which makes both computation and storage burdensome.

In both cases, the forward operator is not peripheral but embedded directly in training. The network outputs are propagated through the ptychographic imaging model, the predicted measurement is compared with the acquired data, and gradients are backpropagated through the full physical pipeline. This is the defining operational feature of PtyINR-style methods (Zhou et al., 2023, Li et al., 4 Sep 2025).

3. Neural parameterizations

The essential representational move in PtyINR is to replace explicit grids with continuous neural fields. For X-ray ptychography, the object and the probe are each parameterized as coordinate-dependent amplitude-phase functions: $$O(\mathbf{r}) = \mathcal{A}_O(\mathbf{r}) \exp\!\left(i\,\varphi_O(\mathbf{r})\right), \qquad P(\mathbf{r}) = \mathcal{A}_P(\mathbf{r}) \exp\!\left(i\,\varphi_P(\mathbf{r})\right)$$ (Li et al., 4 Sep 2025).

The object network is described as a SIREN-based INR with separate amplitude and phase MLPs, each having 3 hidden layers and 512 units per layer, using sinusoidal activations of the form

$\sin(\omega_i \mathbf{W}\mathbf{x} + \mathbf{b}),$

with $\omega_i$ a layer-specific frequency factor, typically around 30 (Li et al., 4 Sep 2025). The probe is represented differently: PtyINR uses a multiresolution hash encoding with a shallow ReLU MLP on top and separate MLPs for probe amplitude and phase. The encoded feature vector is

$$\mathbf{f}(\mathbf{r}) = \bigoplus_{\ell=1}^{N} \sum_{i=1}^{4} w_{\ell,i}\,\mathcal{H}_\ell(g_{\ell,i}),$$

where resolution levels, neighboring grid points, bilinear interpolation weights, and learned hash-table features are combined to form a multiscale coordinate representation (Li et al., 4 Sep 2025).

FPM-INR adopts a different parameterization because its target is a volumetric scene with strong lateral-axial asymmetry. Instead of storing every voxel explicitly, it uses a learnable feature volume $j$0 with $j$1 channels per location,

$j$2

but factorizes it as a 2D feature plane $j$3 over $j$4-$j$5 and a 1D feature vector $j$6 over $j$7, combined by Hadamard product: $j$8 The field is then decoded by two real-valued MLPs—one for amplitude and one for phase—each with two nonlinear layers with ReLU and a final linear layer, giving

$j$9

(Zhou et al., 2023).

These design choices are modality-specific, but their shared logic is clear: the neural representation is chosen to match the structure of the unknown. In X-ray PtyINR, the object and probe are both unknown and are given separate continuous models. In FPM-INR, axial resolution is weaker than lateral resolution, so a low-rank decomposed neural field is used to exploit anisotropy (Zhou et al., 2023, Li et al., 4 Sep 2025).

4. Self-supervised optimization and physics-informed constraints

PtyINR methods are fully self-supervised in the sense that no ground-truth object, probe, or image stack is required during training. In X-ray ptychography, the loss compares measured diffraction patterns $$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$0 and predicted diffraction patterns $$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$1 using SmoothL1 on square-root intensities: $$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$2 A temporary early-stage probe-amplitude regularizer is added,

$$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$3

and the optimization target is

$$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$4

(Li et al., 4 Sep 2025). Practical stabilization also includes random initialization for blind recovery and probe normalization,

$$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$5

In FPM-INR, the reconstruction objective minimizes a smooth L1 loss between measured and estimated intensity images generated by the FPM forward model. The predicted measurement is obtained by decoding the field from the INR, propagating it through the optical model, and applying inverse Fourier transform and intensity formation. Optimization updates both the MLP weights and the parameters of the feature volume, again without external supervision (Zhou et al., 2023).

The significance of the physics-informed constraint is emphasized in both formulations. The networks are not trained to imitate reconstructed images directly; they are required to produce fields that explain the measured data through the imaging physics. This is presented as the key reason the methods avoid the hallucination-prone behavior associated with generic supervised deep learning and remain applicable without a training corpus of labeled examples (Zhou et al., 2023, Li et al., 4 Sep 2025).

5. Fourier ptychographic microscopy as a volumetric PtyINR

FPM-INR is explicitly described as a ptychographic implicit neural representation method for reconstructing FPM image stacks in a compact, continuous, and physics-informed way (Zhou et al., 2023). Its motivating problem is specific: standard FPM methods reconstruct each depth slice independently, a high-resolution stack can contain hundreds of slices and gigapixel-scale lateral dimensions, reconstruction takes tens of minutes on a GPU, and storing the final stack requires gigabytes of memory. This is especially consequential in digital pathology, where wide-field, high-resolution, digitally refocused views of thick or tilted samples are desired without physical $$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$6-stack scanning (Zhou et al., 2023).

The method exploits a structural asymmetry in FPM volumes: laterally, detail is high; axially, effective resolution is lower and the physical content often varies more slowly. This motivates the low-rank decomposed representation $$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$7, which is a tensor-factorized neural field built from a 2D spatial factor and a 1D axial factor (Zhou et al., 2023). Both factors are discretized but sampled continuously using bilinear interpolation in the $$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$8-$$\psi_j(\mathbf{r}) = P(\mathbf{r})\, O(\mathbf{r} + \mathbf{r}_j), \qquad I_j(\mathbf{q}) = \left| \mathcal{F}\{\psi_j(\mathbf{r})\} \right|^2,$$9 plane and linear interpolation along $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$0, so the model supports arbitrary continuous coordinates rather than only a fixed set of slices.

The training procedure includes a $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$1-slice selection strategy: at odd epochs, $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$2-planes are selected uniformly from discretized positions; at even epochs, $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$3-planes are sampled randomly and then interpolated. After optimization, the learned feature volume and MLP weights serve as a compact scene representation from which arbitrary depths can be queried on demand (Zhou et al., 2023).

The blood smear proof-of-concept uses a human blood smear slide, an Olympus PLN $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$4 NA objective, LED array plus LED ring illumination with 68 LEDs, wavelength 522 nm, an Allied Vision Prosilica GT 6400 camera with 3.45 µm pixel pitch, and a sample tilted 4° to the optical axis. The brightfield reference stack spans $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$5 to $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$6 µm in 0.25 µm steps for a total of 161 slices, with a $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$7 ROI (Zhou et al., 2023). A second dataset is a papillary thyroid carcinoma pap smear with thickness about 30 µm, imaged with an Olympus $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$8 NA objective, 145 illumination LEDs, 66 mm LED-sample distance, an ON Semi KAI-29050 camera with 5.5 µm pixel pitch, and an FPM baseline reconstructed 121 $$I_j(\mathbf{q}) = \int_{t_0}^{t_0 + \Delta t} \left| \mathcal{F} \left\{ P(\mathbf{r} - \mathbf{v}t)\, O(\mathbf{r} + \mathbf{r}_j) \right\} \right|^2 dt$$9-slices (Zhou et al., 2023).

Quantitatively, for the blood smear experiment, the all-in-focus image $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$0 error is reported as $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$1 for FPM-INR versus $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$2 for conventional FPM (Zhou et al., 2023). Storage is reduced from 2576 MB for the FPM stack to approximately 32 MB for the learned representation, comprising 32 MB of feature parameters and 8.4 KB of MLP weights, corresponding to an 80.5× reduction (Zhou et al., 2023). On an Nvidia RTX A6000 GPU, FPM-INR achieves blood-smear speedups of 9.8×, 11.8×, 7.5×, and 5.3× for $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$3, $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$4, $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$5, and $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$6 patch sizes, respectively, and thyroid pap smear speedups of 24.7×, 17.9×, 7.3×, and 5.0× for the same patch sizes; the abstract summarizes performance as up to a 25-fold increase in speed and an 80-fold reduction in memory usage (Zhou et al., 2023). Inference throughput is reported as approximately 460 MB/s on the RTX A6000.

These results support the interpretation of FPM-INR as a volumetric instance of the broader PtyINR concept: a ptychographic inverse problem solved by optimizing a neural field under explicit optical constraints, with continuous refocusing and substantial representation compression (Zhou et al., 2023).

6. Unknown-probe X-ray PtyINR

The 2025 X-ray ptychography formulation names the method directly as PtyINR and targets blind reconstruction, where both object and probe must be recovered from raw diffraction data (Li et al., 4 Sep 2025). This addresses a known failure mode of conventional methods: the object and probe are multiplicatively coupled, so optimization can drift to degenerate solutions, convergence depends heavily on initialization, and low overlap or low photon count weakens the constraints.

The paper situates PtyINR relative to several classes of baselines. Conventional iterative solvers include ePIE, DM, RAAR, WASP, APG, and AD/Adorym-style methods, while supervised deep models include PtychoNN, PtychoNet, and PtychoDV (Li et al., 4 Sep 2025). The critique is that iterative solvers are sensitive to overlap, noise, and initialization, whereas supervised models require labeled training data and often assume a known probe. Self-supervised AD-based approaches avoid labels but retain discrete pixel-wise variables.

Simulation studies compare PtyINR against ePIE, DM, WASP, PINN, AD/Adorym, RAAR, and APG under overlap ratios corresponding to approximately 40%, -540%, and -800% nominal settings. The reported findings are that, with known probe, PtyINR outperformed iterative and deep-learning baselines especially at low overlap; with unknown probe, it recovered both object and probe more accurately than ePIE and DM; under extreme low-overlap conditions, many classical methods degraded badly or failed while PtyINR still preserved meaningful structure; and PSNR comparisons showed PtyINR consistently achieved the highest scores for object amplitude/phase and probe amplitude/phase (Li et al., 4 Sep 2025). Under Poisson, Gaussian, and mixed noise, PtyINR produced cleaner reconstructions and beat ePIE and APG in PSNR across all categories, while AD, RAAR, and WASP were more sensitive to noise (Li et al., 4 Sep 2025).

Real experimental validation is reported on NSLS-II HXN beamline data using a LiCo$$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$7 battery particle with FZP probe and a Siemens star test pattern with MLL probe. PtyINR is said to give fewer periodic/grid artifacts, sharper edges, better phase fidelity, more faithful probe recovery, and better line profiles and local detail; as scan step size increased, conventional methods became more artifact-prone while PtyINR remained more stable (Li et al., 4 Sep 2025). Under low exposure, all methods performed similarly at 0.2 s, but at 0.003 s PtyINR significantly outperformed ePIE and APG, RAAR and AD struggled or failed on object amplitude under severe noise, and FRC analysis showed better resolution for PtyINR (Li et al., 4 Sep 2025). Additional tests at 0.01 s and 0.001 s likewise showed more stable behavior than APG, especially for object phase and probe recovery.

Implementation details reported for this system include a PyTorch implementation optimized with Adam, full-batch or mini-batch training, around 3.1 million parameters, approximately 20 MB memory footprint, simulations run on one RTX 4090, experimental reconstructions on an A100 SXM4, and multi-GPU support for large experimental runs. One example runtime is reconstruction of $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$8 diffraction patterns in about 30 minutes on four RTX 3090 GPUs (Li et al., 4 Sep 2025).

7. Interpretation, limitations, and broader significance

Two recurrent themes define the significance of PtyINR. The first is implicit regularization through continuous neural representation. In the X-ray case, the paper explicitly argues that the continuous neural representation suppresses unstable pixel-level noise and avoids some aliasing and grid artifacts, while the architecture choice—SIREN for the object and hash-encoded ReLU for the probe—contributes materially to robustness (Li et al., 4 Sep 2025). In the FPM case, the compact neural field is motivated by structured, sparse, and anisotropic volumetric content, implying that a dense 3D voxel array is not the most natural representation (Zhou et al., 2023).

The second theme is generalization without external supervision. FPM-INR is reported to work on two very different clinical datasets with different optics, NA, LED counts, cameras, patch sizes, and depth ranges, without retraining on external data (Zhou et al., 2023). The X-ray PtyINR paper frames the approach more broadly as a general framework for probe-dependent inverse problems in computational microscopy, with possible extension to near-field ptychography, Bragg ptychography, and other coherent imaging tasks involving unknown structured illumination (Li et al., 4 Sep 2025). This suggests a unifying methodological category: ptychographic reconstruction by neural fields constrained by a known measurement operator.

The limitations are also consistent across the two works. FPM-INR remains optimization-based rather than one-shot; it depends on accurate physics modeling and calibration; its usable refocusing range is empirical; and the simplified low-rank factorization $$I_i(x,y;z) = \left|\mathcal{F}^{-1}\left\{O(k_x-k_{x_i},k_y-k_{y_i})\,P(k_x,k_y;z)\right\}\right|^2,$$9 may limit representation of highly complex volumes with strongly coupled axial and lateral variations (Zhou et al., 2023). X-ray PtyINR is somewhat slower than GPU-accelerated iterative solvers because it has many trainable parameters; it assumes accurate probe positions; it uses a single-mode probe model; fly-scan motion blur may still affect reconstructions; and experimental data remain harder than simulations because of cropping, noise, and system imperfections (Li et al., 4 Sep 2025).

A common misconception is that PtyINR is simply a supervised deep-learning alternative to iterative ptychography. The published formulations do not support that characterization. Both FPM-INR and X-ray PtyINR are optimized directly against measured data and the physical forward model, without requiring paired labels or pretraining (Zhou et al., 2023, Li et al., 4 Sep 2025). Another misconception is that “implicit” means the method dispenses with optics; in fact, the opposite is true. The neural representation replaces the explicit object grid, not the forward operator, which remains central to the reconstruction.

In this sense, PtyINR marks a specific synthesis of neural fields and model-based computational imaging. The object is not stored as pixels or voxels but as a coordinate-conditioned function, the probe may likewise be learned as a continuous field, and the admissible solutions are restricted by ptychographic physics rather than by a learned dataset prior (Zhou et al., 2023, Li et al., 4 Sep 2025).