Learned Single-Pixel Fluorescence Microscopy

• Learned single-pixel fluorescence microscopy is a method that jointly trains a binary encoder and a neural decoder to optimize both measurement and reconstruction.
• It replaces traditional patterned illumination and iterative reconstruction with a data-driven, end-to-end approach that significantly speeds up image recovery.
• The technique supports multispectral imaging and hardware/software co-design, offering practical improvements in real-time biological and fluorescence microscopy.

Searching arXiv for the specified paper and closely related work to ground the article in current literature. Learned single-pixel fluorescence microscopy is a data-driven version of conventional single-pixel imaging for fluorescence microscopy in which both the measurement process and the reconstruction process are learned from data rather than fixed a priori. In the formulation introduced in "Learned Single-Pixel Fluorescence Microscopy" (Tudosie et al., 24 Jul 2025), a self-supervised autoencoder is trained so that the encoder becomes a binary linear measurement operator implementable on a digital micromirror device, while the decoder becomes a neural network that reconstructs images from compressed measurements. The approach sits within the broader lineage of compressive fluorescence microscopy, where dynamic structured illumination and a single-point detector replace a camera and the image is recovered computationally (Chahid et al., 2013). Its distinctive claim is hardware/software co-design: the learned encoder is exported to the microscope and used during acquisition, and the learned decoder replaces per-image variational optimisation.

1. Conventional single-pixel fluorescence microscopy and the learned reformulation

In conventional single-pixel fluorescence microscopy, the sample is illuminated by a sequence of spatial patterns, and a single detector records one scalar per pattern. The resulting measurement vector is a compressed linear projection of the image. Reconstruction is then performed by solving a classical inverse problem, often with total variation minimization (Tudosie et al., 24 Jul 2025).

The learned reformulation changes both stages simultaneously. Instead of using standard patterns such as scrambled Hadamard and reconstructing each image with an optimisation routine such as TVAL3, it learns the measurement matrix $E$ and the reconstruction map $D_\theta$ jointly for a target image distribution. The main claim is therefore not merely that reconstruction is learned, but that sensing itself is learned.

The contrast with earlier compressive fluorescence microscopy is structural rather than physical. The 2013 compressive-sensing microscope used a DMD to generate dynamic structured wide-field excitation patterns and a fast, sensitive single-point detector to collect one scalar measurement per pattern, with reconstruction formulated as a compressed-sensing inverse problem (Chahid et al., 2013). Learned single-pixel fluorescence microscopy preserves that linear multiplexed acquisition model while replacing hand-designed sensing and hand-designed inversion with end-to-end data-driven optimisation.

A common misconception is that learned single-pixel imaging is only a post-processing method. In the learned fluorescence setting, that is incorrect: the encoder is part of the physical device. Another misconception is that single-pixel fluorescence methods are restricted to intensity-only imaging. The learned framework was tested on physically acquired multispectral and intensity data, and the paper treats multispectral reconstruction as a central capability rather than an afterthought (Tudosie et al., 24 Jul 2025).

2. Measurement model, noise model, and inverse problem

The standard SPI forward model used in the learned method is

$y = A x,$

where $x \in \mathbb{R}^{n \times 1}$ is the unknown image, $y \in \mathbb{R}^{m \times 1}$ is the measurement vector, and $A \in \{0,1\}^{m \times n}$ is the binary measurement matrix, with $m \ll n$ in compressed acquisition (Tudosie et al., 24 Jul 2025). Each row of $A$ is reshaped into a 2D illumination pattern for the DMD. The binary constraint is physically required because the DMD can only display binary patterns.

The physically motivated noise model is Poisson-Gaussian,

$$\tilde{y} = \gamma z + \epsilon,\qquad z \sim \mathcal{P}\!\left(\frac{Ax}{\gamma}\right),\qquad \epsilon \sim \mathcal{N}(0,\sigma^2 I),$$

reflecting photon counting and electronic noise. In training, however, the authors simplify to additive Gaussian noise (Tudosie et al., 24 Jul 2025). This discrepancy is one of the paper’s explicit caveats, and it motivates later remarks that future work should model Poisson-Gaussian noise more faithfully.

For conventional compressed single-pixel reconstruction with scrambled Hadamard patterns, the paper gives the TV-regularized inverse problem as

$$\min_x \frac{\mu}{2}\| \mathbf{S}x - y \|_2^2 + \mathrm{TV}(x),$$

where $D_\theta$0 denotes the scrambled Hadamard measurement operator (Tudosie et al., 24 Jul 2025). TV is effective because fluorescence images have spatial smoothness and edges, which TV promotes. The drawbacks listed are equally important: it is iterative and therefore slow, it requires careful tuning of parameters like $D_\theta$1 and $D_\theta$2, performance varies significantly from image to image, and it is not naturally optimized for the actual data distribution.

This motivates the learned replacement. The learned decoder performs reconstruction by a single forward pass rather than an optimisation loop. A plausible implication is that the learned method should be understood as a learned inverse operator specialized to a fluorescence image class, rather than as a generic compressed-sensing solver.

3. Self-supervised autoencoder, binary encoder, and training objective

The method is called LED in the text. It learns an encoder $D_\theta$3, which is the measurement matrix or pattern set, and a decoder $D_\theta$4, which reconstructs the image (Tudosie et al., 24 Jul 2025).

The encoder is constrained to be binary because the DMD can only show on/off pixels. This is a crucial difference from earlier learned SPI work that used $D_\theta$5 patterns, which would require positive and negative exposures. Since one DMD cannot directly implement negative values, the method learns $D_\theta$6 directly. The learned patterns, once trained, end up with a fill factor around 0.5, similar to Hadamard-derived patterns, and exhibit quasi-orthogonal structure. The patterns are not constrained to be structured in advance; structure emerges from training.

The decoder $D_\theta$7 is a linear layer followed by a 2D U-Net, using Mish activations, trained with Adam. The forward model itself remains linear to stay faithful to the microscope physics; only the decoder is nonlinear (Tudosie et al., 24 Jul 2025).

The reconstruction is defined as

$D_\theta$8

with the latent measurements standardized before being passed to the decoder. The full objective combines $D_\theta$9, SSIM, and a binarization penalty:

$y = A x,$0

where $y = A x,$1, $y = A x,$2, and the weights are $y = A x,$3, $y = A x,$4, and $y = A x,$5 (Tudosie et al., 24 Jul 2025). The binarization penalty is

$y = A x,$6

This nonconvex penalty pushes entries toward 0 or 1. The parameter $y = A x,$7 is increased dynamically during training, and once binarization stops improving, the encoder is frozen and the decoder is further trained to adapt to the fully binarized encoder.

The encoder is initialized from a Beta distribution,

$y = A x,$8

which empirically speeds training. Measurements are standardized using instance normalization-like scaling without learnable parameters. This is important because real acquisitions can vary in mean intensity due to laser power, exposure, and sample brightness. Noise is then added in the latent domain during training so that the decoder learns denoising as well as reconstruction (Tudosie et al., 24 Jul 2025).

4. Optical implementation, physical acquisition, and datasets

The microscope is a single-pixel fluorescence microscope built around a DMD that projects binary illumination patterns onto the sample, a laser source, a single-pixel detector for the integrated signal, and, in the multispectral configuration, a spectrometer plus multiple single-pixel channels (Tudosie et al., 24 Jul 2025). During physical testing, the learned encoder is exported to the microscope and used during acquisition. This is the operational meaning of hardware/software co-design in the paper.

The multispectral setup uses the same pattern across multiple wavelengths, with each spectral channel measured separately by the detector array. Thus, one pattern yields a vector of measurements across wavelengths. For physical experiments, the paper collects 3 fields of view of $y = A x,$9. The cells are multispectrally stained, with membrane fluorescence around 512 nm and mitochondria around 599 nm. A CMOS camera provides a spatial intensity ground truth, and in the multispectral case the spectral ground truth is reconstructed via full Hadamard inversion (Tudosie et al., 24 Jul 2025).

Training and evaluation use several datasets: Cyto128 with 100,000 training, 10,000 validation, and 1,000 test images at $x \in \mathbb{R}^{n \times 1}$0; Cyto64 obtained by downsampling Cyto128 to $x \in \mathbb{R}^{n \times 1}$1; STL10 to train a model on natural images, called STL-LED, to test generalization to microscopy images; and a physical fluorescence slide of bovine pulmonary artery endothelial cells (Tudosie et al., 24 Jul 2025).

The broader hardware foundation predates the learned formulation. The earlier compressive-sensing fluorescence microscope demonstrated that a DMD plus single-point detection can recover images of fluorescent beads, fluorescently labeled living cells, and tissue samples with undersampling ratios up to 32, and record hyperspectral images with 128 spectral channels and undersampling ratios up to 64 (Chahid et al., 2013). This earlier work established that compressive fluorescence microscopy is compatible with biological fluorescence microscopy and particularly attractive for higher-dimensional signals.

5. Reconstruction performance, compression behavior, and multispectral recovery

The paper studies both noise behavior and compression behavior on Cyto64 and Cyto128 (Tudosie et al., 24 Jul 2025). Models are trained at six different Gaussian noise levels and tested at $x \in \mathbb{R}^{n \times 1}$2. Performance depends non-monotonically on training noise: training at a much higher noise level than the test noise hurts performance. This suggests that the learned system benefits from matching the noise regime more closely.

Compression is studied at five compression percentages. Higher compression means fewer patterns, hence faster acquisition and less photobleaching, but lower reconstruction quality. The paper observes a roughly 0.1 increase in SSIM for the $x \in \mathbb{R}^{n \times 1}$3 case when moving from 95% compression to 75% compression.

The benchmark comparison is against SH-TVAL3, LE-TVAL3, SH-LD, DCAN, STL-LED, and LED. At $x \in \mathbb{R}^{n \times 1}$4 with 409 patterns and $x \in \mathbb{R}^{n \times 1}$5, LED reports SSIM 0.82, PSNR 26.61 dB, and 14 ms, compared with SH-TVAL3 at SSIM 0.39, PSNR 15.59 dB, and 418 ms, and DCAN at SSIM 0.71, PSNR 23.37 dB, and 4 ms. At $x \in \mathbb{R}^{n \times 1}$6 with 614 patterns and $x \in \mathbb{R}^{n \times 1}$7, LED reports SSIM 0.71, PSNR 25.67 dB, and 26 ms, compared with SH-TVAL3 at SSIM 0.23, PSNR 12.53 dB, and 469 ms (Tudosie et al., 24 Jul 2025). The reported pattern of results is consistent across the section: learned patterns improve over fixed Hadamard patterns, learned reconstruction improves over TVAL3, and using both learned encoder and decoder gives the best quality overall.

The paper states that the smallest learned decoder reconstructs in about 4 ms, while TVAL3 can take about 100 times longer, supporting the claim of two-orders-of-magnitude faster reconstruction (Tudosie et al., 24 Jul 2025). For physical intensity reconstructions at 90% compression on 3 fields of view, the learned encoder helps TVAL3 preserve cell membrane structure better than standard Hadamard patterns, while the learned decoder suppresses background noise. LED combines both benefits.

For physical multispectral fluorescence microscopy, each pattern generates a stack of measurements across 16 spectral channels. The decoder is applied channel-wise, and the latent measurements are standardized. Since different wavelengths have different intensity scales, the normalized means of each channel are stored before standardization and restored after reconstruction. The multispectral reconstructions show strong reduction in reconstruction time, similar visual quality to conventional methods, correct color rendering of cell membranes and mitochondria, and successful recovery of the spectral cube with much less computation (Tudosie et al., 24 Jul 2025).

6. Significance, caveats, and related developments

The advantages claimed for the learned method are better compression-efficiency tradeoff, higher reconstruction quality, much faster reconstruction, parameter stability relative to TVAL3, compatibility with physical hardware because the encoder is binary, and multispectral capability (Tudosie et al., 24 Jul 2025). The paper argues that these properties could benefit diagnosis, biological research, and real-time microscopy, because the reconstruction time is short enough to support frame rates above 10 fps. It also argues that multispectral imaging may be provided at a fraction of the cost of conventional multispectral cameras.

The limitations are stated explicitly. Training currently assumes additive Gaussian noise, while real systems follow Poisson-Gaussian statistics. Further work is needed for lifetime imaging such as FLIM. Alternative binarization strategies and decoder architectures may improve performance, and scale-invariant decoders or better latent normalization may improve multispectral reconstruction (Tudosie et al., 24 Jul 2025). These are methodological caveats rather than refutations of the learned formulation.

A related methodological direction is "Learning Binary Sampling Patterns for Single-Pixel Imaging using Bilevel Optimisation" (Tudosie et al., 26 Aug 2025), which formulates pattern learning as a bilevel problem with a binary sensing matrix, Straight-Through Estimator-based optimisation, and Total Deep Variation. On a microscopy subset of CytoImageNet, it reports that learned binary patterns outperform Gaussian and scrambled Hadamard baselines, especially in highly undersampled regimes. This suggests a convergence of two research lines: end-to-end learned encoder-decoder systems and optimisation-based learned sensing with explicit lower-level regularisers.

A broader conceptual relative is "Calibration-free single-frame super-resolution fluorescence microscopy" (Dostálová et al., 19 May 2025). Although it is not a literal single-pixel detector system, it is described as a learned inverse problem in which a calibration-free convolutional neural network reconstructs a super-resolved fluorophore distribution from a single diffraction-limited fluorescence image. A plausible implication is that learned single-pixel fluorescence microscopy belongs to a wider class of learned fluorescence imaging methods that replace explicit inversion and calibration with data-driven estimators specialized to the measurement process.

In that context, learned single-pixel fluorescence microscopy is best understood not as a departure from compressive fluorescence microscopy, but as its learned generalization: the physical model remains linear multiplexed acquisition with structured illumination and single-point detection, while the sensing matrix and inverse map become trainable objects adapted to the image distribution and the hardware constraints (Chahid et al., 2013).