Single-Pixel Imaging (SPI) Overview

• Single-pixel imaging (SPI) is a computational imaging technique that illuminates a scene with known patterns and reconstructs images from single scalar measurements.
• It employs diverse forward models like Fourier, Hadamard, and Kronecker formulations to optimize acquisition efficiency, noise robustness, and reconstruction speed.
• Recent advances integrate deep learning and hardware innovations, enabling high-resolution, real-time, multi-spectral imaging and even measurement-domain inference for non-traditional applications.

Single-pixel imaging (SPI) is a computational imaging technique in which a scene is illuminated by a sequence of known spatial patterns and a single-pixel detector measures only the total transmitted or reflected intensity for each pattern, rather than a spatially resolved image. In the formulations used across the literature, SPI reconstructs spatial information from sequentially encoded optoelectrical signals without pixelated detector arrays, typically through a linear forward model such as $\mathbf{y}=\mathbf{\Phi x}$ or $\mathbf{A}\mathbf{x}=\mathbf{b}$, where the measurements are inner products between the scene and projected patterns. Reported advantages include high signal-to-noise ratio, wide spectrum range, low cost, compactness, and applicability where conventional array sensors are expensive, unavailable, or impractical (Deng et al., 2018, Bian et al., 2017, Kim et al., 1 Jun 2026).

1. Measurement principle and forward models

The canonical SPI architecture consists of a spatial light modulator, commonly a digital micromirror device (DMD), a sequence of structured illumination patterns, and a single bucket detector. For each projected pattern, the detector records one scalar response. In its conventional vectorized form, the measurement model is written as

$\mathbf{y}=\mathbf{\Phi x},$

where $\mathbf{x}$ is the unknown image, $\mathbf{\Phi}$ is the sensing matrix formed by the vectorized patterns, and $\mathbf{y}$ is the measurement vector. Each measurement is therefore the inner product between the scene and one projected pattern (Deng et al., 2018).

This linear description appears in multiple algorithmic families. In Fourier SPI, the projected masks are sinusoidal Fourier basis functions, and the detector sequence corresponds to ordered acquisition of Fourier coefficients. For an object $O(x,y)$, the Fourier coefficient at spatial frequency $(u,v)$ is measured through cosine and sine illuminations, so the detector sequence itself becomes a feature vector in the measurement domain (Jiao, 2018). In Hadamard-based SPI, the image is represented in a Walsh-ordered basis, with coefficients recovered through basis scans or through complementary positive and inverse binary patterns suited to DMD hardware (Zhao et al., 2021).

A more structured two-dimensional formulation appears in Kronecker SPI, where the image is kept as a matrix instead of being fully vectorized: $Y=\Phi_h X \Phi_v^\top.$ This yields a tensor reconstruction problem

$$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$

and avoids the computational overheads associated with previous gradient-descent modules based on vectorized SPI (Qu et al., 2024).

Complementary and differential measurements are recurring implementation devices. Because DMDs display binary or nonnegative patterns, a signed mask $\mathbf{A}\mathbf{x}=\mathbf{b}$0 is often split into

$\mathbf{A}\mathbf{x}=\mathbf{b}$1

and the corresponding measurements are differenced to recover a signed coefficient while improving detection SNR (Yu et al., 2022, Zhao et al., 2021). This measurement-level normalization and differencing also underpins several ghost-imaging-style and instant reconstruction schemes (Yang et al., 2020).

2. Bases, ordering strategies, and inverse reconstruction

SPI reconstruction is typically underdetermined, so priors and basis design are central. Conventional compressive SPI commonly uses global sparsity priors in the DCT domain or total variation (TV), while correlation-based methods and direct inverse transforms remain attractive when the measurement basis is orthogonal or highly structured (Deng et al., 2018). In a unified experimental comparison of reconstruction algorithms, the reported conclusions were that, to obtain comparable reconstruction accuracy, the compressive-sensing-based TV method requires the least measurements and consumes the least running time for small-scale reconstruction; the CGD and AP methods run fastest in large-scale cases; and the TV and AP methods are the most robust to measurement noise (Bian et al., 2017).

The choice of measurement basis strongly affects both acquisition efficiency and reconstruction complexity. Random matrices are described as causing measurement blindness, computational difficulty, storage burden, and sampling-rate limitation. Deterministic Hadamard bases became an alternative because they are orthogonal, binary-valued, easy to implement on spatial modulators, and compatible with fast transforms and complementary modulation (Yu et al., 2022). The same work introduced the concept of selection history to encode the recursive folding process of Hadamard rows and columns, thereby explaining the formation mechanisms and conversion relationships of several orderings, including natural order, sequency, Russian dolls, origami, and cake-cutting. On that basis it proposed a weight sort (WH), with simulation and experimental results showing the ordering performance

$\mathbf{A}\mathbf{x}=\mathbf{b}$2

especially at low sampling ratios (Yu et al., 2022).

Fourier-space acquisition provides a different route to efficiency. Efficient SPI (eSPI) projects sinusoidal patterns for importance sampling of the scene’s spatial spectrum, exploits the centrosymmetric conjugation property $\mathbf{A}\mathbf{x}=\mathbf{b}$3 of real-valued images, and sequentially projects two $\mathbf{A}\mathbf{x}=\mathbf{b}$4-phase-shifted sinusoidal patterns to obtain each Fourier coefficient in the most informative spatial-frequency bands. The reported consequence is a reduction in requisite patterns by two orders of magnitude compared with conventional SPI, while avoiding large inverse optimizations in favor of direct spectral acquisition followed by inverse Fourier transform (Bian et al., 2015).

These developments clarify a recurrent distinction in SPI research. One class of methods improves the conditioning of the inverse problem by imposing stronger priors, such as TV, sparse representation, or structured denoisers. Another class improves the measurement process itself through deterministic bases, better orderings, and importance sampling. A plausible implication is that much of SPI performance derives from the co-design of sensing and inversion rather than from either component in isolation.

3. Learned priors, uncertainty estimation, and deep reconstruction

Learned priors entered SPI first as explicit sparse models and later as deep neural reconstructions. A representative intermediate step is convolutional sparse coding (CSC)-enhanced SPI, which combines a global prior $\mathbf{A}\mathbf{x}=\mathbf{b}$5 with a learned local prior $\mathbf{A}\mathbf{x}=\mathbf{b}$6 under the constraint

$\mathbf{A}\mathbf{x}=\mathbf{b}$7

Here the overcomplete convolutional dictionary is learned from natural images, and the resulting reconstruction is solved by ADMM. Experiments on synthetic and real data reported PSNR improvements of about $\mathbf{A}\mathbf{x}=\mathbf{b}$8 dB to $\mathbf{A}\mathbf{x}=\mathbf{b}$9 dB when CSC is added to TV, and about $\mathbf{y}=\mathbf{\Phi x},$0 dB to $\mathbf{y}=\mathbf{\Phi x},$1 dB when CSC is added to DCT, with the gain often more pronounced for DCT than for TV (Deng et al., 2018).

Deep learning then shifted emphasis from handcrafted priors to learned inverse mappings. SPI-GAN reconstructs from a crude minimum-$\mathbf{y}=\mathbf{\Phi x},$2-norm estimate using a ResNet-like generator and a VGG-like discriminator, with a composite loss combining MSE, perceptual similarity, and adversarial terms. It reported $\mathbf{y}=\mathbf{\Phi x},$3 dB PSNR and $\mathbf{y}=\mathbf{\Phi x},$4 SSIM at a $\mathbf{y}=\mathbf{\Phi x},$5 sampling ratio, about a $\mathbf{y}=\mathbf{\Phi x},$6 dB PSNR gain over the prior state of the art, and reconstruction speeds supporting about $\mathbf{y}=\mathbf{\Phi x},$7–$\mathbf{y}=\mathbf{\Phi x},$8 fps depending on the sampling ratio (Karim et al., 2021). BCS-UNet combined block compressive sensing with a fully convolutional reconstruction model, using a strictly binary measurement matrix suitable for DMD-based SPI and showing that a model trained on natural images could reconstruct images obtained from an SPI setup while being priorly trained on a different domain (Lau et al., 2022).

Deep unfolding and transformer-based models further integrated the physics of acquisition into the network architecture. HATNet unfolds tensor ISTA on a Kronecker SPI model into alternating tensor gradient descent and hybrid-attention multiscale denoising. On Set11 it reported PSNR/SSIM values of $\mathbf{y}=\mathbf{\Phi x},$9 at $\mathbf{x}$0, $\mathbf{x}$1 at $\mathbf{x}$2, $\mathbf{x}$3 at $\mathbf{x}$4, and $\mathbf{x}$5 at $\mathbf{x}$6, while also reducing GPU memory and inference time relative to vectorized SPI through the Kronecker formulation (Qu et al., 2024).

Reliability, however, became a separate problem. Bayesian CNNs were introduced to approximate pixel-wise uncertainty in SPI reconstructions by minimizing a negative log-likelihood during training and using Monte Carlo dropout during prediction. The predicted uncertainty maps were reported to correlate with absolute reconstruction error, with most errors concentrated at image edges, and the framework remained informative across varying compression ratios and noise levels (Shang et al., 2021). This supports the narrower but important claim that deep SPI reconstruction is not only an image-synthesis problem but also a confidence-estimation problem.

A later synthesis of data-driven and self-supervised approaches appears in SPI-DNIO, which combines a pretrained data-driven network with a deep image prior network in a dual-network iterative optimization loop. The stated motivation is that DIP-Net often requires thousands of iterations, while DD-Net performs optimally only when the target resembles the training set; SPI-DNIO uses the DD-Net output as initialization and refines it with measurement consistency, with experiments using $\mathbf{x}$7 iterations and reporting strong indoor and outdoor long-range performance (Shi et al., 27 Nov 2025).

4. Hardware realizations, spectral extension, and high-resolution operation

SPI hardware has expanded well beyond visible-light proof-of-concept systems. A large-area black phosphorus–molybdenum disulfide (bP-MoS$\mathbf{x}$8) photodiode was used to build a high-resolution visible-to-extended-shortwave-infrared SPI system, achieving $\mathbf{x}$9 pixels in the visible and $\mathbf{\Phi}$0 pixels in extended SWIR. This was reported as a $\mathbf{\Phi}$1 increase in pixel count over earlier van der Waals material SPI demonstrations limited to $\mathbf{\Phi}$2, with visible LEDs at $\mathbf{\Phi}$3, $\mathbf{\Phi}$4, and $\mathbf{\Phi}$5 and a $\mathbf{\Phi}$6 laser for SWIR imaging. The same work introduced compressed sampling based on a cyclic S-matrix, enabling efficient reconstruction through circular convolution and Fourier transforms, a $\mathbf{\Phi}$7 reduction in imaging time, and a simple edge-detection procedure obtained by high-pass filtering the photodiode output voltage (Khaleghi et al., 3 Oct 2025).

Near-UV SPI has also been demonstrated with a one-step-fabricated, lead-free K$\mathbf{\Phi}$8CuBr$\mathbf{\Phi}$9 photoconductor. Under $\mathbf{y}$0 nm illumination, the detector showed low dark current on the order of $\mathbf{y}$1 A, rise and recovery times of $\mathbf{y}$2 and $\mathbf{y}$3, and successful reconstruction of near-UV images, with signal-to-noise ratio improving from $\mathbf{y}$4 to $\mathbf{y}$5 dB as illumination irradiance increased (Kim et al., 1 Jun 2026).

At the system level, several works address the long-standing resolution bottleneck. Full native DMD resolution of $\mathbf{y}$6 has been reported for sparse scenes using binary, differential, non-adaptive sampling and fast regularized inversion. In one implementation, $\mathbf{y}$7 measurements at a compression ratio of about $\mathbf{y}$8 yielded an acquisition time of about $\mathbf{y}$9 s and reconstruction time of about $O(x,y)$0 s, with particularly strong performance when the field of view is limited but a priori unknown (Stojek et al., 2022). A related framework for spatially sparse scenes reported $O(x,y)$1 resolution, a compression ratio of $O(x,y)$2, and an acquisition rate of $O(x,y)$3 Hz at $O(x,y)$4 kHz DMD operation, with reconstruction latency comparable to acquisition time on a mid-tier desktop GPU (Pastuszczak et al., 1 Sep 2025).

Real-time and on-chip SPI has been pursued along a different axis. Instant single-pixel imaging (ISPI) implements the instant ghost imaging algorithm

$O(x,y)$5

on an FPGA, so reconstruction occurs during acquisition rather than afterward. Using a $O(x,y)$6 kHz DMD and $O(x,y)$7 measurements for a $O(x,y)$8 image, the reported frame rate was $O(x,y)$9 fps with almost zero reconstruction time and strong robustness to slowly varying optical background noise (Yang et al., 2020).

Platform	Reported regime	Reported result
bP-MoS$(u,v)$0 SPI (Khaleghi et al., 3 Oct 2025)	Visible to ex-SWIR	$(u,v)$1 visible, $(u,v)$2 ex-SWIR
K$(u,v)$3CuBr$(u,v)$4 SPI (Kim et al., 1 Jun 2026)	Near-UV, $(u,v)$5 nm	SNR improved from $(u,v)$6 to $(u,v)$7 dB
DMD-native sparse-scene SPI (Stojek et al., 2022)	High resolution	$(u,v)$8, about $(u,v)$9 compression
Real-time sparse-scene SPI (Pastuszczak et al., 1 Sep 2025)	High resolution, dynamic	$Y=\Phi_h X \Phi_v^\top.$0 Hz at $Y=\Phi_h X \Phi_v^\top.$1 kHz DMD operation
ISPI FPGA system (Yang et al., 2020)	On-chip real time	$Y=\Phi_h X \Phi_v^\top.$2 fps at $Y=\Phi_h X \Phi_v^\top.$3

A common misconception is that SPI is intrinsically low-resolution or confined to monochrome visible-light imaging. Reported systems span full-color photon-counting SPI, visible-to-extended-SWIR SPI, near-UV SPI, and native-$Y=\Phi_h X \Phi_v^\top.$4 sparse-scene acquisition, all using a single detector and structured modulation (Zhao et al., 2021, Khaleghi et al., 3 Oct 2025, Kim et al., 1 Jun 2026, Stojek et al., 2022).

5. Color, motion, 3D calibration, and non-image-domain tasks

SPI has expanded from monochrome still imaging to multiplexed color, motion compensation, 3D metrology, and tasks that bypass image reconstruction altogether. Full-color photon-counting SPI integrates time-correlated single-photon counting (TCSPC) with time-division multiplexing. Three monochromatic pulsed beams at $Y=\Phi_h X \Phi_v^\top.$5 nm, $Y=\Phi_h X \Phi_v^\top.$6 nm, and $Y=\Phi_h X \Phi_v^\top.$7 nm are assigned preset delays, separated in the TCSPC histogram by about $Y=\Phi_h X \Phi_v^\top.$8 ns with peak widths of about $Y=\Phi_h X \Phi_v^\top.$9 ns, and decoded into RGB Hadamard measurements in a single round of measurement. For a $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$0 image, differential Hadamard acquisition uses $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$1 patterns at a DMD projection rate of $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$2 Hz, yielding an acquisition time of $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$3 s; only about $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$4 of the laser pulses generate a photon count, confirming operation in the single-photon regime (Zhao et al., 2021).

Motion presents a fundamental challenge because SPI assumes a stationary scene during the pattern sequence. Multi-channel SPI for a composite motion target addresses this by splitting the signal into RGB wavelength channels, estimating multiple centroids, and inferring the target’s full rigid 2D motion state from translation and rotation. Using six Fourier localization masks interleaved with imaging masks and a DMD flip rate of $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$5 kHz, the reported theoretical perception frame rate reaches up to $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$6 Hz, while the pseudo-inverse transformation method enables fast motion-compensated reconstruction for translation, rotation, periodic, and non-periodic motion within a two-dimensional plane (Chongwu et al., 16 Apr 2025).

Three-dimensional SPI introduces an additional calibration burden because the system does not directly observe a conventional 2D image. A calibration-field approach, CaliF, uses a single fringe image of a cube to infer a dense 3D pointmap, enabling both projector-side and SPDG-side calibration from one image rather than hundreds. The paper compared calibration-image burden as $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$7 images for Ma et al., $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$8 images for Niu et al., and $$\hat{X}=\arg\min_X \frac{1}{2}\|Y-\Phi_h X \Phi_v^\top\|_F^2+\lambda\mathcal{R}(X),$$9 image for the CaliF method, with reported real-experiment reconstruction errors of $\mathbf{A}\mathbf{x}=\mathbf{b}$00 mm RMSE for a cube and $\mathbf{A}\mathbf{x}=\mathbf{b}$01 mm RMSE for a sphere (Ma et al., 2024).

Several works treat SPI not primarily as an imaging modality but as a measurement-domain inference or computation engine. In Fourier SPI, a naive Bayes classifier was trained directly on the single-pixel intensity sequence, reaching over $\mathbf{A}\mathbf{x}=\mathbf{b}$02 accuracy with only $\mathbf{A}\mathbf{x}=\mathbf{b}$03 illuminations, corresponding to a sampling ratio of about $\mathbf{A}\mathbf{x}=\mathbf{b}$04, even though the reconstructed image at that sampling level is extremely poor (Jiao, 2018). SPI has also been used as an optoelectronic decision system for interactive Tic-Tac-Toe, where nine one-square illumination patterns infer the $\mathbf{A}\mathbf{x}=\mathbf{b}$05 board state and a precomputed Minimax lookup table yields an unbeatable move policy (Jiao et al., 2022). In a further abstraction, an optical Ising machine encodes a combinational optimization problem as a grayscale weight map and uses evolutionary illumination patterns to solve number partition and graph maximum cut through single-pixel inner-product measurements (Huang et al., 2022).

These examples support a broader interpretation of SPI as a structured measurement platform. When the scene, task, or latent state is sufficiently constrained, the scalar measurement stream may be used for inference, control, or optimization without forming a conventional image.

6. Trade-offs, assumptions, and recurring limitations

Despite its diversity, SPI research is organized by a stable set of trade-offs. The central inverse problem is ill-posed because the number of measurements is far smaller than the number of image pixels, so performance depends on the balance among capture efficiency, computational complexity, and robustness to noise (Bian et al., 2017). The practical application of SPI has repeatedly been described as restricted by low reconstruction quality and large measurement count, particularly for correlation-based methods and under aggressive undersampling (Deng et al., 2018).

Algorithmic gains often rely on assumptions that delimit the operating regime. CSC-enhanced SPI assumes that the target scene shares structural statistics with the training database, that the learned dictionary is representative enough, and that the image can be expressed well as CSC plus a global sparse transform model; the same work explicitly notes that only $\mathbf{A}\mathbf{x}=\mathbf{b}$06 natural images were used for dictionary training, so performance could improve with more training data or class-specific dictionary learning (Deng et al., 2018). Deep data-driven methods have analogous dependence on training distribution. SPI-DNIO states that DD-Net performs optimally only when the target closely resembles the features present in its training set, while DIP-Net is more adaptable but often requires thousands of iterations and can overfit noise without early stopping (Shi et al., 27 Nov 2025). BCS-UNet, by contrast, reported that pretraining on natural images could still transfer to real SPI data, which suggests that learned priors can generalize across domains when the acquisition model and reconstruction architecture are matched carefully (Lau et al., 2022).

Hardware and scene assumptions are equally consequential. High-resolution real-time frameworks at $\mathbf{A}\mathbf{x}=\mathbf{b}$07 are targeted primarily at spatially sparse scenes, line art, handwritten digits, or scenes with large empty regions; the iterative enhancement is more versatile but more noise-sensitive, whereas the neural network enhancement is faster and often more accurate on familiar sparse scenes but less general and can introduce artifacts on out-of-distribution content (Pastuszczak et al., 1 Sep 2025). The near-UV K$\mathbf{A}\mathbf{x}=\mathbf{b}$08CuBr$\mathbf{A}\mathbf{x}=\mathbf{b}$09 detector must respond in the sub-$\mathbf{A}\mathbf{x}=\mathbf{b}$10 regime for a $\mathbf{A}\mathbf{x}=\mathbf{b}$11 kHz structured illumination sequence, and the bP-MoS$\mathbf{A}\mathbf{x}=\mathbf{b}$12 photodiode requires about $\mathbf{A}\mathbf{x}=\mathbf{b}$13 higher optical intensity in SWIR/MWIR than in the visible because of lower responsivity (Kim et al., 1 Jun 2026, Khaleghi et al., 3 Oct 2025).

Noise robustness remains method dependent. TV and AP were reported as the most robust algorithms under measurement noise in a comparative study (Bian et al., 2017). ISPI’s differencing-based robustness to optical background noise is explicitly conditioned on the background being a slow variation signal compared to the bucket signal (Yang et al., 2020). For Bayesian deep SPI, uncertainty maps consistently highlight the edges of image features, where undersampling and noise make reconstruction most difficult (Shang et al., 2021).

A second common misconception is that SPI always requires full image reconstruction before any useful inference can be performed. Direct classification from the intensity sequence, measurement-domain edge extraction through high-pass filtering of the voltage stream, lookup-table game play, and optical Ising optimization all contradict that assumption within their stated operating conditions (Jiao, 2018, Khaleghi et al., 3 Oct 2025, Jiao et al., 2022, Huang et al., 2022).

Taken together, the literature indicates that SPI is best understood not as a single reconstruction algorithm but as a family of computational imaging systems built around structured illumination, one-dimensional sensing, and task-specific inversion. The technical frontier has therefore moved in several coupled directions at once: stronger priors, more structured measurements, broader spectral hardware, faster reconstruction, and tighter integration between sensing, inference, and control.