Instant Ghost Imaging (IGI): Real-Time Noise Reduction

• Instant Ghost Imaging is a computational imaging method that reconstructs images by correlating temporal differences between successive frames.
• IGI leverages a difference-based algorithm that dramatically reduces memory usage and computational overhead compared to conventional ghost imaging.
• FPGA-based hardware implementations demonstrate IGI’s capability for embedded, real-time imaging in challenging, noise-intensive environments.

Instant Ghost Imaging (IGI) is a computational imaging methodology and hardware paradigm that reconstructs object images by correlating temporal differences between successive measurements, fundamentally enhancing noise robustness and enabling real-time, highly resource-efficient implementation. IGI offers intrinsic resilience against spatiotemporally varying optical background noise, drastic reductions in memory and computational overhead, and pipeline architectures for simultaneous acquisition and reconstruction. It redefines the operational limits of conventional ghost imaging (GI) and single-pixel imaging (SPI), paving the way for embedded, real-time applications in hostile and dynamic environments (Yang et al., 2020, Yang et al., 2019, Yang et al., 2020).

1. Foundations: From Ghost Imaging to IGI

Conventional ghost imaging reconstructs the spatial profile of an object using the second-order intensity correlation between two beams from a pseudo-thermal or thermal source. The canonical GI setup includes:

• Test arm: Employing a bucket (single-pixel integrating) detector measuring the total transmitted or reflected intensity $S_n$ of each stochastic speckle pattern as it traverses the object.
• Reference arm: Using a spatially resolving detector capturing the reference speckle $I_n(x)$ at each pixel $x$.

The GI reconstruction is based on ensemble-averaged fluctuations: $$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$ with $\langle\cdot\rangle$ denoting the ensemble average over $N$ speckle realizations.

IGI redefines the core operation by forming correlations not of absolute intensities but of temporal differences: $$\Delta S_n \equiv S_{n+1} - S_n,\quad \Delta I_n(x) \equiv I_{n+1}(x) - I_n(x)$$

$$G_{\text{IGI}}(x) = \frac{1}{N} \sum_{n=1}^N \Delta S_n \cdot \Delta I_n(x)$$

This difference-based scheme eliminates the need for global accumulation and offline averaging, yielding real-time pipeline architectures and exceptional noise suppression (Yang et al., 2020, Yang et al., 2019).

2. Algorithmic Structure and Hardware Realization

The IGI estimator is mathematically equivalent to conventional GI in the noise-free limit but uniquely supports incremental computation. The operational flow comprises:

1. Acquisition of $S_{n+1}, I_{n+1}(x)$.
2. Computation of $\Delta S_n, \Delta I_n(x)$.
3. Update of a running accumulator $I_n(x)$0.
4. Overwrite registers for the next iteration.

The image is given after $I_n(x)$1 frames as $I_n(x)$2 (or with normalization factors, $I_n(x)$3 depending on convention).

IGI's difference-only loop means that memory requirements are independent of $I_n(x)$4 and scale only with image pixel count $I_n(x)$5. Tabulated comparison:

Feature	Conventional GI	IGI
Memory requirement	$I_n(x)$6	$I_n(x)$7
Real-time reconstruction	No	Yes
Post-processing needed	Yes	No

Hardware implementations utilize FPGA-based pipelines with per-pixel accumulators and difference logic, requiring no external DRAM even at megapixel scales. An FPGA system (Kintex-7 XC7K325T) demonstrated IGI with two CMOS sensors, achieving 500 Hz sensor rates and $I_n(x)$8-fold memory reduction versus PC-based GI (Yang et al., 2019, Yang et al., 2020).

3. Noise Rejection: Analytical and Experimental Evidence

IGI's primary advantage is the intrinsic rejection of slowly varying or common-mode noise. Modeling measured signals as $I_n(x)$9, $x$0:

• Conventional GI: Residual correlations from $x$1 directly degrade $x$2, and when $x$3 image recovery collapses.
• IGI: With most realistic background noise being spatially or temporally smooth ($x$4), contributions from $x$5 vanish, leaving the target speckle correlation unaffected.

Experimental observations confirm these analytical insights:

• With trigonometric-function amplitude noise (LED, up to $x$6 a.u.), GI failed at $x$7, while IGI maintained image contrast for $x$8 up to $x$9.
• Image visibility ($$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$0) for IGI remained $$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$1 versus nearly $$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$2 for GI under strong noise (Yang et al., 2020).

IGI thus enables clear image reconstruction under optical background levels that are orders of magnitude above the useful dynamic range of traditional GI.

4. Extensions: Single-Pixel and Wavelength-Division Multiplexed Instant GI

The IGI algorithm generalizes to single-pixel imaging (SPI) by correlating differences between sequential pattern projections and bucket-detector readings: $$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$3 An FPGA-based system driving a DMD at 20 kHz achieved real-time 32$$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$432 pixel reconstructions at up to 25 fps. All computation was performed on-chip, eliminating the need for host PC memory or offline matrix inversion (Yang et al., 2020).

For ultra-fast, single-shot ghost imaging, wavelength-division multiplexing (WDM) enables the parallel encoding of thousands of speckles via a thin wavelength-dependent diffuser and a broadband lamp:

• Each wavelength channel $$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$5 produces a speckle $$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$6, and a single spectrometer exposure captures all $$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$7 in parallel.
• Reconstruction is performed via nonnegative quadratic minimization (alternating projection), as

$$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$8

where $$G_{\text{GI}}(x) = \left\langle \left(S_n - \langle S \rangle\right) \left(I_n(x) - \langle I(x) \rangle\right)\right\rangle \approx \frac{1}{N} \sum_{n=1}^N \left(S_n - \bar S\right)\left(I_n(x) - \bar I(x)\right)$$9 is the sampling matrix of all speckles at all wavelengths.

WDM instant GI demonstrates dynamic-scene imaging at a 50 ms exposure time, limited only by spectrometer readout, not pattern projection (Deng et al., 2017).

5. Performance Metrics and Comparative Analysis

Crucial performance figures for IGI versus conventional GI include:

• Signal-to-Noise Ratio: For GI,

$\langle\cdot\rangle$0

SNR collapses for high background noise. For IGI,

$\langle\cdot\rangle$1

which is unaffected as long as noise is slow compared to $\langle\cdot\rangle$2.

• Latency and Throughput: IGI achieves “zero-time” reconstruction; update occurs during frame/projection acquisition, with no additional delay for aggregation or inversion.
• Resource Efficiency: Memory usage is reduced by orders of magnitude (examples: 26.9 Gbit for GI vs. 0.9 Mbit for IGI at $\langle\cdot\rangle$3 pixels), and processing can be accomplished with minimal logic and DSP resources on mid-range FPGAs.
• Compatibility and Scalability: Compatible with a wide range of sources (thermal, single-pixel, DMD-based), achievable in environment-immune scenarios (harsh lighting, field deployment) (Yang et al., 2019, Yang et al., 2020).

6. Applications, Limitations, and Prospects

IGI supports practical embedded imaging modalities where traditional GI is prohibitive:

• Applications:
  • Real-time LiDAR and remote sensing using single-pixel or array detectors.
  • Background-immune or low-cost imagers for biomedical, security, or environmental monitoring.
  • Imaging in radiation, underwater, or turbulent atmospheric conditions.
  • Secure-imaging platforms leveraging on-chip randomization and pattern encryption.
• Limitations:
  • Current resolution limits are set by available on-chip memory and accumulator width.
  • For ultra-high resolutions (e.g., HD video), external DDR3 or future ASICs may be required.
  • Pattern basis selection and DMD speed bottlenecks constrain minimum frame time and SNR; orthogonal bases (Hadamard, Fourier) suggest further efficiency gains.
• Future Directions:
  • Further integration into CMOS sensors or ASICs for even lower power and device size.
  • Coupling IGI with compressed sensing frameworks to minimize sample count while retaining image quality.
  • Extension to higher-order correlation and non-linear imaging regimes.

7. Summary and Significance

Instant Ghost Imaging transforms the foundational paradigm of ghost imaging from a resource-intensive, offline process into a streaming, hardware-efficient, noise-resilient, and real-time pipeline. By leveraging frame-to-frame differential correlations, IGI achieves noise immunity superior to prior art, balances image contrast and computational overhead, and enables single-chip, application-embedded imaging across optical, terahertz, and x-ray regimes. Experimental and analytical evidence confirm that IGI expands the operational domain of ghost imaging into real-world, background-intensive scenarios previously inaccessible to conventional approaches (Yang et al., 2020, Yang et al., 2019, Yang et al., 2020).