DiffuserCam: Lensless Computational Imaging

• DiffuserCam is a lensless computational imaging system that encodes 3D scenes into pseudorandom caustic patterns, enabling volumetric reconstruction via compressed sensing.
• It employs calibrated point spread functions with a shift-invariant approximation, using sparse regularization and FFT-based methods to efficiently reconstruct high-resolution images.
• Recent extensions integrate spectral, polarization, and learned inversion models, demonstrating versatile adaptations for high-dimensional and task-specific imaging applications.

DiffuserCam is a lensless computational camera in which a thin diffuser is placed directly in front of a conventional image sensor, so that the diffuser replaces the lens as the primary optical encoding element. Instead of forming a focused image, each scene point produces a pseudorandom caustic pattern on the sensor, and computational inversion reconstructs a 2D image or a higher-dimensional scene representation from the resulting multiplexed measurement. In the original formulation, DiffuserCam was introduced as a compact, single-exposure 3D imaging system that reconstructs a volumetric scene from one 2D sensor image by combining a physical approximation, a simple calibration scheme, and sparse regularization (Antipa et al., 2017).

1. Origins and defining concept

DiffuserCam emerged as a lensless alternative to conventional refractive imaging. Conventional cameras use lenses to form a focused 2D image of a 3D scene on a sensor, whereas DiffuserCam places a diffuser a few millimeters in front of the sensor and treats the resulting caustic pattern as a coded measurement of the scene. The diffuser acts like a random microlens ensemble: each 3D point source produces a distinctive pseudorandom caustic pattern, and different 3D positions produce shifted and scaled variants of that pattern. Because these patterns are rich in high spatial frequencies and sufficiently uncorrelated, they support compressed sensing reconstruction of a large voxel grid from a much smaller number of sensor measurements (Antipa et al., 2017).

The original system was presented as a lensless, single-exposure 3D imaging architecture. It used a diffuser placed in front of a standard image sensor, reconstructed on the order of 100 million 3D voxels from a single 1.3 megapixel image after $2\times 2$ binning, and emphasized that the effective resolution varies significantly with scene content. A central conclusion of that work was that resolution in computational cameras cannot be described solely by optical two-point criteria; it also depends on the inverse problem and on the sparsity or complexity of the scene (Antipa et al., 2017).

Subsequent work generalized the term “DiffuserCam” beyond that original 3D instrument. In later literature, “DiffuserCam” denotes a family of lensless imaging systems in which a diffuser or thin scatterer produces a known, complex, spatial encoding of the scene, and reconstruction is performed computationally from that coded measurement. This broader family includes 2D photographic systems, hyperspectral and polarization extensions, temporal compressive imaging, single-pixel variants, and recent learning-based reconstructions that use diffusion priors or operator-learning architectures (Monakhova et al., 2020).

2. Optical principle and forward models

The core optical principle is global multiplexing by a diffuser-generated point spread function. In the original 3D system, the incoherent scene is discretized as a set of voxels labeled by $(x,y,z)$, and the sensor measurement is the superposition of the corresponding voxel PSFs: $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$ or, in vector form,

$b = H v.$

To make this tractable, DiffuserCam adopts a shift-invariant approximation at each depth plane, so that a single on-axis PSF $h(x',y';z)$ can represent the full depth slice under a convolutional model (Antipa et al., 2017).

Under that approximation, the 3D scene is encoded as a sum of 2D convolutions over depth: $$b(x',y') = C \sum_{z} \bigg[v\!\left(\frac{-x'}{m}, \frac{-y'}{m}, z\right) * h(x',y';z)\bigg],$$ where $C$ is a cropping operator and $m$ is a paraxial magnification factor. This factorization is essential because it converts a nominally petabyte-scale operator into a depth-indexed bank of calibrated PSFs that can be applied efficiently with FFT-based convolutions (Antipa et al., 2017).

Later papers often simplify DiffuserCam further to a 2D linear shift-invariant model,

$y = h * x + \eta,$

especially for photographic or planar-scene tasks. This form appears explicitly in recent reconstruction work based on Fourier Neural Operators, which argues that the global support of the diffuser PSF makes the inverse problem especially well aligned with spectral-domain operators. In Fourier space, the model becomes

$$\hat{y}(\mathbf{k}) = \hat{h}(\mathbf{k})\,\hat{x}(\mathbf{k}) + \hat{\eta}(\mathbf{k}),$$

highlighting that DiffuserCam reconstruction is fundamentally a global spectral inverse problem rather than a local denoising task (Ekec et al., 17 Apr 2026).

Other variants retain the same linear-operator viewpoint while changing the detector or the encoded dimensions. The single-pixel diffuser camera, for example, places the object image on a rotating diffuser and records a single scalar per diffuser angle,

$(x,y,z)$0

so that the diffuser patterns $(x,y,z)$1 act as a time-varying sensing matrix. Spectral DiffuserCam augments the sensor with a tiled spectral filter array and encodes a hyperspectral cube $(x,y,z)$2 through a combined diffuser-plus-filter operator. Polarization and 5D systems extend the same framework to additional polarization and temporal axes by adding a striped polarization mask or rolling-shutter timing to the optical stack (Liu et al., 2021).

3. Calibration and inversion

DiffuserCam relies on calibration rather than closed-form geometric optics. In the original 3D implementation, calibration is reduced to recording one on-axis PSF per depth plane. A point source is placed on axis at each desired depth $(x,y,z)$3, and the resulting caustic pattern is stored as $(x,y,z)$4. This makes calibration practical: the prototype used 128 depth planes from $(x,y,z)$5 mm to $(x,y,z)$6 mm from the diffuser, rather than calibrating every lateral voxel independently (Antipa et al., 2017).

Reconstruction in the original system is posed as a sparsity-regularized inverse problem: $(x,y,z)$7 where $(x,y,z)$8 is either the identity for native sparsity or a finite-difference operator for 3D total variation. The solver is an ADMM algorithm designed around the convolutional structure of $(x,y,z)$9 and the circulant structure of the regularizer, so that the dominant updates reduce to diagonal operations in the Fourier domain. This makes volumes with tens to hundreds of millions of voxels computationally tractable, although reconstruction still requires minutes and substantial memory (Antipa et al., 2017).

A distinct calibration-and-inversion pattern appears in the single-pixel diffuser camera. There the system is calibrated by rotating the diffuser from $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$0 to $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$1, recording the intensity patterns $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$2 at the diffuser plane, and resizing them to a $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$3 grid for reconstruction. Rather than solving an explicit sparse inverse problem, that work uses differential ghost imaging: $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$4 This yields $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$5 reconstructions at sampling ratios of $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$6, $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$7, and $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$8, with best quality around $$b(x',y') = \sum_{(x,y,z)} v(x,y,z)\, h(x',y';x,y,z),$$9, corresponding to $b = H v.$0 diffuser angles and a minimum acquisition time of 14.4 s (Liu et al., 2021).

Calibration accuracy remains a defining constraint across the family. Later work on spatially varying deconvolution for DiffuserCam stresses that a single center PSF is an inaccurate model for wide field of view operation, because the diffuser is close to the sensor and off-axis angles produce different PSFs. That work addresses the problem with learned multi-kernel deconvolution initialized from a single calibrated PSF, thereby approximating spatial variation without exhaustive multi-location calibration (Cai et al., 2024).

4. Architectural extensions and high-dimensional variants

One of the most consequential developments was the extension from volumetric intensity imaging to other encoded dimensions. Spectral DiffuserCam added a spectral filter array directly on the sensor, arranged as a grid of $b = H v.$1 super-pixels, each containing an $b = H v.$2 mosaic of narrowband filters for 64 spectral channels spanning 386–898 nm. The prototype sensor was cropped to $b = H v.$3 pixels, and reconstruction recovered a hyperspectral datacube using a forward model that combines spatial convolution by the diffuser PSF with wavelength-dependent pointwise modulation by the filter array. The system reported peak wavelength localization within $b = H v.$4 nm across the band and sub-super-pixel spatial resolution, with two-point resolution around 0.19 super-pixels and more realistic multi-point resolution around 0.3 super-pixels (Monakhova et al., 2020).

A single-pixel branch of the DiffuserCam family pursued the opposite hardware extreme. Instead of a 2D sensor, it used a single amplified photodetector and a rotating ground-glass diffuser to encode a 2D object into a 1D temporal signal. A hyperspectral extension was then implemented by adding a low-cost transmission grating and binning a compact CMOS sensor into a $b = H v.$5 line array, so that each wavelength channel behaved like its own “single pixel.” The demonstrated spectral range was approximately 426–637 nm, with example bands at $b = H v.$6 nm and $b = H v.$7 nm, reconstructed independently per wavelength channel into a 3D spatio-spectral cube (Liu et al., 2021).

Temporal and task-oriented variants also appeared. A temporal compressive edge-imaging system based on a lensless diffuser camera used the rolling shutter of a CMOS sensor so that different sensor rows corresponded to different times during a single exposure. Instead of reconstructing intensity frames and applying an edge detector afterward, the edge operator was inserted into the forward model: $b = H v.$8 allowing direct recovery of a time sequence of edge images from one diffuser measurement. The work reported that this direct edge reconstruction produced higher PSNR and information entropy than reconstruct-then-filter pipelines across sampling rates from 20% to 90% (Zheng et al., 2023).

Polarization and multi-dimensional extensions followed the same logic. A lensless polarization camera combined a diffuser with a striped polarization mask made from linear polarizer film, with stripe orientations $b = H v.$9, to recover four linear polarization sub-images from a single snapshot. Diffuser-mCam went further, combining a thick diffuser, a four-strip polarization mask, and rolling-shutter timing to reconstruct 5D data $h(x',y';z)$0: 2D space, 3 spectral bands, 4 linear polarization states, and 36 time bins. In that system, the full 5D mode used 12 spectral–polarization PSFs and operated at a reported sampling rate of 2.5% per channel, while reduced-dimensional modes supported temporal-only, multispectral-only, or polarization-only reconstruction from the same hardware by selecting different subsets of calibrated PSFs (Kraicer et al., 17 Mar 2026).

5. Learned reconstruction and generative priors

As DiffuserCam systems matured, reconstruction moved from sparsity-based optimization toward learned inverse models. A key intermediate step was the recognition that diffuser-based imaging is strongly non-local: each scene point influences a large fraction of the sensor, and the corresponding inverse problem is poorly matched to purely local CNN architectures. Work on FourierNets used DiffuserCam as the prototypical non-local optical encoder and showed that shallow networks with global Fourier convolutions surpass locality-biased UNets in this setting. On the lensless Mirflickr dataset, a FourierNet achieved MSE $h(x',y';z)$1, LPIPS 0.20, MS-SSIM 0.882, and PSNR 24.8 dB, outperforming both a direct UNet and the hybrid Le-ADMM-U model while remaining fast at inference (Deb et al., 2021).

This operator-centric viewpoint was pushed further by a dedicated Fourier Neural Operator reconstruction framework. Using a compact DiffuserCam prototype and a 25,000-image natural-scene dataset, that work trained both an FNO and a U-Net only on $h(x',y';z)$2 data, then evaluated across unseen resolutions. The FNO improved over a comparable-size U-Net by $h(x',y';z)$3 dB in PSNR and $h(x',y';z)$4 in SSIM at $h(x',y';z)$5, while the same trained model reconstructed $h(x',y';z)$6 and $h(x',y';z)$7 measurements with less than $h(x',y';z)$8 dB PSNR loss and no retraining. This was presented as a resolution-agnostic consequence of learning an operator in Fourier space rather than a discretization-specific local mapping (Ekec et al., 17 Apr 2026).

Another line of work brought large generative priors into lensless imaging. DifuzCam replaced a conventional lens with a static amplitude mask near the sensor, adopted the FlatCam separable forward model

$h(x',y';z)$9

and reconstructed images with a learned separable transform, a ControlNet-like conditioning path, and a frozen Stable Diffusion 2.1 backbone. The paper emphasized that a diffusion loss alone causes degeneracy toward pure text-to-image generation, and therefore introduced an additional separable reconstruction loss

$$b(x',y') = C \sum_{z} \bigg[v\!\left(\frac{-x'}{m}, \frac{-y'}{m}, z\right) * h(x',y';z)\bigg],$$0

to force the system to preserve measurement information. On its own prototype dataset, the proposed method with text guidance reported PSNR 21.58 dB, SSIM 0.541, LPIPS 0.276, and CLIP 24.38, while the ablation without the separable loss collapsed to PSNR 9.84 and SSIM 0.175 despite a high CLIP score (Yosef et al., 2024).

PhoCoLens addressed a central DiffuserCam difficulty: the trade-off between data fidelity and photorealism. Its first stage used spatially varying deconvolution to approximate the range-space component of the signal, explicitly motivated by the inadequacy of a single shift-invariant PSF across the field of view. Its second stage conditioned a frozen Stable Diffusion model on that range-space reconstruction to inject plausible high-frequency detail while maintaining consistency with the measured content. On the DiffuserCam dataset of 25,000 paired images, PhoCoLens reported PSNR 24.12 dB, SSIM 0.748, and LPIPS 0.161, with LPIPS substantially better than earlier baselines even when PSNR was only marginally different. The paper framed this as a principled range/null-space decomposition of the reconstruction problem (Cai et al., 2024).

6. Resolution, trade-offs, and broader significance

DiffuserCam’s compactness comes from replacing conventional optics with randomness plus computation, but this substitution introduces characteristic trade-offs. The original 3D camera emphasized that its voxel grid should be chosen to match experimentally measured multi-point resolution rather than idealized two-point criteria. At around 20 mm depth, two-point lateral resolution was approximately $$b(x',y') = C \sum_{z} \bigg[v\!\left(\frac{-x'}{m}, \frac{-y'}{m}, z\right) * h(x',y';z)\bigg],$$1–$$b(x',y') = C \sum_{z} \bigg[v\!\left(\frac{-x'}{m}, \frac{-y'}{m}, z\right) * h(x',y';z)\bigg],$$2, while a more realistic multi-point or USAF-style resolution was approximately $$b(x',y') = C \sum_{z} \bigg[v\!\left(\frac{-x'}{m}, \frac{-y'}{m}, z\right) * h(x',y';z)\bigg],$$3–$$b(x',y') = C \sum_{z} \bigg[v\!\left(\frac{-x'}{m}, \frac{-y'}{m}, z\right) * h(x',y';z)\bigg],$$4. The paper traced this gap to the condition number of local submatrices of the forward operator, showing that effective resolution degrades with object complexity but does not degrade without bound (Antipa et al., 2017).

Field-of-view dependence and model mismatch constitute a second recurring limitation. Shift invariance is typically accurate near the optical axis and degrades toward high field angles. This appears in the original DiffuserCam as reduced edge-of-field fidelity, in PhoCoLens as the motivation for spatially varying deconvolution, and in the lensless polarization camera as sensitivity to mask-sensor spacing and mask mismatch. In the polarization system, the dominant degradation was attributed not to the diffuser itself but to blur and structural deviations introduced by the polarization mask, particularly its effective distance of about 1.36 mm from the sensor (Cai et al., 2024).

Hardware simplicity also trades against speed and control. Rotating-diffuser and single-pixel implementations are broadband and inexpensive but mechanically slow; the single-pixel diffuser camera reported a minimum acquisition time of 14.4 s for 8.8% sampling. DMD-based single-pixel systems can switch faster and provide orthogonal Hadamard-like patterns, whereas rotating diffusers provide only pseudo-random grayscale masks whose correlations limit efficiency. Similar trade-offs arise in multi-dimensional systems: Diffuser-mCam achieves 5D snapshot capture in a $$b(x',y') = C \sum_{z} \bigg[v\!\left(\frac{-x'}{m}, \frac{-y'}{m}, z\right) * h(x',y';z)\bigg],$$5, 6 g package, but full 5D inversion is extremely underdetermined and computationally demanding, which is why the paper advocates modality-switchable reduced reconstructions when the full datacube is unnecessary (Liu et al., 2021).

Taken together, these developments position DiffuserCam as a general lensless imaging paradigm rather than a single device. The recurring pattern is stable across implementations: a diffuser or thin scatterer produces a calibrated, spatially extended encoding; the scene is inferred from a coded sensor measurement by solving an inverse problem; and the scientific challenges revolve around conditioning, calibration fidelity, field variation, priors, and computational tractability. This suggests that DiffuserCam is best understood not merely as “a diffuser in front of a sensor,” but as a framework for replacing optical complexity with a jointly designed encoder, calibration protocol, and reconstruction algorithm across 3D, spectral, polarization, temporal, and learned-imaging regimes (Zheng et al., 18 Jul 2025).