Adaptive foveated single-pixel imaging with dynamic super-sampling (1607.08236v1)

Abstract: As an alternative to conventional multi-pixel cameras, single-pixel cameras enable images to be recorded using a single detector that measures the correlations between the scene and a set of patterns. However, to fully sample a scene in this way requires at least the same number of correlation measurements as there are pixels in the reconstructed image. Therefore single-pixel imaging systems typically exhibit low frame-rates. To mitigate this, a range of compressive sensing techniques have been developed which rely on a priori knowledge of the scene to reconstruct images from an under-sampled set of measurements. In this work we take a different approach and adopt a strategy inspired by the foveated vision systems found in the animal kingdom - a framework that exploits the spatio-temporal redundancy present in many dynamic scenes. In our single-pixel imaging system a high-resolution foveal region follows motion within the scene, but unlike a simple zoom, every frame delivers new spatial information from across the entire field-of-view. Using this approach we demonstrate a four-fold reduction in the time taken to record the detail of rapidly evolving features, whilst simultaneously accumulating detail of more slowly evolving regions over several consecutive frames. This tiered super-sampling technique enables the reconstruction of video streams in which both the resolution and the effective exposure-time spatially vary and adapt dynamically in response to the evolution of the scene. The methods described here can complement existing compressive sensing approaches and may be applied to enhance a variety of computational imagers that rely on sequential correlation measurements.

• The paper presents an adaptive foveated strategy that dynamically targets high-resolution areas to optimize single-pixel imaging.
• It employs a digital micro-mirror device to vary resolution in real time using weighted-averaging and linear-constraints reconstruction methods.
• Experimental results demonstrate a four-fold reduction in capture time while maintaining detailed imaging of dynamic scenes.

Adaptive Foveated Single-Pixel Imaging with Dynamic Super-Sampling

This paper addresses the inherent limitations of conventional single-pixel imaging systems, primarily focusing on their reduced frame rates due to the requirement of at least as many measurements as the pixels in the reconstructed image. By introducing a novel adaptive foveated imaging strategy inspired by biological vision systems, the researchers present a framework that enhances the temporal and spatial efficiency of single-pixel imaging systems.

Overview

Conventional multi-pixel cameras capture images using an array of sensors, each corresponding to a pixel in the final image. In contrast, single-pixel cameras record images based on correlation measurements between the scene and a set of spatial patterns. This approach enables imaging in conditions where traditional cameras face challenges, such as in specific wavelengths where multi-pixel sensors are unavailable or through scattering media.

To address the trade-off between resolution and frame-rate in single-pixel imaging, the authors employ a foveated vision strategy, akin to vision mechanisms observed in animals. The central innovation is the dynamic adaptation of resolution spatially across the field of view. The area of interest within a scene is imaged at a high resolution (the fovea), while less critical areas are imaged at a lower resolution, thus reducing the total number of required measurements and increasing the frame rate.

Methodology

The methodology involves dynamically varying the resolution of different regions of the scene based on their importance or motion. By leveraging a digital micro-mirror device for manipulating masking patterns, the researchers implement a space-varied resolution system. The fovea, representing a high-resolution area, adjusts its position based on detected motion or predefined importance, ensuring that regions of interest are always captured in detail.

Two primary reconstruction strategies are explored:

1. Weighted-Averaging: This method involves averaging data from multiple sub-frames, with an emphasis on cell area to favor higher resolution data. It provides real-time reconstruction with enhanced SNR by integrating data across different frames.
2. Linear-Constraints: By solving a system of linear equations corresponding to the integral constraints set by each frame's cell data, this technique achieves higher resolution reconstructions. While computationally intensive, it yields a more detailed image through deconvolution against a spatially variant PSF.

Results and Analysis

Experimental results demonstrate a four-fold reduction in the time required to capture high-resolution images of dynamic features, such as motion in a scene. This efficiency is achieved while concurrently accumulating high-resolution details of slower moving or static regions over successive frames. The paper presents a combination of theoretical development and practical implementation, illustrating adaptive dynamics in foveated imaging.

Implications and Future Directions

The adaptive foveated approach offers significant potential for enhancing single-pixel imaging, particularly in fields constrained by challenging imaging conditions. The introduction of real-time, dynamic spatial adaptation provides a pathway to more intelligent and resource-efficient imaging systems. Potential future developments could integrate complex motion-flow algorithms or machine learning techniques for further improvement in fovea targeting and scene analysis.

By bridging the gap between the spatial resolution and temporal acquisition speed, this research lays the groundwork for advancements in computational imaging and its applications across various technical disciplines.