Deep Fully-Connected Networks for Video Compressive Sensing (1603.04930v2)

Abstract: In this work we present a deep learning framework for video compressive sensing. The proposed formulation enables recovery of video frames in a few seconds at significantly improved reconstruction quality compared to previous approaches. Our investigation starts by learning a linear mapping between video sequences and corresponding measured frames which turns out to provide promising results. We then extend the linear formulation to deep fully-connected networks and explore the performance gains using deeper architectures. Our analysis is always driven by the applicability of the proposed framework on existing compressive video architectures. Extensive simulations on several video sequences document the superiority of our approach both quantitatively and qualitatively. Finally, our analysis offers insights into understanding how dataset sizes and number of layers affect reconstruction performance while raising a few points for future investigation. Code is available at Github: https://github.com/miliadis/DeepVideoCS

• The paper presents a deep learning framework that maps compressed video measurements directly to video frames using fully-connected networks.
• It demonstrates significant improvements in reconstruction quality with higher PSNR and SSIM and reduced computational cost compared to traditional methods.
• The approach aligns with existing hardware capabilities by using a repetitive block measurement matrix, enabling real-time high-speed imaging applications.

Deep Fully-Connected Networks for Video Compressive Sensing

The paper "Deep Fully-Connected Networks for Video Compressive Sensing" by Iliadis, Spinoulas, and Katsaggelos presents a novel approach to solving the high-demand problem of reconstructing video from compressed measurements using deep learning techniques. The authors contribute to the field by proposing a deep fully-connected network architecture aimed at improving both the efficiency and quality of video reconstruction in the context of temporal compressive sensing (CS).

Overview and Methodology

Traditional video compressive sensing seeks to enhance the temporal resolution by leveraging physical and algorithmic innovations that enable high-speed video capture beyond the inherent frame rate limitations of conventional cameras. However, current reconstruction algorithms tend to be computationally intensive and impractically slow for widespread use.

In contrast, this paper introduces a method utilizing deep fully-connected networks to directly map CS measurements to video frames. The research begins by demonstrating that even a simple linear mapping between compressed measurements and video frames can achieve promising results. Building upon this foundation, the authors extend the model to more sophisticated deep network architectures, significantly improving upon both computational cost and reconstruction quality when compared to established methods. The networks are trained using an extensive dataset, consisting of diverse high-definition video sequences to learn the mapping effectively.

Key Contributions and Results

The authors identify several key contributions in this work:

1. Introduction of a Deep Learning Framework for Video CS: The authors effectively introduce a deep learning-based architecture utilizing fully-connected neural networks, setting a precedent in the domain of temporal video CS.
2. Performance Improvements: Experimental results demonstrate significant gains in PSNR and SSIM compared to leading methods such as GMM-based reconstructions. For instance, the seven-layer network trained on a large dataset (10 million samples) exhibits notable improvements in quality metrics with the advantage of reduced computational expense.
3. Real-World Applicability: By constructing a measurement matrix with a repetitive block structure, the proposed method aligns with existing hardware capabilities, aiding potential real-world adoption.

Practical and Theoretical Implications

The implications of successfully applying deep learning to the video CS problem are substantial. Practically, these advancements suggest the possibility of deploying high-speed imaging applications at a reduced computational and economic cost. This is particularly relevant in fields requiring real-time video processing, such as surveillance, biomedical imaging, and any application where high-frame-rate cameras are often too costly.

From a theoretical perspective, the findings reinforce the potential of deep learning paradigms in solving inverse problems traditionally dominated by heuristic or iterative approaches. This paper invites further research into the use of deep networks for similar inverse problems in compressive sensing and beyond, possibly extending into domains like MRI reconstruction or other sensing applications.

Future Directions

While the results are promising, several future research avenues are highlighted, including exploring the utility of deeper and more complex network architectures such as CNNs or RNNs, evaluating the application of this method to real-world video sequences captured via temporal CS cameras, and testing the performance of these networks when subjected to noise and other real-world distortions.

In summary, the paper successfully demonstrates the viability and benefits of employing deep fully-connected networks for video compressive sensing, offering significant performance improvements and paving the way for more efficient and practical high-speed imaging solutions.