Missing Data Reconstruction in Remote Sensing image with a Unified Spatial-Temporal-Spectral Deep Convolutional Neural Network (1802.08369v1)

Abstract: Because of the internal malfunction of satellite sensors and poor atmospheric conditions such as thick cloud, the acquired remote sensing data often suffer from missing information, i.e., the data usability is greatly reduced. In this paper, a novel method of missing information reconstruction in remote sensing images is proposed. The unified spatial-temporal-spectral framework based on a deep convolutional neural network (STS-CNN) employs a unified deep convolutional neural network combined with spatial-temporal-spectral supplementary information. In addition, to address the fact that most methods can only deal with a single missing information reconstruction task, the proposed approach can solve three typical missing information reconstruction tasks: 1) dead lines in Aqua MODIS band 6; 2) the Landsat ETM+ Scan Line Corrector (SLC)-off problem; and 3) thick cloud removal. It should be noted that the proposed model can use multi-source data (spatial, spectral, and temporal) as the input of the unified framework. The results of both simulated and real-data experiments demonstrate that the proposed model exhibits high effectiveness in the three missing information reconstruction tasks listed above.

• The paper introduces a deep learning framework that unifies spatial, temporal, and spectral reconstruction to address missing data in remote sensing imagery.
• It details a novel STS-CNN architecture featuring multi-scale convolutions, skip connections, and dilated convolutions for robust feature extraction.
• Experimental results demonstrate superior performance with improved PSNR, SSIM, and CC metrics across various reconstruction challenges.

Unified Spatial-Temporal-Spectral Deep Convolutional Neural Network for Missing Data Reconstruction in Remote Sensing

The paper under discussion presents a novel approach to addressing the pervasive issue of missing data in remote sensing imagery caused by internal sensor malfunctions or poor atmospheric conditions such as thick cloud cover. Authored by a team led by Qiang Zhang and including members like Qiangqiang Yuan and Chao Zeng, this work is slated for publication in the IEEE Transactions on Geoscience and Remote Sensing. It introduces a unified spatial-temporal-spectral (STS) framework leveraging a deep convolutional neural network (CNN) model to reconstruct missing information across various scenarios typical in remote sensing data collection.

Methodology Overview

Unlike existing methodologies which primarily tackle isolated reconstruction tasks—such as spatial-based, spectral-based, or temporal-based methods—the proposed framework utilizes a comprehensive approach that concurrently digs into the spatial, temporal, and spectral domains to boost accuracy and efficiency. This unified model is referred to as STS-CNN. The paper asserts that this approach can handle significant reconstruction tasks, including remedying dead lines in Aqua MODIS band 6, addressing the Landsat ETM+ Scan Line Corrector-off problem, and removing thick cloud interference.

The authors meticulously document the architecture of the STS-CNN, emphasizing the deployment of multi-scale convolutional feature extraction units, skip connections, and dilated convolutions within the network design. These architectural innovations aim at enhancing contextual information representation, promoting resilience against challenges like image registration errors, and providing enriched feature learning for robust recovery across different tasks.

Key Contributions and Results

In terms of contributions, the paper highlights three main advancements:

1. It introduces a deep learning-based methodology that employs a non-linear end-to-end mapping framework across spatial, spectral, and temporal dimensions to reconstruct missing remote sensing data.
2. The STS-CNN framework can harness multiple data sources, allowing it to achieve superior recovery accuracy not feasible with mono-focused methods.
3. The universality of the approach is demonstrated through its application to various common reconstruction challenges in the field, showcasing its versatility and adaptability.

The experimental section is detailed to affirm the effectiveness of the STS-CNN model using both simulated and real-data scenarios. The authors provide quantitative measures, including Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Correlation Coefficients (CC); these metrics collectively illustrate the proposed method's capability to outperform traditional methods on both visual perception and quantitative evaluation fronts.

Implications and Future Work

The implications of this research are meaningful for both practical applications and theoretical exploration. Practically, the developed model can be integrated into remote sensing image processing systems to enhance the utility of data compromised by missing information, thus improving earth observation capabilities. Theoretically, this approach offers a cohesive framework for understanding and leveraging the interplay between spatial, spectral, and temporal features in deep learning contexts.

Looking ahead, one notable limitation acknowledged by the authors is the occurrence of spectral distortions and blurring when handling cloud removal using temporal data. To mitigate this, future work could explore integrating a priori constraints or enhancing the model's rigor through innovative architectural designs.

In conclusion, this paper stands as a substantial contribution to remote sensing data processing, marrying deep learning advancements with geospatial analysis capabilities to offer a robust solution to data corruption issues pervasive in satellite imagery.