PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening (1805.03371v4)

Abstract: This paper addresses the problem of remote sensing image pan-sharpening from the perspective of generative adversarial learning. We propose a novel deep neural network based method named PSGAN. To the best of our knowledge, this is one of the first attempts at producing high-quality pan-sharpened images with GANs. The PSGAN consists of two components: a generative network (i.e., generator) and a discriminative network (i.e., discriminator). The generator is designed to accept panchromatic (PAN) and multispectral (MS) images as inputs and maps them to the desired high-resolution (HR) MS images and the discriminator implements the adversarial training strategy for generating higher fidelity pan-sharpened images. In this paper, we evaluate several architectures and designs, namely two-stream input, stacking input, batch normalization layer, and attention mechanism to find the optimal solution for pan-sharpening. Extensive experiments on QuickBird, GaoFen-2, and WorldView-2 satellite images demonstrate that the proposed PSGANs not only are effective in generating high-quality HR MS images and superior to state-of-the-art methods and also generalize well to full-scale images.

• The paper introduces PSGAN, a GAN-based approach that treats pan-sharpening as an image generation problem by fusing PAN and MS images.
• It employs a two-stream input architecture with attention mechanisms to enhance spatial detail and spectral fidelity over traditional methods.
• Experimental results on QuickBird, GaoFen-2, and WorldView-2 datasets show PSGAN outperforms state-of-the-art methods using metrics like SAM, CC, and ERGAS.

Overview of "PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-sharpening"

This paper presents a novel approach to remote sensing image pan-sharpening through a Generative Adversarial Network (GAN) named PSGAN. The task of pan-sharpening involves fusing high-resolution panchromatic (PAN) images and low-resolution multispectral (MS) images to create high-resolution multispectral images, combining both spatial and spectral information. The authors propose that treating pan-sharpening as an image generation problem can lead to higher quality results as compared to traditional techniques.

Methodology and Architecture

The PSGAN model is composed of two primary components: the generator and the discriminator. The generator takes PAN and MS images as inputs and outputs high-resolution MS images. The discriminator evaluates the fidelity of these generated images by distinguishing them from real HR MS images. Several architectural configurations are explored, including two-stream input, stacking input, batch normalization layers, and attention mechanisms. The authors conclude that a two-stream input architecture, which processes PAN and MS separately before merging, yields superior results.

Experimental Validation

Extensive experiments were conducted using satellite images from QuickBird, GaoFen-2, and WorldView-2. The proposed PSGAN method demonstrated effectiveness in producing high-quality pan-sharpened images, outperforming state-of-the-art methods in terms of spatial and spectral fidelity. Quantitative evaluations were performed using metrics such as Spectral Angle Mapper (SAM), Correlation Coefficient (CC), and ERGAS among others. Notably, the results from PSGAN, particularly with the two-stream configuration, were exceptionally strong, highlighting its capacity for high-quality image generation.

Implications and Future Directions

The implications of this research are substantial for remote sensing applications, such as environmental monitoring and urban planning, where high-resolution multispectral images are critical. The success of PSGAN suggests that GAN-based architectures could replace traditional pan-sharpening methods, offering more precise detail and preserving spectral integrity.

Looking forward, the paper suggests exploring unsupervised learning approaches to further enhance the generalization capability of PAN and MS images without preprocessing. Furthermore, integrating perceptual losses or leveraging additional spectral information could potentially improve the consistency between synthetic and real images, eliminating the residual smoothness observed in some outputs.

Conclusion

This paper marks a significant contribution to the field of remote sensing image processing by leveraging GANs for pan-sharpening tasks. While feature richness from GAN-based models enhances image quality, the challenge remains in further optimizing these models for scalable and unsupervised settings. This research lays a foundation for future works aiming to extend deep learning methodologies to broader remote sensing applications.