Hyperspectral pansharpening: a review

Abstract: Pansharpening aims at fusing a panchromatic image with a multispectral one, to generate an image with the high spatial resolution of the former and the high spectral resolution of the latter. In the last decade, many algorithms have been presented in the literature for pansharpening using multispectral data. With the increasing availability of hyperspectral systems, these methods are now being adapted to hyperspectral images. In this work, we compare new pansharpening techniques designed for hyperspectral data with some of the state of the art methods for multispectral pansharpening, which have been adapted for hyperspectral data. Eleven methods from different classes (component substitution, multiresolution analysis, hybrid, Bayesian and matrix factorization) are analyzed. These methods are applied to three datasets and their effectiveness and robustness are evaluated with widely used performance indicators. In addition, all the pansharpening techniques considered in this paper have been implemented in a MATLAB toolbox that is made available to the community.

• The paper provides a comprehensive overview of hyperspectral pansharpening by comparing various methods such as CS, MRA, hybrid, Bayesian, and matrix factorization techniques.
• The study evaluates these methods using datasets like Moffett Field and Camargue with metrics such as CC, SAM, and RMSE to assess spatial and spectral fidelity.
• The paper highlights that while Bayesian methods deliver high accuracy at a computational cost, MRA techniques offer a balanced approach, paving the way for future deep learning integrations.

Overview of Hyperspectral Pansharpening Techniques

The field of hyperspectral pansharpening represents a critical intersection of image processing and remote sensing, aiming to fuse high spatial resolution panchromatic (PAN) images with high spectral resolution hyperspectral (HS) imagery. This integration intends to produce images possessing both the fine spatial details of the PAN images and the rich spectral information of the HS data. The reviewed paper comprehensively surveys the methodologies developed for this challenge, comparing diverse pansharpening techniques and adapting them to hyperspectral contexts.

Categories of Pansharpening Techniques

The paper delineates the pansharpening methods into several well-defined classes:

1. Component Substitution (CS): Techniques such as Principal Component Analysis (PCA) and Gram-Schmidt (GS) fall under this category. They work by projecting the higher spectral resolution image into an alternate space, separating spatial and spectral information. The spectral distortion noted in traditional CS methods is addressed through enhanced versions like GS Adaptive (GSA), which adaptively estimates spectral correlations to reduce distortion.
2. Multiresolution Analysis (MRA): Methods here, including Smoothing Filter-based Intensity Modulation (SFIM) and Laplacian Pyramid, apply spatial filters to derive spatial details for injection into the HS data. These techniques are praised for their temporal coherence and spectral consistency while being computationally efficient.
3. Hybrid Methods: Guided Filter PCA (GFPCA) exemplifies hybrid methods that integrate CS and MRA concepts to enhance spatial resolution while minimizing spectral distortion.
4. Bayesian Approaches: Leveraging probabilistic frameworks, these techniques model the inverse problem of image fusion, utilizing priors to handle the inherent ill-posedness. Naive Gaussian and sparsity-promoted priors stand as representative models, with recent advancements like HySure incorporating both spatial and spectral characteristics comprehensively.
5. Matrix Factorization: Techniques such as Coupled Non-negative Matrix Factorization (CNMF) focus on learning a basis for the signal subspace from observed HS data, enabling the fusion with high spatial resolution inputs such as PAN images.

Experimental Evaluation and Findings

Numerous datasets, including Moffett Field, Camargue, and Garons, were utilized to validate the performance of the reviewed methods. The paper adheres to Wald’s protocol for quality assessment and employs metrics like cross-correlation (CC), spectral angle mapper (SAM), and root mean squared error (RMSE) to evaluate the pansharpened images against semi-synthetic references derived from real-life HS data.

Results emphasize that while Bayesian methods often yield superior performance due to their robust modeling of data degradations, they are computationally intensive. On the other side, methods like MRA provide a practical balance, yielding satisfactory spatial fidelity and spectral consistency with lower computational demands. PCA and GFPCA were noted for their shortcomings, particularly for HS pansharpening, where excessive spectral distortion or insufficient spatial enhancement was observed.

Implications and Future Directions

The paper highlights the continuous evolution of hyperspectral pansharpening methods, underscoring the trade-offs between computational efficiency and fidelity in spatial and spectral domains. The findings suggest potential refinements in existing methodologies, such as the enhancement of MRA techniques and further optimization in Bayesian inference algorithms to balance computational loads while maintaining robustness against spectral distortion.

Future research directions may explore the integration of deep learning approaches, which could leverage vast datasets to learn intricate mappings between PAN and HS data, potentially overcoming current limitations in versatility and generalization.

The open MATLAB toolbox provided alongside the research facilitates ongoing experimentation and development in the hyperspectral community, inviting collaboration and innovation to refine current methods and explore novel fusion paradigms.