A novel approach to combine spatial and spectral information from hyperspectral images

Abstract: This article proposes a generic framework to process jointly the spatial and spectral information of hyperspectral images. First, sub-images are extracted. Then each of these sub-images follows two parallel workflows, one dedicated to the extraction of spatial features and the other dedicated to the extraction of spectral features. Finally, the extracted features are merged, producing as many scores as sub-images. Two applications are proposed, illustrating different spatial and spectral processing methods. The first one is related to the characterization of a teak wood disk, in an unsupervised way. It implements tensors of structure for the spatial branch, simple averaging for the spectral branch and multi-block principal component analysis for the fusion process. The second application is related to the early detection of apple scab on leaves. It implements co-occurrence matrices for the spatial branch, singular value decomposition for the spectral branch and multiblock partial least squares discriminant analysis for the fusion process. Both applications demonstrate the interest of the proposed method for the extraction of relevant spatial and spectral information and show how promising this new approach is for hyperspectral imaging processing.