2000 character limit reached
Hyperspectral Reconstruction of Skin Through Fusion of Scattering Transform Features (2404.10030v1)
Published 15 Apr 2024 in eess.IV and cs.LG
Abstract: Hyperspectral imagery (HSI) is an established technique with an array of applications, but its use is limited due to both practical and technical issues associated with spectral devices. The goal of the ICASSP 2024 'Hyper-Skin' Challenge is to extract skin HSI from matching RGB images and an infrared band. To address this problem we propose a model using features of the scattering transform - a type of convolutional neural network with predefined filters. Our model matches and inverts those features, rather than the pixel values, reducing the complexity of matching while grouping similar features together, resulting in an improved learning process.
- M. J. Khan et al., “Modern trends in hyperspectral image analysis: A review,” IEEE Access, vol. 6, pp. 14118 – 14129, 2018.
- P. C. Ng et al., “Hyperspectral skin vision challenge: Can your camera see beyond your skin?,” in ICASSP 2024 SPGC IEEE International Conference on Acoustics, Speech and Signal Processing Grand Challenge, 2024.
- P. C. Ng et al., “Hyper-skin: A hyperspectral dataset for reconstructing facial skin-spectra from RGB images,” in Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
- S. Mallat, “Group invariant scattering,” Communications in Pure and Applied Mathematics, vol. 65, no. 10, pp. 1331–1398, 2012.
- “The pre-image problem for Laplacian eigenmaps utilizing l1subscript𝑙1l_{1}italic_l start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT regularization with applications to data fusion,” Inverse Problems, vol. 33, no. 7, pp. 074006, 2017.
- T. Angles and S. Mallat, “Generative networks as inverse problems with scattering transforms,” in Proc. ICLR, 2018.
- M. Andreux et al., “Kymatio: Scattering transforms in Python,” Journal of Machine Learning Research, vol. 21, no. 60, pp. 1–6, 2020.
- Y. Cai et al., “MST++: Multi-stage spectral-wise transformer for efficient spectral reconstruction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 745–755.
- “3-d Fourier scattering transform and classification of hyperspectral images,” Transactions on Geoscience and Remote Sensing, vol. 59, no. 12, pp. 10312–10327, 2021.