Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

HSR-Diff:Hyperspectral Image Super-Resolution via Conditional Diffusion Models (2306.12085v1)

Published 21 Jun 2023 in cs.CV and eess.IV

Abstract: Despite the proven significance of hyperspectral images (HSIs) in performing various computer vision tasks, its potential is adversely affected by the low-resolution (LR) property in the spatial domain, resulting from multiple physical factors. Inspired by recent advancements in deep generative models, we propose an HSI Super-resolution (SR) approach with Conditional Diffusion Models (HSR-Diff) that merges a high-resolution (HR) multispectral image (MSI) with the corresponding LR-HSI. HSR-Diff generates an HR-HSI via repeated refinement, in which the HR-HSI is initialized with pure Gaussian noise and iteratively refined. At each iteration, the noise is removed with a Conditional Denoising Transformer (CDF ormer) that is trained on denoising at different noise levels, conditioned on the hierarchical feature maps of HR-MSI and LR-HSI. In addition, a progressive learning strategy is employed to exploit the global information of full-resolution images. Systematic experiments have been conducted on four public datasets, demonstrating that HSR-Diff outperforms state-of-the-art methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Bayesian sparse representation for hyperspectral image super resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3631–3640, 2015.
  2. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214–223. PMLR, 2017.
  3. Super-resolution for hyperspectral and multispectral image fusion accounting for seasonal spectral variability. IEEE Transactions on Image Processing, 29:116–127, 2019.
  4. Crossvit: Cross-attention multi-scale vision transformer for image classification. In Proceedings of the IEEE/CVF international conference on computer vision, pages 357–366, 2021.
  5. Wavegrad: Estimating gradients for waveform generation. In Proceedings of the International Conference on Learning Represent, 2021.
  6. Fusion of hyperspectral and multispectral images: A novel framework based on generalization of pan-sharpening methods. IEEE Geoscience and Remote Sensing Letters, 11(8):1418–1422, 2014.
  7. Improving hyperspectral image segmentation by applying inverse noise weighting and outlier removal for optimal scale selection. ISPRS Journal of Photogrammetry and Remote Sensing, 171:348–366, 2021.
  8. Deep hyperspectral image sharpening. IEEE transactions on neural networks and learning systems, 29(11):5345–5355, 2018.
  9. Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016.
  10. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2):295–307, 2015.
  11. Hyperspectral image super-resolution with optimized rgb guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11661–11670, 2019.
  12. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  13. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
  14. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
  15. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  16. Bam: Bilateral activation mechanism for image fusion. In Proceedings of the 29th ACM International Conference on Multimedia, pages 4315–4323, 2021.
  17. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018.
  18. Auto-encoding variational bayes. In Proceedings of the International Conference on Learning Represent, 2013.
  19. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4681–4690, 2017.
  20. Fusing hyperspectral and multispectral images via coupled sparse tensor factorization. IEEE Transactions on Image Processing, 27(8):4118–4130, 2018.
  21. Spectral–spatial classification of hyperspectral imagery with 3d convolutional neural network. Remote Sensing, 9(1):67, 2017.
  22. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021.
  23. Gamba Paolo. Pavia centre and university. https://www.ehu.eus/ccwintco/index.php?title===Hypersp ectral_Remote_Sensing_Scenes, 2011.
  24. Unsupervised sparse dirichlet-net for hyperspectral image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2511–2520, 2018.
  25. Variational inference with normalizing flows. In International conference on machine learning, pages 1530–1538. PMLR, 2015.
  26. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning, pages 1278–1286. PMLR, 2014.
  27. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  28. Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. In Proceedings of the International Conference on Learning Represent, 2017.
  29. A latent encoder coupled generative adversarial network (le-gan) for efficient hyperspectral image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 60:1–19, 2022.
  30. Nvae: A deep hierarchical variational autoencoder. Advances in Neural Information Processing Systems, 33:19667–19679, 2020.
  31. Pixel recurrent neural networks. In International conference on machine learning, pages 1747–1756. PMLR, 2016.
  32. Convolutional lstm-based hierarchical feature fusion for multispectral pan-sharpening. IEEE Transactions on Geoscience and Remote Sensing, 60:1–16, 2022.
  33. Hyperspectral image super-resolution with deep priors and degradation model inversion. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2814–2818. IEEE, 2022.
  34. Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7):3658–3668, 2015.
  35. Mhf-net: An interpretable deep network for multispectral and hyperspectral image fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  36. Material based object tracking in hyperspectral videos. IEEE Transactions on Image Processing, 29:3719–3733, 2020.
  37. Hyperspectral image superresolution using unidirectional total variation with tucker decomposition. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13:4381–4398, 2020.
  38. Grafting transformer on automatically designed convolutional neural network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60:1–16, 2022.
  39. Hyperspectral and multispectral image fusion via deep two-branches convolutional neural network. Remote Sensing, 10(5):800, 2018.
  40. Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE transactions on image processing, 19(9):2241–2253, 2010.
  41. Airborne hyperspectral data over chikusei. Space Appl. Lab., Univ. Tokyo, Tokyo, Japan, Tech. Rep. SAL-2016-05-27, 5, 2016.
  42. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5728–5739, 2022.
  43. 3-d-anas: 3-d asymmetric neural architecture search for fast hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60:1–19, 2021.
  44. Unsupervised adaptation learning for hyperspectral imagery super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3073–3082, 2020.
  45. A survey of hyperspectral image super-resolution technology. In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pages 4476–4479. IEEE, 2021.
  46. Ssr-net: Spatial–spectral reconstruction network for hyperspectral and multispectral image fusion. IEEE Transactions on Geoscience and Remote Sensing, 59(7):5953–5965, 2020.
  47. Coupled convolutional neural network with adaptive response function learning for unsupervised hyperspectral super resolution. IEEE Transactions on Geoscience and Remote Sensing, 59(3):2487–2502, 2020.
Citations (26)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now