Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model (2403.17460v1)

Published 26 Mar 2024 in eess.IV and cs.CV

Abstract: Reference-based super-resolution (RefSR) has the potential to build bridges across spatial and temporal resolutions of remote sensing images. However, existing RefSR methods are limited by the faithfulness of content reconstruction and the effectiveness of texture transfer in large scaling factors. Conditional diffusion models have opened up new opportunities for generating realistic high-resolution images, but effectively utilizing reference images within these models remains an area for further exploration. Furthermore, content fidelity is difficult to guarantee in areas without relevant reference information. To solve these issues, we propose a change-aware diffusion model named Ref-Diff for RefSR, using the land cover change priors to guide the denoising process explicitly. Specifically, we inject the priors into the denoising model to improve the utilization of reference information in unchanged areas and regulate the reconstruction of semantically relevant content in changed areas. With this powerful guidance, we decouple the semantics-guided denoising and reference texture-guided denoising processes to improve the model performance. Extensive experiments demonstrate the superior effectiveness and robustness of the proposed method compared with state-of-the-art RefSR methods in both quantitative and qualitative evaluations. The code and data are available at https://github.com/dongrunmin/RefDiff.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Darts: Double attention reference-based transformer for super-resolution. arXiv preprint arXiv:2307.08837, 2023.
  2. Wele Gedara Chaminda Bandara and Vishal M Patel. A transformer-based siamese network for change detection. In IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium, pages 207–210. IEEE, 2022.
  3. Reference-based image super-resolution with deformable attention transformer. In European Conference on Computer Vision, pages 325–342. Springer, 2022.
  4. Multitask learning for large-scale semantic change detection. Computer Vision and Image Understanding, 187:102783, 2019.
  5. Rrsgan: Reference-based super-resolution for remote sensing image. IEEE Transactions on Geoscience and Remote Sensing, 60:1–17, 2021.
  6. Real-world remote sensing image super-resolution via a practical degradation model and a kernel-aware network. ISPRS Journal of Photogrammetry and Remote Sensing, 191:155–170, 2022.
  7. Large-scale land cover mapping with fine-grained classes via class-aware semi-supervised semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16783–16793, 2023a.
  8. An adaptive image fusion method for sentinel-2 images and high-resolution images with long-time intervals. International Journal of Applied Earth Observation and Geoinformation, 121:103381, 2023b.
  9. Implicit diffusion models for continuous super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10021–10030, 2023.
  10. Blind super-resolution with iterative kernel correction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1604–1613, 2019.
  11. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems, 30, 2017.
  12. Cascaded diffusion models for high fidelity image generation. The Journal of Machine Learning Research, 23(1):2249–2281, 2022.
  13. Task decoupled framework for reference-based super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5931–5940, 2022.
  14. Histif: A new spatiotemporal image fusion method for high-resolution monitoring of crops at the subfield level. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13:4607–4626, 2020.
  15. Robust reference-based super-resolution via c2-matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2103–2112, 2021.
  16. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35:26565–26577, 2022.
  17. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 2012.
  18. Srdiff: Single image super-resolution with diffusion probabilistic models. Neurocomputing, 479:47–59, 2022a.
  19. Hyperspectral and panchromatic images fusion based on the dual conditional diffusion models. IEEE Transactions on Geoscience and Remote Sensing, 2023.
  20. Wavelet-based texture reformation network for image super-resolution. IEEE Transactions on Image Processing, 31:2647–2660, 2022b.
  21. Diffbir: Towards blind image restoration with generative diffusion prior. arXiv preprint arXiv:2308.15070, 2023.
  22. Super-resolution-based change detection network with stacked attention module for images with different resolutions. IEEE Transactions on Geoscience and Remote Sensing, 60:1–18, 2021.
  23. Learning token-aligned representations with multimodel transformers for different-resolution change detection. IEEE Transactions on Geoscience and Remote Sensing, 60:1–13, 2022.
  24. Masa-sr: Matching acceleration and spatial adaptation for reference-based image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6368–6377, 2021.
  25. Stair: A generic and fully-automated method to fuse multiple sources of optical satellite data to generate a high-resolution, daily and cloud-/gap-free surface reflectance product. Remote Sensing of Environment, 214:87–99, 2018.
  26. Land cover change detection with heterogeneous remote sensing images: Review, progress, and perspective. Proceedings of the IEEE, 2022.
  27. A deep learning model for incorporating temporal information in haze removal. Remote Sensing of Environment, 274:113012, 2022.
  28. Acdmsr: Accelerated conditional diffusion models for single image super-resolution. arXiv preprint arXiv:2307.00781, 2023.
  29. Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2337–2346, 2019.
  30. Cross-sensor remote sensing imagery super-resolution via an edge-guided attention-based network. ISPRS Journal of Photogrammetry and Remote Sensing, 199:226–241, 2023.
  31. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713–4726, 2022.
  32. Resdiff: Combining cnn and diffusion model for image super-resolution. arXiv preprint arXiv:2303.08714, 2023.
  33. Dynamicearthnet: Daily multi-spectral satellite dataset for semantic change segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21158–21167, 2022.
  34. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
  35. Exploiting diffusion prior for real-world image super-resolution. arXiv preprint arXiv:2305.07015, 2023.
  36. Semantic image synthesis via diffusion models. arXiv preprint arXiv:2207.00050, 2022.
  37. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 606–615, 2018.
  38. Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1905–1914, 2021.
  39. Hsr-diff: hyperspectral image super-resolution via conditional diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7083–7093, 2023.
  40. Blind super-resolution for remote sensing images via conditional stochastic normalizing flows. arXiv preprint arXiv:2210.07751, 2022.
  41. Coarse-to-fine embedded patchmatch and multi-scale dynamic aggregation for reference-based super-resolution. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2768–2776, 2022.
  42. Diffir: Efficient diffusion model for image restoration. arXiv preprint arXiv:2303.09472, 2023.
  43. Dual-diffusion: Dual conditional denoising diffusion probabilistic models for blind super-resolution reconstruction in rsis. arXiv preprint arXiv:2305.12170, 2023.
  44. Learning texture transformer network for image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5791–5800, 2020a.
  45. Semantic change detection with asymmetric siamese networks. arXiv preprint arXiv:2010.05687, 2020b.
  46. Pixel-aware stable diffusion for realistic image super-resolution and personalized stylization. arXiv preprint arXiv:2308.14469, 2023.
  47. Reference-based super-resolution method for remote sensing images with feature compression module. Remote Sensing, 15(4):1103, 2023.
  48. Making low-resolution satellite images reborn: a deep learning approach for super-resolution building extraction. Remote Sensing, 13(15):2872, 2021.
  49. Rrsr: Reciprocal reference-based image super-resolution with progressive feature alignment and selection. In European Conference on Computer Vision, pages 648–664. Springer, 2022.
  50. Swcare: Switchable learning and connectivity-aware refinement method for multi-city and diverse-scenario road mapping using remote sensing images. International Journal of Applied Earth Observation and Geoinformation, 127:103665, 2024.
  51. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018.
  52. Contrastive attention-guided multi-level feature registration for reference-based super-resolution. ACM Transactions on Multimedia Computing, Communications and Applications, 20(2):1–21, 2023.
  53. Unsupervised change detection by cross-resolution difference learning. IEEE Transactions on Geoscience and Remote Sensing, 60:1–16, 2021.
  54. Signet: A siamese graph convolutional network for multi-class urban change detection. Remote Sensing, 15(9):2464, 2023.
  55. Geometry enhanced reference-based image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6123–6132, 2023.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now