Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SatDM: Synthesizing Realistic Satellite Image with Semantic Layout Conditioning using Diffusion Models (2309.16812v1)

Published 28 Sep 2023 in cs.CV and eess.IV

Abstract: Deep learning models in the Earth Observation domain heavily rely on the availability of large-scale accurately labeled satellite imagery. However, obtaining and labeling satellite imagery is a resource-intensive endeavor. While generative models offer a promising solution to address data scarcity, their potential remains underexplored. Recently, Denoising Diffusion Probabilistic Models (DDPMs) have demonstrated significant promise in synthesizing realistic images from semantic layouts. In this paper, a conditional DDPM model capable of taking a semantic map and generating high-quality, diverse, and correspondingly accurate satellite images is implemented. Additionally, a comprehensive illustration of the optimization dynamics is provided. The proposed methodology integrates cutting-edge techniques such as variance learning, classifier-free guidance, and improved noise scheduling. The denoising network architecture is further complemented by the incorporation of adaptive normalization and self-attention mechanisms, enhancing the model's capabilities. The effectiveness of our proposed model is validated using a meticulously labeled dataset introduced within the context of this study. Validation encompasses both algorithmic methods such as Frechet Inception Distance (FID) and Intersection over Union (IoU), as well as a human opinion study. Our findings indicate that the generated samples exhibit minimal deviation from real ones, opening doors for practical applications such as data augmentation. We look forward to further explorations of DDPMs in a wider variety of settings and data modalities. An open-source reference implementation of the algorithm and a link to the benchmarked dataset are provided at https://github.com/obaghirli/syn10-diffusion.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Generative adversarial networks, 2014.
  2. Conditional generative adversarial nets. ArXiv, abs/1411.1784, 2014.
  3. Image-to-image translation with conditional adversarial networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5967–5976, 2016.
  4. High-resolution image synthesis and semantic manipulation with conditional gans. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8798–8807, 2017.
  5. cgans with projection discriminator. ArXiv, abs/1802.05637, 2018.
  6. A style-based generator architecture for generative adversarial networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4396–4405, 2018.
  7. Semantic image synthesis with spatially-adaptive normalization. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2332–2341, 2019.
  8. Large scale gan training for high fidelity natural image synthesis. ArXiv, abs/1809.11096, 2018.
  9. Improved techniques for training gans. ArXiv, abs/1606.03498, 2016.
  10. Wasserstein gan. ArXiv, abs/1701.07875, 2017.
  11. Progressive growing of gans for improved quality, stability, and variation. ArXiv, abs/1710.10196, 2017.
  12. The gan landscape: Losses, architectures, regularization, and normalization. ArXiv, abs/1807.04720, 2018.
  13. Gan dissection: Visualizing and understanding generative adversarial networks. ArXiv, abs/1811.10597, 2018.
  14. Diffusion models beat gans on image synthesis. ArXiv, abs/2105.05233, 2021.
  15. A note on the evaluation of generative models. CoRR, abs/1511.01844, 2015.
  16. Bayesian learning via stochastic gradient langevin dynamics. In International Conference on Machine Learning, 2011.
  17. Deep unsupervised learning using nonequilibrium thermodynamics. ArXiv, abs/1503.03585, 2015.
  18. Generative modeling by estimating gradients of the data distribution. In Neural Information Processing Systems, 2019.
  19. Denoising diffusion probabilistic models. ArXiv, abs/2006.11239, 2020.
  20. Improved denoising diffusion probabilistic models. ArXiv, abs/2102.09672, 2021.
  21. Denoising diffusion implicit models. ArXiv, abs/2010.02502, 2020.
  22. Progressive distillation for fast sampling of diffusion models, 2022.
  23. Cascaded diffusion models for high fidelity image generation. J. Mach. Learn. Res., 23:47:1–47:33, 2021.
  24. Jonathan Ho. Classifier-free diffusion guidance. ArXiv, abs/2207.12598, 2022.
  25. High-resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, 2021.
  26. Adding conditional control to text-to-image diffusion models. ArXiv, abs/2302.05543, 2023.
  27. Generative adversarial networks for realistic synthesis of hyperspectral samples. IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, pages 4359–4362, 2018.
  28. Realistic river image synthesis using deep generative adversarial networks. In Frontiers in Water, 2020.
  29. Generating synthetic multispectral satellite imagery from sentinel-2. ArXiv, abs/2012.03108, 2020.
  30. Gan generation of synthetic multispectral satellite images. In Remote Sensing, 2020.
  31. Self-attending task generative adversarial network for realistic satellite image creation. 2022 IEEE Aerospace Conference (AERO), pages 1–9, 2021.
  32. Geogan: A conditional gan with reconstruction and style loss to generate standard layer of maps from satellite images. ArXiv, abs/1902.05611, 2019.
  33. Image to image translation : Generating maps from satellite images. ArXiv, abs/2105.09253, 2021.
  34. Satgan: Satellite image generation using conditional adversarial networks. 2021 International Conference on Communication information and Computing Technology (ICCICT), pages 1–6, 2021.
  35. Conditional progressive generative adversarial network for satellite image generation. ArXiv, abs/2211.15303, 2022.
  36. Sar-to-optical image translation based on conditional generative adversarial networks - optimization, opportunities and limits. Remote. Sens., 11:2067, 2019.
  37. Rareplanes: Synthetic data takes flight. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 207–217, 2020.
  38. Enhancing remote sensing image super-resolution with efficient hybrid conditional diffusion model. Remote. Sens., 15:3452, 2023.
  39. Cloud removal in remote sensing using sequential-based diffusion models. Remote. Sens., 15:2861, 2023.
  40. Ddrf: Denoising diffusion model for remote sensing image fusion. ArXiv, abs/2304.04774, 2023.
  41. Generate your own scotland: Satellite image generation conditioned on maps, 2023.
  42. Semantic image synthesis via diffusion models. ArXiv, abs/2207.00050, 2022.
  43. Non-local neural networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7794–7803, 2017.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now