Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Detecting Out-Of-Distribution Earth Observation Images with Diffusion Models (2404.12667v1)

Published 19 Apr 2024 in cs.CV, cs.AI, and cs.LG

Abstract: Earth Observation imagery can capture rare and unusual events, such as disasters and major landscape changes, whose visual appearance contrasts with the usual observations. Deep models trained on common remote sensing data will output drastically different features for these out-of-distribution samples, compared to those closer to their training dataset. Detecting them could therefore help anticipate changes in the observations, either geographical or environmental. In this work, we show that the reconstruction error of diffusion models can effectively serve as unsupervised out-of-distribution detectors for remote sensing images, using them as a plausibility score. Moreover, we introduce ODEED, a novel reconstruction-based scorer using the probability-flow ODE of diffusion models. We validate it experimentally on SpaceNet 8 with various scenarios, such as classical OOD detection with geographical shift and near-OOD setups: pre/post-flood and non-flooded/flooded image recognition. We show that our ODEED scorer significantly outperforms other diffusion-based and discriminative baselines on the more challenging near-OOD scenarios of flood image detection, where OOD images are close to the distribution tail. We aim to pave the way towards better use of generative models for anomaly detection in remote sensing.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Segdiff: Image segmentation with diffusion probabilistic models. arXiv e-prints, pages arXiv–2112, 2021.
  2. Brian D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313–326, 1982.
  3. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981–17993, 2021.
  4. Satdm: Synthesizing realistic satellite image with semantic layout conditioning using diffusion models. arXiv preprint arXiv:2309.16812, 2023.
  5. Ddpm-cd: Remote sensing change detection using denoising diffusion probabilistic models. arXiv preprint arXiv:2206.11892, 2022.
  6. Dynamical regimes of diffusion models. arXiv preprint arXiv:2402.18491, 2024.
  7. Align your latents: High-resolution video synthesis with latent diffusion models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  8. Posterior network: Uncertainty estimation without ood samples via density-based pseudo-counts. Advances in Neural Information Processing Systems, 33:1356–1367, 2020.
  9. L-C Chen. Rethinking atrous convolution for semantic image segmentation. Computing Research Repository, 2017.
  10. Hybrid DNN-Dirichlet Anomaly Detection and Ranking: Case of Burned Areas Discovery. 60:1–16.
  11. Improving reconstruction autoencoder out-of-distribution detection with mahalanobis distance. CoRR, abs/1812.02765, 2018.
  12. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780–8794, 2021.
  13. Generate your own scotland: Satellite image generation conditioned on maps. NeurIPS 2023 Workshop on Diffusion Models, 2023.
  14. Sentinel-2 cloud mask catalogue, 2020.
  15. Diffguard: Semantic mismatch-guided out-of-distribution detection using pre-trained diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1579–1589, 2023.
  16. Out-of-distribution detection in satellite image classification. In RobustML Workshop at ICLR 2021, pages 1–5. ICRL, 2021.
  17. Likelihood-based out-of-distribution detection with denoising diffusion probabilistic models. BMVC, 2023.
  18. Denoising diffusion models for out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 2948–2957, 2023.
  19. Enhancing remote sensing image super-resolution with efficient hybrid conditional diffusion model. Remote Sensing, 15(13), 2023.
  20. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  21. Video diffusion models. arXiv:2204.03458, 2022.
  22. Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(24):695–709, 2005.
  23. SpaceNet 8 - The Detection of Flooded Roads and Buildings. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1471–1479.
  24. To trust or not to trust a classifier. Advances in neural information processing systems, 31, 2018.
  25. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35:26565–26577, 2022.
  26. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems, 2022a.
  27. Enhancing diffusion-based image synthesis with robust classifier guidance. Transactions on Machine Learning Research, 2022b.
  28. Slime: Segment like me. In The Twelfth International Conference on Learning Representations, 2023.
  29. Diffusionsat: A generative foundation model for satellite imagery. In The Twelfth International Conference on Learning Representations, 2024.
  30. Variational diffusion models. Advances in neural information processing systems, 34:21696–21707, 2021.
  31. Specgrad: Diffusion probabilistic model based neural vocoder with adaptive noise spectral shaping. 2022.
  32. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761, 2020.
  33. Training confidence-calibrated classifiers for detecting out-of-distribution samples. In International Conference on Learning Representations, 2018.
  34. Diffusion-lm improves controllable text generation. Advances in Neural Information Processing Systems, 35:4328–4343, 2022.
  35. Diffusion model with detail complement for super-resolution of remote sensing. Remote Sensing, 14(19), 2022.
  36. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464–21475, 2020.
  37. Unsupervised out-of-distribution detection with diffusion inpainting. In Proceedings of the 40th International Conference on Machine Learning. JMLR.org, 2023.
  38. Entropic out-of-distribution detection. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2021.
  39. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018.
  40. Self-Supervised Learning for Generalizable Out-of-Distribution Detection. 34(04):5216–5223.
  41. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427–436, 2015.
  42. Spontaneous symmetry breaking in generative diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
  43. High-resolution image synthesis with latent diffusion models. 2021.
  44. Crash: Raw audio score-based generative modeling for controllable high-resolution drum sound synthesis. arXiv preprint arXiv:2106.07431, 2021.
  45. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487, 2022.
  46. Diffusion models for earth observation use-cases: from cloud removal to urban change detection. ArXiv, abs/2402.06684, 2023.
  47. Rsdiff: Remote sensing image generation from text using diffusion model. arXiv preprint arXiv:2309.02455, 2023.
  48. SSD: A Unified Framework for Self-Supervised Outlier Detection.
  49. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, pages 2256–2265, Lille, France, 2015. PMLR.
  50. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
  51. Generative modeling by estimating gradients of the data distribution, 2020.
  52. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
  53. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning, pages 20827–20840. PMLR, 2022.
  54. Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23(7):1661–1674, 2011.
  55. Vim: Out-of-distribution with virtual-logit matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4921–4930, 2022.
  56. Protein structure generation via folding diffusion. Nature Communications, 15(1):1059, 2024.
  57. Ediffsr: An efficient diffusion probabilistic model for remote sensing image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 2023.
  58. Generalized Out-of-Distribution Detection: A Survey.
  59. Se (3) diffusion model with application to protein backbone generation. arXiv preprint arXiv:2302.02277, 2023.
  60. Understanding Failures in Out-of-Distribution Detection with Deep Generative Models. In Proceedings of the 38th International Conference on Machine Learning, pages 12427–12436. PMLR.
  61. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3836–3847, 2023.
  62. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018.
  63. Yibo Zhou. Rethinking reconstruction autoencoder-based out-of-distribution detection. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7369–7377, 2022.
  64. Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources. 5(4):8–36.
  65. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International Conference on Learning Representations, 2018.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now