Papers
Topics
Authors
Recent
Search
2000 character limit reached

Segment Any Change

Published 2 Feb 2024 in cs.CV | (2402.01188v4)

Abstract: Visual foundation models have achieved remarkable results in zero-shot image classification and segmentation, but zero-shot change detection remains an open problem. In this paper, we propose the segment any change models (AnyChange), a new type of change detection model that supports zero-shot prediction and generalization on unseen change types and data distributions. AnyChange is built on the segment anything model (SAM) via our training-free adaptation method, bitemporal latent matching. By revealing and exploiting intra-image and inter-image semantic similarities in SAM's latent space, bitemporal latent matching endows SAM with zero-shot change detection capabilities in a training-free way. We also propose a point query mechanism to enable AnyChange's zero-shot object-centric change detection capability. We perform extensive experiments to confirm the effectiveness of AnyChange for zero-shot change detection. AnyChange sets a new record on the SECOND benchmark for unsupervised change detection, exceeding the previous SOTA by up to 4.4% F$_1$ score, and achieving comparable accuracy with negligible manual annotations (1 pixel per image) for supervised change detection. Code is available at https://github.com/Z-Zheng/pytorch-change-models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  2. Unsupervised change detection using convolutional-autoencoder multiresolution features. IEEE Transactions on Geoscience and Remote Sensing, 60:1–19, 2022.
  3. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  4. A theoretical framework for unsupervised change detection based on change vector analysis in the polar domain. IEEE Transactions on Geoscience and Remote Sensing, 45(1):218–236, 2006.
  5. Automatic analysis of the difference image for unsupervised change detection. IEEE Transactions on Geoscience and Remote sensing, 38(3):1171–1182, 2000.
  6. Using satellite imagery to understand and promote sustainable development. Science, 371(6535):eabe8628, 2021.
  7. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sensing, 12(10):1662, 2020.
  8. Adversarial instance augmentation for building change detection in remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60:1–16, 2021a.
  9. Remote sensing image change detection with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2021b.
  10. Exchange means change: An unsupervised single-temporal change detection framework based on intra-and inter-image patch exchange. ISPRS Journal of Photogrammetry and Remote Sensing, 206:87–105, 2023.
  11. Digital change detection in forest ecosystems with remote sensing imagery. Remote sensing reviews, 13(3-4):207–234, 1996.
  12. Fully convolutional siamese networks for change detection. In ICIP, pp.  4063–4067. IEEE, 2018.
  13. Adapting segment anything model for change detection in hr remote sensing images. arXiv preprint arXiv:2309.01429, 2023.
  14. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
  15. Unsupervised deep slow feature analysis for change detection in multi-temporal remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 57(12):9976–9992, 2019.
  16. xbd: A dataset for assessing building damage from satellite imagery. arXiv preprint arXiv:1911.09296, 2019.
  17. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9.
  18. Segment Anything. In ICCV, pp.  4015–4026, October 2023.
  19. Microsoft coco: Common objects in context. In ECCV, pp.  740–755. Springer, 2014.
  20. Change-aware sampling and contrastive learning for satellite images. In CVPR, pp.  5261–5270, 2023.
  21. Seasonal contrast: Unsupervised pre-training from uncurated remote sensing data. In ICCV, pp.  9414–9423, 2021.
  22. Peft: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft, 2022.
  23. Nielsen, A. A. The regularized iteratively reweighted mad method for change detection in multi-and hyperspectral data. IEEE Transactions on Image processing, 16(2):463–478, 2007.
  24. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
  25. Otsu, N. A threshold selection method from gray-level histograms. IEEE transactions on systems, man, and cybernetics, 9(1):62–66, 1979.
  26. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.  8748–8763. PMLR, 18–24 Jul 2021.
  27. Unsupervised deep change vector analysis for multiple-change detection in vhr images. IEEE Transactions on Geoscience and Remote Sensing, 57(6):3677–3693, 2019.
  28. S2looking: A satellite side-looking dataset for building change detection. Remote Sensing, 13(24):5094, 2021.
  29. A critical synthesis of remotely sensed optical image change detection techniques. Remote Sensing of Environment, 160:1–14, 2015.
  30. An empirical study of remote sensing pretraining. IEEE Transactions on Geoscience and Remote Sensing, 2022.
  31. Slow feature analysis for change detection in multispectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 52(5):2858–2874, 2013.
  32. Unsupervised change detection in multitemporal vhr images based on deep kernel pca convolutional mapping network. IEEE Transactions on Cybernetics, 52(11):12084–12098, 2021.
  33. Change detection of built-up land: A framework of combining pixel-based detection and object-based recognition. ISPRS Journal of Photogrammetry and Remote Sensing, 119:402–414, 2016.
  34. Asymmetric siamese networks for semantic change detection in aerial images. IEEE Transactions on Geoscience and Remote Sensing, 60:1–18, 2021.
  35. Using publicly available satellite imagery and deep learning to understand economic well-being in africa. Nature communications, 11(1):2583, 2020.
  36. Fast segment anything. arXiv preprint arXiv:2306.12156, 2023.
  37. Change is Everywhere: Single-temporal supervised object change detection in remote sensing imagery. In ICCV, pp.  15193–15202, 2021.
  38. ChangeMask: Deep multi-task encoder-transformer-decoder architecture for semantic change detection. ISPRS Journal of Photogrammetry and Remote Sensing, 183:228–239, 2022.
  39. Scalable multi-temporal remote sensing change data generation via simulating stochastic change process. In ICCV, pp.  21818–21827, 2023.
  40. Remote sensing of land change: A multifaceted perspective. Remote Sensing of Environment, 282:113266, 2022.
Citations (7)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 68 likes about this paper.

Sign Up for Free