DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection

Abstract: Remote sensing change detection is crucial for understanding the dynamics of our planet's surface, facilitating the monitoring of environmental changes, evaluating human impact, predicting future trends, and supporting decision-making. In this work, we introduce a novel approach for change detection that can leverage off-the-shelf, unlabeled remote sensing images in the training process by pre-training a Denoising Diffusion Probabilistic Model (DDPM) - a class of generative models used in image synthesis. DDPMs learn the training data distribution by gradually converting training images into a Gaussian distribution using a Markov chain. During inference (i.e., sampling), they can generate a diverse set of samples closer to the training distribution, starting from Gaussian noise, achieving state-of-the-art image synthesis results. However, in this work, our focus is not on image synthesis but on utilizing it as a pre-trained feature extractor for the downstream application of change detection. Specifically, we fine-tune a lightweight change classifier utilizing the feature representations produced by the pre-trained DDPM alongside change labels. Experiments conducted on the LEVIR-CD, WHU-CD, DSIFN-CD, and CDD datasets demonstrate that the proposed DDPM-CD method significantly outperforms the existing state-of-the-art change detection methods in terms of F1 score, IoU, and overall accuracy, highlighting the pivotal role of pre-trained DDPM as a feature extractor for downstream applications. We have made both the code and pre-trained models available at https://github.com/wgcban/ddpm-cd

• The paper introduces a two-stage approach where DDPMs are pre-trained as self-supervised feature extractors for remote sensing change detection.
• It demonstrates significant improvements in accuracy and robustness compared to state-of-the-art methods using metrics like F1-score and IoU.
• The method leverages multi-scale feature extraction from various diffusion timesteps, paving the way for broader applications of generative models.

Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection

The paper "DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection" investigates a novel approach to remote sensing change detection by utilizing pre-trained Denoising Diffusion Probabilistic Models (DDPMs) as feature extractors. Change detection in remote sensing involves analyzing images acquired at different time intervals to identify temporal changes on the Earth’s surface, which is essential for numerous applications, including environmental monitoring, disaster assessment, and urban planning.

Methodology

This research introduces a two-stage process for leveraging DDPMs in change detection. Initially, the DDPM is pre-trained on a large corpus of unlabeled remote sensing images obtained from sources like Google Earth, enabling the model to learn the underlying data distribution without relying on labeled datasets. These DDPMs, commonly utilized for their superior performance in image synthesis tasks, are adapted here as a robust self-supervised feature extractor. The model undergoes a diffusion process, which involves gradually adding and then removing Gaussian noise from the data using a Markov chain framework. The capability of DDPMs to model complex distributions allows extraction of meaningful and discriminative feature representations from inputs, which are crucial for detecting changes in remote sensing imagery.

In the second stage, a lightweight change detection classifier is fine-tuned using the features extracted from the pre-trained DDPM, in collaboration with supervised change labels. This process entails generating feature representations at multiple scales and diffusion time steps and then feeding these representations into a hierarchical change decoder. The decoder processes both pre-change and post-change images to produce a change probability map, which is then compared to the ground truth using cross-entropy loss for optimization.

Experimental Evaluation

The study demonstrates that DDPM-CD achieves superior performance over state-of-the-art change detection methods across multiple datasets, including LEVIR-CD, WHU-CD, DSIFN-CD, and CDD. The performance metrics utilized include F1-score, Intersection over Union (IoU), and overall accuracy (OA), where the DDPM-CD method consistently outperformed existing methods. The results showcase the effectiveness of leveraging pre-trained DDPMs as feature extractors, resulting in significant improvements in accuracy and robustness in detecting changes within remote sensing images.

A key finding of this study is the substantial enhancement in change detection performance observed when combining features obtained at different noise levels, revealing that feature representations sampled from various timesteps contribute to improved detector robustness and accuracy.

Implications and Future Directions

The implications of leveraging DDPMs for feature extraction in change detection are significant due to their ability to use unlabeled data effectively. This enhances the utilization of vast amounts of remote sensing data which are not labeled, overcoming one of the primary limitations of deep learning-based models that rely heavily on large annotated datasets.

Additionally, this work underscores the effectiveness of generative models beyond their conventional domain of image synthesis, opening new avenues for applying DDPMs in other areas of remote sensing and computer vision. The robustness of DDPMs in handling noisy data further enhances their application in dynamic environments where image quality might be compromised by atmospheric conditions or other perturbations.

Future developments could focus on optimizing DDPMs for more efficient implementation, considering their computational complexity. Moreover, exploring the integration of other generative models in change detection or extending this methodology to multimodal datasets could further expand its applicability.

In summary, this research presents a compelling case for the use of self-supervised DDPMs in remote sensing change detection, offering substantial practical benefits and establishing a foundation for future explorations in the domain.