DynamicEarth M-C-I: Open-Vocabulary Change Detection

• DynamicEarth M-C-I is a modular, training-free framework that segments change detection into mask generation, comparator, and identifier stages.
• It leverages advanced foundation models like SAM, DINOv2, and CLIP to robustly detect and classify landscape changes without additional training.
• Its modular design enables flexible component upgrades and improved cross-domain generalization in multi-temporal remote sensing applications.

DynamicEarth M-C-I (Mask-Comparator-Identifier) is a modular, training-free framework for open-vocabulary change detection (OVCD) in multi-temporal remote sensing, defined by a sequential architecture in which candidate regions (masks) are generated, changes are localized and scored, and semantic identities are assigned using open-vocabulary models. The approach enables robust detection and classification of arbitrary landscape changes across any category by decoupling change segmentation, comparison, and identification, leveraging recent advances in foundation models for vision and language (Li et al., 22 Jan 2025).

1. Fundamental Pipeline Structure and Algorithmic Details

The M-C-I (Mask-Comparator-Identifier) pipeline is instantiated as a three-stage architecture:

• M (Mask Generation): Candidate segmentation masks are automatically proposed from each input image using a foundation model such as SAM (Segment Anything Model), operating in "automatic mode" to cover all plausible objects or regions.
• C (Comparator): For each mask, latent feature representations are extracted (typically from DINOv2) and compared between the two temporal images using a normalized inner product. The change score for mask $m$ is

$$\mathcal{D}(m) = -\frac{z_1[m]}{\lVert z_1[m] \rVert_2} \cdot \frac{z_2[m]}{\lVert z_2[m] \rVert_2} \in [-1,1]$$

Changes exceeding a pre-set or learned threshold $\beta$ are retained.

• I (Identifier): Masked features are aggregated (from a model such as CLIP ViT via SegEarth-OV) and compared via cosine similarity to open-vocabulary text prompts, enabling classification beyond fixed taxonomies:

$$\mathrm{sim}(m, t) = \frac{\langle f_{\mathrm{img}}[m], f_{\mathrm{txt}}[t] \rangle}{\|f_{\mathrm{img}}[m]\|\;\|f_{\mathrm{txt}}[t]\|}$$

The maximum similarity determines the predicted semantic class for each change mask.

A variant, denoted ovcdblue, reorders these steps as Identifier → Mask → Comparator, using open-vocabulary detection to first find candidate objects, followed by mask refinement and change analysis.

This modular design abstracts mask generation, change localization, and semantic identification, allowing each component to be exchanged or updated independently. Integration with models such as Grounding-DINO, APE, and Molmo for detection, SAM2 or HQ-SAM for masking, and Florence-2 or GroundingLLM for identification illustrates this flexibility (Li et al., 22 Jan 2025).

2. Mathematical Formulation and Model Integration

Candidate mask proposals $m$ are generated per-image; for each, feature maps $Z_1$, $Z_2$ (shape $C \times H' \times W'$) are transformed to vector representations via masked average pooling:

$$\mathrm{MaskedAvgPool}(Z, m) = \frac{1}{\sum_i m_i} \sum_{i: m_i=1} Z_i \in \mathbb{R}^C$$

Masks are selected for downstream processing via non-maximum suppression (NMS) on unioned proposals from both times. The comparator computes the latent change score; masks exceeding threshold $\beta$ are classified.

The identifier employs open-vocabulary vision-LLMs, pairing masked image embeddings with class text vectors obtained from CLIP or similar architectures. Prompt ensembling (foreground-noun synonyms, background distractors) improves robustness across domains. This architecture is fully decoupled from the training data biases of supervised pixel-wise change detectors, and is inherently open-vocabulary (Li et al., 22 Jan 2025).

3. Performance Benchmarks and Quantitative Evaluation

Performance of M-C-I (ovcdred) variants has been extensively validated on major benchmarks. Representative results (IoU, F1-score, Table 1 in LaTeX from (Li et al., 22 Jan 2025)):

Method (M–C–I)	LEVIR-CD F₁	WHU-CD F₁	S2Looking F₁	BANDON F₁
SAM–DINO–SegEarth-OV	49.7	53.7	36.7	26.5
SAM–DINOv2–SegEarth-OV	53.6	57.7	38.5	30.2
SAM2–DINOv2–SegEarth-OV	50.5	58.1	37.6	30.1

Multi-class results (Table 2) for the SECOND dataset, showing per-class IoU and F1 for "Building", "Tree", "Water", etc., document robust cross-category and cross-domain generalization, outperforming supervised baselines in open-set evaluations. See the original (Li et al., 22 Jan 2025) for full tabular data.

4. Advantages, Generalization, and Modularity

Key advantages of the M-C-I paradigm:

• Training-free operation: The pipeline solely employs off-the-shelf foundation models, requiring no further dataset-specific fine-tuning for new categories.
• Open-vocabulary capability: Unconstrained by fixed class lists; any class describable in natural language can be queried, limited only by the CLIP vocabulary.
• Instance-level change detection: The mask-level granularity, with latent feature comparison, avoids pixel-wise false positives/negatives stemming from illumination, seasonal, or viewpoint variations.
• Modularity: Components (masker, comparator, identifier) are independently swappable. Upgrades (e.g., DINOv3 in place of DINOv2, temporal maskers for video) are possible without re-training the overall system.
• Cross-dataset robustness: Outperforms many supervised and unsupervised approaches on unseen target domains, due to foundation model generalization (Li et al., 22 Jan 2025).

5. Limitations and Prospects for Extension

Noted limitations include:

• Mask recall: SAM and related maskers may miss very small or occluded changes, or ambiguously delineated regions.
• Prompt sensitivity: Performance for rare or nuanced categories is limited by prompt design; manual engineering remains necessary in some cases.
• Threshold tuning: The $$\mathcal{D}(m) = -\frac{z_1[m]}{\lVert z_1[m] \rVert_2} \cdot \frac{z_2[m]}{\lVert z_2[m] \rVert_2} \in [-1,1]$$0 threshold for comparator scores must be set per domain, which requires minimal but nonzero calibration.
• Segmentation granularity: M-C-I detects changes at the instance (whole-mask) level; fine-grained part changes within existing objects may not be flagged.
• Failure propagation in ovcdblue: The I-first variant will miss changes if open-vocab detectors fail to localize the object in either temporal image.

Potential extensions include uncertainty-aware comparator scores, LLM-aided prompt generation, learnable thresholds, adapters for hyperspectral or SAR modalities, and hybrid approaches that use labeled data to refine the output via small adapters or fine-tuning. Temporal maskers (e.g., SAM2 video) could enable joint or multi-frame change sequencing (Li et al., 22 Jan 2025).

6. Relation to Other Domains and Methodological Connections

While the M-C-I method is motivated by land cover change detection in remote sensing, the architecture and foundation model wiring are domain-agnostic and can be transported to any image-based change detection scenario. The separation of candidate localization, latent change scoring, and open-vocabulary indexing is conceptually analogous to precursor work in zero-shot object detection, segmentation, and vision-language mapping.

By recasting change detection as a compositional pipeline, DynamicEarth M-C-I makes explicit the underlying data flows, scoring functions, and class assignment, facilitating rigorous performance evaluation, error analysis, and targeted improvement of each module (Li et al., 22 Jan 2025).

---

Summary Table: M-C-I Stages and Model Integration

Stage	Function	Example Model(s)
M	Mask proposal	SAM, SAM2
C	Latent change scoring	DINO, DINOv2
I	Open-vocabulary classifier	SegEarth-OV, CLIP ViT

This structure supports scalable, generalizable, and explainable open-vocabulary change detection across multi-temporal Earth observation scenarios.