---
title: Category-Agnostic Change Head in OVCD
url: https://www.emergentmind.com/topics/category-agnostic-change-head
type: topic
---

# Category-Agnostic Change Head in OVCD

Searching arXiv for the cited paper and closely related context papers.

The **Category-Agnostic Change Head** (CACH) is a change-detection adapter introduced within **Seg2Change** to adapt an open-vocabulary semantic segmentation model to remote-sensing change detection, specifically **open-vocabulary change detection (OVCD)**, which integrates vision and language to detect changes across arbitrary categories [2604.11231]. In this formulation, change detection is decoupled into a binary, category-agnostic localization stage and a separate semantic indexing stage. CACH therefore detects whether a pixel has changed without assuming predefined land-cover classes, after which the changed regions are indexed to specific classes by an open-vocabulary semantic segmentation model. The design is presented together with the **CA-CDD** dataset and is reported to achieve state-of-the-art OVCD performance, including **+9.52 IoU on WHU-CD** and **+5.50 mIoU on SECOND** [2604.11231].

## 1. Task definition and conceptual role

CACH is proposed to address a specific limitation of existing change detection methods: they are described as being **limited to predefined classes in training datasets**, which constrains scalability in real-world scenarios [2604.11231]. The target setting is OVCD, where the objective is not only to identify change between bi-temporal remote-sensing images but also to support arbitrary, text-driven semantic categories.

Within this setting, CACH functions as a **category-agnostic binary change localizer**. It is inserted as an adapter, or **“change head,”** on top of a **DINOv2-Base** backbone, while the segmentation model, identified as **SegEarth-OV3**, remains frozen [2604.11231]. This separation is central: CACH predicts a binary change map, and the open-vocabulary semantic segmentation system supplies class labels. A common misconception is that category-agnostic change detection discards semantics altogether. In the Seg2Change formulation, this is not the case; the category-agnostic stage localizes change, and semantic change tuples are recovered afterward by combining the binary map with per-pixel class predictions from the frozen open-vocabulary segmenter [2604.11231].

This architecture suggests a modular interpretation of OVCD: binary transition detection can be learned independently of the full class vocabulary, while semantic specificity is delegated to a text-conditioned segmentation model. A plausible implication is improved extensibility to categories not enumerated during change-head training, because the learned CACH parameters are not tied to a fixed semantic label set.

## 2. Position within the Seg2Change pipeline

The end-to-end data flow is explicitly defined as

\[
I^1, I^2 \rightarrow \text{DINOv2} \rightarrow \{F_i^t\} \rightarrow \text{FMM} \rightarrow \{\tilde F_i^t\}
\rightarrow \text{BDFM} \rightarrow \{D_i\}
\rightarrow \text{EDQA} \rightarrow \{\tilde D_i\}
\rightarrow \text{MoE} \rightarrow \{X_i^d\}
\rightarrow \text{ResUp} \rightarrow P_{ca} \rightarrow M_{ca}
\]

where \(I^1\) and \(I^2\) are bi-temporal RGB images and \(M_{ca}\) is the final binary change map [2604.11231].

The pipeline begins with bi-temporal images \(I^1, I^2 \in \mathbb{R}^{H \times W \times 3}\), exemplified with \(H=W=512\). A pre-trained **DINOv2-Base encoder** with **patch size \(p_s=14\)** and **feature dimension \(768\)** extracts four feature maps from layers \(\{2,5,8,11\}\) for each timestamp \(t \in \{1,2\}\) [2604.11231]. These intermediate representations are then normalized into a multiscale pyramid by the **Feature Modulation Module (FMM)** and passed through progressively more change-specific operators: difference fusion, query-guided attention, expert aggregation, and residual upsampling.

The role of CACH in inference is equally explicit. First, \(I^1, I^2 \rightarrow M_{ca}\) through the change head. Separately, the open-vocabulary semantic segmentation system produces semantic maps \(M^1\) and \(M^2\) from image-text inputs. Category-specific change maps are then obtained as

\[
M_{ch}^1 = M_{ca} \odot M^1, \qquad M_{ch}^2 = M_{ca} \odot M^2
\]

and for binary tasks one may sum \(M_{ch} = M_{ch}^1 + M_{ch}^2\) [2604.11231]. This arrangement makes clear that CACH is not a replacement for the open-vocabulary segmentation model; it is an adapter that makes that model usable for change detection without additional fine-tuning of the OVSS head.

## 3. Architectural composition

CACH is built from four named modules—**FMM, BDFM, EDQA, and ResUp**—with a **Mixture-of-Experts (MoE)** refinement stage inserted before upsampling [2604.11231]. The following summary organizes the implementation-facing structure.

| Component | Input/output role | Key specification |
|---|---|---|
| FMM | Converts DINOv2 features into pyramid features | \(C_2=48\), \(C_5=64\), \(C_8=80\), \(C_{11}=96\) |
| BDFM | Computes difference-aware fused features | Uses absolute difference and attention mask |
| EDQA | Applies window-based cross-attention | Window size \(S_w=9\) |
| MoE | Refines each spatial feature | \(N_e=4\) experts |
| ResUp | Recovers full-resolution prediction | Produces \(P_{ca} \in \mathbb{R}^{2 \times H \times W}\) |

The **Feature Modulation Module** projects and resizes the four temporal feature maps into a common multiscale pyramid. For layer \(i=2\), the operation is **\(1\times1\) convolution followed by \(4\times\) upsampling** using **deconv \(4\times4\), stride 4**. For \(i=5\), it is **\(1\times1\) convolution followed by \(2\times\) upsampling** using **deconv \(2\times2\), stride 2**. For \(i=8\), a **\(1\times1\) convolution** is applied with no scale change. For \(i=11\), a **\(1\times1\) convolution** is followed by **\(2\times\) downsampling** using **conv \(3\times3\), stride 2** [2604.11231].

The **Bi-temporal Difference Fusion Module** computes an attention mask from the absolute difference \(|\tilde F_i^1 - \tilde F_i^2|\), reweights each temporal feature using that mask, and then fuses the two enhanced streams into a difference feature \(D_i\) [2604.11231]. Architecturally, this means that difference information is not used only as a residual cue; it actively gates feature enhancement before fusion.

The **Effective Difference Query Attention** stage forms a guidance feature \(G_i^t\) by concatenating \(D_i\) with \(\tilde F_i^t\) and applying a \(1\times1\) convolution. Both \(D_i\) and \(G_i^t\) are partitioned into **non-overlapping windows of size \(S_w \times S_w\), with \(S_w=9\)**, after which window-based cross-attention is computed using \(Q_i\), \(K_i\), and \(V_i^t\) projections [2604.11231]. The attended timestamp-specific outputs are merged with two \(3\times3\) convolutions to produce \(\tilde D_i\).

The **Mixture-of-Experts** stage operates per spatial location \(p\) in \(\tilde D_i(p)\). A softmax gate produces weights in \(\mathbb{R}^{N_e}\) with **\(N_e=4\)**, each expert is a two-layer MLP with **GELU**, and the expert outputs are aggregated into \(X_i^d(p)\) [2604.11231]. This suggests a learned specialization mechanism over heterogeneous local change patterns.

Finally, **Residual Upsampler (ResUp)** linearly projects each \(X_i^d\) with a \(1\times1\) convolution, recursively combines scales by residual addition, and upsamples with bilinear interpolation until full image resolution is recovered. The final prediction is a **two-channel map** \(P_{ca}\), and the binary map \(M_{ca}\) is obtained by \(\arg\max(P_{ca})\) [2604.11231].

## 4. Mathematical formulation

The central computations of CACH are given explicitly in the Seg2Change specification [2604.11231]. At each scale \(i\), the attention mask is

\[
Att_i = \sigma(\mathrm{Conv}_{3\times 3}(|\tilde F_i^1 - \tilde F_i^2|)).
\]

The BDFM enhancement and fusion steps are

\[
X_i^t = \mathrm{ReLU}\bigl(\mathrm{Conv}_{3\times 3}(\tilde F_i^t + Att_i \odot \tilde F_i^t)\bigr)
\]

and

\[
D_i = \mathrm{ReLU}\bigl(\mathrm{Conv}_{3\times 3}\bigl(\mathrm{ReLU}(\mathrm{Conv}_{3\times 3}([X_i^1;X_i^2])) \odot Att_i\bigr)\bigr).
\]

For EDQA, the projected tensors are

\[
Q_i = \phi_{q_i}(D_i), \qquad K_i = \phi_{k_i}(D_i), \qquad V_i^t = \phi_{v_i}^t(G_i^t),
\]

and the timestamp-specific attended feature is formed by windowed softmax attention followed by projection. The architecture summary writes

\[
A_i = \mathrm{Softmax}\left(\frac{Q_i K_i^T}{\sqrt{d_i}} + b_i\right), \qquad \tilde D_i^t = \phi_{proj}(A_i \cdot V_i^t),
\]

after which the two timestamps are merged by stacked \(3\times3\) convolutions into \(\tilde D_i\) [2604.11231].

The MoE refinement is defined at each pixel \(p\) as

\[
weight_G(p) = \mathrm{Softmax}(W_g\,\tilde D_i(p) + b_g),
\]

\[
expert_j(\tilde D_i(p)) = W_{o,j}\,\mathrm{GELU}(W_{h,j}\,\tilde D_i(p)+b_{h,j}) + b_{o,j},
\]

\[
X_i^d(p)=\sum_{j=1}^{N_e} weight_{G,j}(p)\,expert_j(\tilde D_i(p)).
\]

The loss design contains three terms. Let \(\delta_{\uparrow}\) denote the full ResUp upsampler to \(H \times W\), let the ground-truth change mask be \(y^l \in \{0,1\}^{H \times W}\), and define the inverted mask \(\tilde y^l = 1-y^l\). Then

\[
L_{cd} = \mathrm{BCE}\bigl(\delta_{\uparrow}(X_2^d, X_5^d, X_8^d, X_{11}^d),\,y^l\bigr),
\]

\[
L_{ups} = \sum_{i \in \{2,5,8,11\}} \mathrm{BCE}\bigl(\delta_{\uparrow}(X_i^d),\,y^l\bigr),
\]

\[
L_{sim} = \bigl[1-\cos\bigl(\delta_{\uparrow}(\tilde F^1),\,\delta_{\uparrow}(\tilde F^2)\bigr)\bigr]\odot \tilde y^l,
\]

\[
L_{total} = \alpha L_{cd} + \beta L_{ups} + \upsilon L_{sim},
\]

with **\(\alpha=0.8\), \(\beta=0.1\), and \(\upsilon=0.1\)** [2604.11231]. The presence of \(L_{sim}\) over unchanged regions indicates that feature consistency between timestamps is explicitly encouraged where no change is annotated.

## 5. Integration with open-vocabulary segmentation and category indexing

CACH is integrated with open-vocabulary semantic segmentation through a strict division of labor. The **OVSS model outputs per-pixel logits over an open set of text-driven classes**, and a **softmax + argmax** is applied to obtain a discrete class index per pixel [2604.11231]. CACH supplies the binary support \(M_{ca}\), and only pixels with \(M_{ca}=1\) are considered changed.

At inference, each changed pixel \(p\) carries class label \(c^1(p)\) from time 1 or \(c^2(p)\) from time 2; combining them yields a **semantic change tuple \((c^1 \rightarrow c^2)\)** [2604.11231]. This indexing mechanism is therefore post hoc rather than jointly decoded within the change head. A common misunderstanding is to treat CACH as a direct predictor of semantic transition labels. The specification instead defines it as a detector of changed regions whose outputs are subsequently intersected with semantic maps.

Class naming is text-conditioned. The class names are specified by prompting the OVSS model with **synonyms**, exemplified by **“building,” “roof,” “house,”** and using the **highest-confidence subclass** [2604.11231]. This indicates that semantic indexing depends on prompt engineering within the frozen segmentation model, whereas CACH itself remains category-agnostic.

The frozen–trainable partition is also explicit: **no additional fine-tuning of the OVSS head is required**, and **only the CACH parameters, approximately \(3.9\,\mathrm{M}\), are learned** [2604.11231]. This suggests that the adapter is intended as a lightweight augmentation relative to retraining a full open-vocabulary segmentation stack for change detection.

## 6. Data regime, evaluation protocol, and reported performance

The training regime for category-agnostic change learning is built around **CA-CDD**, a pre-training set formed by re-annotating existing change-detection datasets into purely binary, category-agnostic masks [2604.11231]. The listed composition is:

| Source split | Pairs |
|---|---:|
| SECOND train | 2,968 |
| JL1-CD train | 1,000 |
| CNAM-CD train | 1,000 |

These are described as being **re-annotated into purely binary, category-agnostic change masks**, yielding **approximately 4,968 pairs**, where all changed regions are marked **regardless of land-cover type** [2604.11231]. This construction is significant because it aligns the supervision signal with the design of CACH: binary localization is learned independently of semantic taxonomy.

The evaluation benchmarks span several change-detection subdomains: **building CD** with **WHU-CD [660 test]** and **LEVIR-CD [512]**; **land-cover CD** with **DSIFN [48]** and **CLCD [120]**; and **semantic CD** with **SC-SCD [322]** and **SECOND [1,694]** [2604.11231]. The metrics listed are **Precision**, **Recall**, **F1**, **IoU**, **Overall Accuracy**, and **Kappa**, with per-class definitions for \(Pre^c\), \(Rec^c\), \(F1^c\), and \(IoU^c\), and the stated Kappa formulation [2604.11231].

Implementation is reported in **PyTorch** on a **single NVIDIA RTX 4090**. Training uses **\(512\times512\)** input crops, **batch size 4**, **20 epochs**, **learning rate \(1\times10^{-3}\)**, and the **AdamW optimizer** [2604.11231]. The inference pipeline is emphasized as requiring **no thresholds to tune**: first infer \(M_{ca}\) with CACH, then infer semantic maps \(M^1\) and \(M^2\) with SegEarth-OV3, and finally compute \(M_{ca}\odot M^1\) and \(M_{ca}\odot M^2\) [2604.11231].

The reported aggregate result is that **Seg2Change**, using CACH as its change adapter, achieves **state-of-the-art OVCD performance**, including **+9.52 IoU on WHU-CD** and **+5.50 mIoU on SECOND** [2604.11231]. Because these gains are reported for the full framework rather than an isolated ablation of CACH in the provided summary, the most conservative interpretation is that CACH is a principal enabling component of the observed OVCD performance rather than the sole source of the improvement.

## 7. Technical significance and interpretive considerations

CACH formalizes a specific answer to the problem of transferring open-vocabulary segmentation models to change detection: instead of forcing a segmentation model to directly infer semantic transitions, it inserts a compact, trainable head that predicts a **binary change prior** and then uses the frozen segmenter for semantic attribution [2604.11231]. In this sense, the method is architecturally hybrid: it combines dense feature differencing, local cross-attention, expert routing, and multiscale upsampling with language-conditioned segmentation.

Several design choices are notable. First, the head is **category-agnostic** at the supervision level, which matches the CA-CDD binary re-annotation strategy. Second, the use of **window-based cross-attention** in EDQA constrains attention to local neighborhoods of size \(9\times9\), which suggests an emphasis on localized temporal correspondence rather than global token interaction. Third, the **MoE** stage introduces adaptive per-pixel computation over **four experts**, implying that different spatial regions may require distinct nonlinear refinements before decoding.

The method also clarifies what category-agnosticity does and does not mean. It does mean that the learned change detector is not trained on predefined semantic categories. It does not mean that the final system lacks semantic outputs; those arise from indexing changed pixels with the open-vocabulary segmentation predictions and combining labels across timestamps into semantic change tuples \((c^1 \rightarrow c^2)\) [2604.11231].

A plausible implication is that CACH provides an interface layer between foundation-model-style segmentation backbones and remote-sensing change detection benchmarks, allowing open-vocabulary semantics to be incorporated without retraining the segmentation model itself. Within Seg2Change, that interface is implemented by a **3.9M-parameter** adapter whose training objective, architecture, and inference path are specified in sufficient detail for reproduction from scratch [2604.11231].

Source: https://www.emergentmind.com/topics/category-agnostic-change-head