Category-Agnostic Change Head in OVCD

• Category-Agnostic Change Head is a change-detection adapter that localizes binary changes without relying on predefined land-cover classes.
• It combines multiscale feature modulation, window-based cross-attention, and a mixture-of-experts design to refine spatial change predictions.
• By interfacing with a frozen open-vocabulary segmentation model, CACH enables semantic indexing post-detection, achieving state-of-the-art OVCD performance.

Searching arXiv for the cited paper and closely related context papers.

The Category-Agnostic Change Head (CACH) is a change-detection adapter introduced within Seg2Change to adapt an open-vocabulary semantic segmentation model to remote-sensing change detection, specifically open-vocabulary change detection (OVCD), which integrates vision and language to detect changes across arbitrary categories (Su et al., 13 Apr 2026). In this formulation, change detection is decoupled into a binary, category-agnostic localization stage and a separate semantic indexing stage. CACH therefore detects whether a pixel has changed without assuming predefined land-cover classes, after which the changed regions are indexed to specific classes by an open-vocabulary semantic segmentation model. The design is presented together with the CA-CDD dataset and is reported to achieve state-of-the-art OVCD performance, including +9.52 IoU on WHU-CD and +5.50 mIoU on SECOND (Su et al., 13 Apr 2026).

1. Task definition and conceptual role

CACH is proposed to address a specific limitation of existing change detection methods: they are described as being limited to predefined classes in training datasets, which constrains scalability in real-world scenarios (Su et al., 13 Apr 2026). The target setting is OVCD, where the objective is not only to identify change between bi-temporal remote-sensing images but also to support arbitrary, text-driven semantic categories.

Within this setting, CACH functions as a category-agnostic binary change localizer. It is inserted as an adapter, or “change head,” on top of a DINOv2-Base backbone, while the segmentation model, identified as SegEarth-OV3, remains frozen (Su et al., 13 Apr 2026). This separation is central: CACH predicts a binary change map, and the open-vocabulary semantic segmentation system supplies class labels. A common misconception is that category-agnostic change detection discards semantics altogether. In the Seg2Change formulation, this is not the case; the category-agnostic stage localizes change, and semantic change tuples are recovered afterward by combining the binary map with per-pixel class predictions from the frozen open-vocabulary segmenter (Su et al., 13 Apr 2026).

This architecture suggests a modular interpretation of OVCD: binary transition detection can be learned independently of the full class vocabulary, while semantic specificity is delegated to a text-conditioned segmentation model. A plausible implication is improved extensibility to categories not enumerated during change-head training, because the learned CACH parameters are not tied to a fixed semantic label set.

2. Position within the Seg2Change pipeline

The end-to-end data flow is explicitly defined as

$$I^1, I^2 \rightarrow \text{DINOv2} \rightarrow \{F_i^t\} \rightarrow \text{FMM} \rightarrow \{\tilde F_i^t\} \rightarrow \text{BDFM} \rightarrow \{D_i\} \rightarrow \text{EDQA} \rightarrow \{\tilde D_i\} \rightarrow \text{MoE} \rightarrow \{X_i^d\} \rightarrow \text{ResUp} \rightarrow P_{ca} \rightarrow M_{ca}$$

where $I^1$ and $I^2$ are bi-temporal RGB images and $M_{ca}$ is the final binary change map (Su et al., 13 Apr 2026).

The pipeline begins with bi-temporal images $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$, exemplified with $H=W=512$. A pre-trained DINOv2-Base encoder with patch size $p_s=14$ and feature dimension $768$ extracts four feature maps from layers $\{2,5,8,11\}$ for each timestamp $t \in \{1,2\}$ (Su et al., 13 Apr 2026). These intermediate representations are then normalized into a multiscale pyramid by the Feature Modulation Module (FMM) and passed through progressively more change-specific operators: difference fusion, query-guided attention, expert aggregation, and residual upsampling.

The role of CACH in inference is equally explicit. First, $I^1$0 through the change head. Separately, the open-vocabulary semantic segmentation system produces semantic maps $I^1$1 and $I^1$2 from image-text inputs. Category-specific change maps are then obtained as

$I^1$3

and for binary tasks one may sum $I^1$4 (Su et al., 13 Apr 2026). This arrangement makes clear that CACH is not a replacement for the open-vocabulary segmentation model; it is an adapter that makes that model usable for change detection without additional fine-tuning of the OVSS head.

3. Architectural composition

CACH is built from four named modules—FMM, BDFM, EDQA, and ResUp—with a Mixture-of-Experts (MoE) refinement stage inserted before upsampling (Su et al., 13 Apr 2026). The following summary organizes the implementation-facing structure.

Component	Input/output role	Key specification
FMM	Converts DINOv2 features into pyramid features	$I^1$5, $I^1$6, $I^1$7, $I^1$8
BDFM	Computes difference-aware fused features	Uses absolute difference and attention mask
EDQA	Applies window-based cross-attention	Window size $I^1$9
MoE	Refines each spatial feature	$I^2$0 experts
ResUp	Recovers full-resolution prediction	Produces $I^2$1

The Feature Modulation Module projects and resizes the four temporal feature maps into a common multiscale pyramid. For layer $I^2$2, the operation is $I^2$3 convolution followed by $I^2$4 upsampling using deconv $I^2$5, stride 4. For $I^2$6, it is $I^2$7 convolution followed by $I^2$8 upsampling using deconv $I^2$9, stride 2. For $M_{ca}$0, a $M_{ca}$1 convolution is applied with no scale change. For $M_{ca}$2, a $M_{ca}$3 convolution is followed by $M_{ca}$4 downsampling using conv $M_{ca}$5, stride 2 (Su et al., 13 Apr 2026).

The Bi-temporal Difference Fusion Module computes an attention mask from the absolute difference $M_{ca}$6, reweights each temporal feature using that mask, and then fuses the two enhanced streams into a difference feature $M_{ca}$7 (Su et al., 13 Apr 2026). Architecturally, this means that difference information is not used only as a residual cue; it actively gates feature enhancement before fusion.

The Effective Difference Query Attention stage forms a guidance feature $M_{ca}$8 by concatenating $M_{ca}$9 with $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$0 and applying a $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$1 convolution. Both $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$2 and $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$3 are partitioned into non-overlapping windows of size $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$4, with $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$5, after which window-based cross-attention is computed using $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$6, $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$7, and $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$8 projections (Su et al., 13 Apr 2026). The attended timestamp-specific outputs are merged with two $I^1, I^2 \in \mathbb{R}^{H \times W \times 3}$9 convolutions to produce $H=W=512$0.

The Mixture-of-Experts stage operates per spatial location $H=W=512$1 in $H=W=512$2. A softmax gate produces weights in $H=W=512$3 with $H=W=512$4, each expert is a two-layer MLP with GELU, and the expert outputs are aggregated into $H=W=512$5 (Su et al., 13 Apr 2026). This suggests a learned specialization mechanism over heterogeneous local change patterns.

Finally, Residual Upsampler (ResUp) linearly projects each $H=W=512$6 with a $H=W=512$7 convolution, recursively combines scales by residual addition, and upsamples with bilinear interpolation until full image resolution is recovered. The final prediction is a two-channel map $H=W=512$8, and the binary map $H=W=512$9 is obtained by $p_s=14$0 (Su et al., 13 Apr 2026).

4. Mathematical formulation

The central computations of CACH are given explicitly in the Seg2Change specification (Su et al., 13 Apr 2026). At each scale $p_s=14$1, the attention mask is

$p_s=14$2

The BDFM enhancement and fusion steps are

$p_s=14$3

and

$p_s=14$4

For EDQA, the projected tensors are

$p_s=14$5

and the timestamp-specific attended feature is formed by windowed softmax attention followed by projection. The architecture summary writes

$p_s=14$6

after which the two timestamps are merged by stacked $p_s=14$7 convolutions into $p_s=14$8 (Su et al., 13 Apr 2026).

The MoE refinement is defined at each pixel $p_s=14$9 as

$768$0

$768$1

$768$2

The loss design contains three terms. Let $768$3 denote the full ResUp upsampler to $768$4, let the ground-truth change mask be $768$5, and define the inverted mask $768$6. Then

$768$7

$768$8

$768$9

$\{2,5,8,11\}$0

with $\{2,5,8,11\}$1, $\{2,5,8,11\}$2, and $\{2,5,8,11\}$3 (Su et al., 13 Apr 2026). The presence of $\{2,5,8,11\}$4 over unchanged regions indicates that feature consistency between timestamps is explicitly encouraged where no change is annotated.

5. Integration with open-vocabulary segmentation and category indexing

CACH is integrated with open-vocabulary semantic segmentation through a strict division of labor. The OVSS model outputs per-pixel logits over an open set of text-driven classes, and a softmax + argmax is applied to obtain a discrete class index per pixel (Su et al., 13 Apr 2026). CACH supplies the binary support $\{2,5,8,11\}$5, and only pixels with $\{2,5,8,11\}$6 are considered changed.

At inference, each changed pixel $\{2,5,8,11\}$7 carries class label $\{2,5,8,11\}$8 from time 1 or $\{2,5,8,11\}$9 from time 2; combining them yields a semantic change tuple $t \in \{1,2\}$0 (Su et al., 13 Apr 2026). This indexing mechanism is therefore post hoc rather than jointly decoded within the change head. A common misunderstanding is to treat CACH as a direct predictor of semantic transition labels. The specification instead defines it as a detector of changed regions whose outputs are subsequently intersected with semantic maps.

Class naming is text-conditioned. The class names are specified by prompting the OVSS model with synonyms, exemplified by “building,” “roof,” “house,” and using the highest-confidence subclass (Su et al., 13 Apr 2026). This indicates that semantic indexing depends on prompt engineering within the frozen segmentation model, whereas CACH itself remains category-agnostic.

The frozen–trainable partition is also explicit: no additional fine-tuning of the OVSS head is required, and only the CACH parameters, approximately $t \in \{1,2\}$1, are learned (Su et al., 13 Apr 2026). This suggests that the adapter is intended as a lightweight augmentation relative to retraining a full open-vocabulary segmentation stack for change detection.

6. Data regime, evaluation protocol, and reported performance

The training regime for category-agnostic change learning is built around CA-CDD, a pre-training set formed by re-annotating existing change-detection datasets into purely binary, category-agnostic masks (Su et al., 13 Apr 2026). The listed composition is:

Source split	Pairs
SECOND train	2,968
JL1-CD train	1,000
CNAM-CD train	1,000

These are described as being re-annotated into purely binary, category-agnostic change masks, yielding approximately 4,968 pairs, where all changed regions are marked regardless of land-cover type (Su et al., 13 Apr 2026). This construction is significant because it aligns the supervision signal with the design of CACH: binary localization is learned independently of semantic taxonomy.

The evaluation benchmarks span several change-detection subdomains: building CD with WHU-CD [660 test] and LEVIR-CD [512]; land-cover CD with DSIFN [48] and CLCD [120]; and semantic CD with SC-SCD [322] and SECOND [1,694] (Su et al., 13 Apr 2026). The metrics listed are Precision, Recall, F1, IoU, Overall Accuracy, and Kappa, with per-class definitions for $t \in \{1,2\}$2, $t \in \{1,2\}$3, $t \in \{1,2\}$4, and $t \in \{1,2\}$5, and the stated Kappa formulation (Su et al., 13 Apr 2026).

Implementation is reported in PyTorch on a single NVIDIA RTX 4090. Training uses $t \in \{1,2\}$6 input crops, batch size 4, 20 epochs, learning rate $t \in \{1,2\}$7, and the AdamW optimizer (Su et al., 13 Apr 2026). The inference pipeline is emphasized as requiring no thresholds to tune: first infer $t \in \{1,2\}$8 with CACH, then infer semantic maps $t \in \{1,2\}$9 and $I^1$00 with SegEarth-OV3, and finally compute $I^1$01 and $I^1$02 (Su et al., 13 Apr 2026).

The reported aggregate result is that Seg2Change, using CACH as its change adapter, achieves state-of-the-art OVCD performance, including +9.52 IoU on WHU-CD and +5.50 mIoU on SECOND (Su et al., 13 Apr 2026). Because these gains are reported for the full framework rather than an isolated ablation of CACH in the provided summary, the most conservative interpretation is that CACH is a principal enabling component of the observed OVCD performance rather than the sole source of the improvement.

7. Technical significance and interpretive considerations

CACH formalizes a specific answer to the problem of transferring open-vocabulary segmentation models to change detection: instead of forcing a segmentation model to directly infer semantic transitions, it inserts a compact, trainable head that predicts a binary change prior and then uses the frozen segmenter for semantic attribution (Su et al., 13 Apr 2026). In this sense, the method is architecturally hybrid: it combines dense feature differencing, local cross-attention, expert routing, and multiscale upsampling with language-conditioned segmentation.

Several design choices are notable. First, the head is category-agnostic at the supervision level, which matches the CA-CDD binary re-annotation strategy. Second, the use of window-based cross-attention in EDQA constrains attention to local neighborhoods of size $I^1$03, which suggests an emphasis on localized temporal correspondence rather than global token interaction. Third, the MoE stage introduces adaptive per-pixel computation over four experts, implying that different spatial regions may require distinct nonlinear refinements before decoding.

The method also clarifies what category-agnosticity does and does not mean. It does mean that the learned change detector is not trained on predefined semantic categories. It does not mean that the final system lacks semantic outputs; those arise from indexing changed pixels with the open-vocabulary segmentation predictions and combining labels across timestamps into semantic change tuples $I^1$04 (Su et al., 13 Apr 2026).

A plausible implication is that CACH provides an interface layer between foundation-model-style segmentation backbones and remote-sensing change detection benchmarks, allowing open-vocabulary semantics to be incorporated without retraining the segmentation model itself. Within Seg2Change, that interface is implemented by a 3.9M-parameter adapter whose training objective, architecture, and inference path are specified in sufficient detail for reproduction from scratch (Su et al., 13 Apr 2026).