STAR-IOD: Scale-decoupled Topology Alignment with Pseudo-label Refinement for Remote Sensing Incremental Object Detection

Abstract: Remote sensing imagery typically arrives in the form of continuous data streams. Traditional detectors often forget previously learned categories when learning new ones; therefore, research on Remote Sensing Incremental Object Detection (RS-IOD) is of great significance. However, existing methods largely overlook the intra-class scale variations prevalent in remote sensing scenes, which undermines the effectiveness of knowledge transfer and old knowledge preservation. Moreover, RS-IOD also suffers from missing annotations, which cause the model to misclassify old-class instances as background. To address these challenges, we propose a novel framework, STAR-IOD. First, we introduce a Subspace-decoupled Topology Distillation (STD) module to transfer structural knowledge, explicitly aligning inter-class topological relationships and mitigating intra-class representation discrepancies induced by scale shifts. Furthermore, we introduce the Clustering-driven Pseudo-label Generator (CPG), a plug-and-play module that leverages K-Means clustering to dynamically identify class-specific thresholds, thereby guaranteeing an accurate distinction between true positive targets and background noise and alleviating the issue of missing annotations for old classes. We also constructed two Remote Sensing Incremental Object Detection datasets, DIOR-IOD and DOTA-IOD to facilitate research on RS-IOD. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches by 1.7% and 2.1% mAP on DIOR-IOD and DOTA-IOD, respectively, effectively alleviating catastrophic forgetting while preserving strong detection performance on both base and novel classes. The code and dataset are released at: https://github.com/zyt95579/STAR-IOD.

• The paper introduces STAR-IOD, a framework that incorporates scale-adaptive topology distillation and clustering-driven pseudo-label refinement for remote sensing incremental object detection.
• The methodology partitions intra-class features into scale-specific subspaces to mitigate catastrophic forgetting and improves pseudo-label quality via adaptive thresholding using K-means clustering.
• Empirical results on DIOR-IOD and DOTA-IOD benchmarks show improved mAP by up to 2.1%, highlighting the framework’s balanced performance across varied object scales.

STAR-IOD: A Scale-Decoupled Topology Alignment and Pseudo-Label Refinement Framework for Remote Sensing Incremental Object Detection

Introduction

Incremental Object Detection (IOD) in remote sensing seeks to enable models to continually acquire new classes from sequential data streams without catastrophic forgetting of previously learned categories. Remote sensing imagery presents unique challenges: substantial intra-class scale variation, significant inter-class similarity due to high-altitude acquisition, and substantial annotation sparsity (especially for old classes during incremental phases). Existing IOD methods, primarily developed for natural images, perform suboptimally when directly transferred to remote sensing due to their inability to address these issues. The STAR-IOD framework introduces a synergistic approach, addressing intra-class scale dependencies and the unreliability of conventional pseudo-labels in remote sensing incremental object detection (RS-IOD).

Context: Challenges and Motivation

Remote sensing objects exhibit prominent scale variation, even within a single semantic category, leading to dispersive feature distributions (Figure 1). Moreover, incremental annotation protocols label only new-class objects at each step, resulting in unlabeled old-class instances and ambiguous background supervision during training. Catastrophic forgetting is thus greatly amplified, especially for fine/coarse scale objects, and is further exacerbated by poor-quality pseudo-labels under complex backgrounds.

Figure 1: (a) and (b) depict multi-scale variations for ships and planes in remote sensing; (c) visualizes scale-dependent feature separation using t-SNE.

To address the above, the STAR-IOD framework promotes stability (retention of prior knowledge) and plasticity (efficient assimilation of novel categories) through two main components: Subspace-decoupled Topology Distillation (STD) and a Clustering-driven Pseudo-label Generator (CPG).

Methodology

Overall Framework

STAR-IOD utilizes Grounding DINO as its architectural backbone. The model integrates two principal modules—STD and CPG—into the incremental detection pipeline, seamlessly orchestrating feature space alignment and pseudo-label reliability respectively to mitigate forgetting/plasticity trade-offs.

Figure 2: High-level structure of STAR-IOD, illustrating the interplay between subspace topology alignment and adaptive pseudo-label generation.

Subspace-decoupled Topology Distillation (STD)

Standard knowledge distillation and prototype-based feature alignment in IOD aggregate intra-class features into a monolithic prototype, ignoring scale-induced variability. STAR-IOD employs Scale-adaptive Instance Partitioning (SIP) to decouple features by predicted bounding box area into small, medium, and large-scale subspaces, stratifying the topology modeling process.

Within each subspace:

• Prototype Aggregation leverages confidence-weighted means, down-weighting uncertain predictions and increasing prototype discriminativity.
• Geographical Context Injection augments semantic structure with a global background anchor, improving object-background separation.
• Topological Structure Construction models the inter-class affinity as a relation matrix (pairwise prototype distances), which is normalized via a Boltzmann transformation to yield a probabilistic affinity distribution.
• Distillation Loss is then defined as a KL divergence between teacher and student inter-class affinity distributions within each scale subspace. The total loss is dynamically scaled to balance gradients when varying the temperature parameter.

This scale-partitioned topology alignment directly targets the scale-induced drift of old-class feature clusters, yielding both semantically consistent and scale-coherent knowledge transfer.

Figure 3: STD module decouples and aligns inter-class topology in small, medium, and large feature subspaces.

Clustering-driven Pseudo-label Generator (CPG)

Fixed-threshold pseudo-labeling is ill-suited for RS-IOD due to class-dependent confidence drift, background clutter, and occlusion-induced score overlap. CPG maintains a class-specific confidence score bank, aggregates outputs over multiple batches, and applies 1D K-means clustering to the aggregated predictions. The lower-bound of the high-confidence cluster is used as an adaptive, data-driven threshold for pseudo-label generation, ensuring robust foreground-background separation across diverse categories.

A spatial de-duplication step guarantees that pseudo-labels do not overlap with annotated new-class instances, thereby refining the supervision signal and suppressing error accumulation from duplicated or misaligned proposals.

Figure 4: CPG adaptively determines category-specific thresholds via K-means clustering, enhancing pseudo-label quality for historical categories.

Experimental Evaluation

Benchmarks

To standardize evaluation and enable fair comparison, the authors release DIOR-IOD and DOTA-IOD, incremental variants of major remote sensing detection datasets. Both are partitioned to maximize co-occurrence between old and new categories across incremental phases, emphasizing the challenge of unlabeled old-class objects (Figure 5).

Figure 5: Distribution of co-occurring old–new category objects highlights the prevalence of unlabeled old-class instances in incremental data.

Main Results

Comprehensive comparisons under both two-step and multi-step protocols demonstrate that STAR-IOD achieves the highest mean AP (mAP) on both benchmarks, outperforming state-of-the-art (SOTA) methods by 1.7% (DIOR-IOD) and 2.1% (DOTA-IOD). Crucially, the superiority holds consistently over all object scales (small, medium, large), indicating both robustness to scale variance and balanced optimization.

Qualitative assessments corroborate the numerical findings: STAR-IOD significantly reduces forgetting-induced missed detections and misclassifications, especially for old-class objects under challenging scale and background variations (Figure 6).

Figure 6: STAR-IOD suppresses old-class forgetting and mitigates misclassification relative to prior SOTA on DIOR-IOD.

Further, in ablation trials, the removal of STD (scale partitioning) or the replacement of CPG with fixed-threshold or alternate adaptive methods degrades both old-class retention and overall accuracy, highlighting the indispensability of these modules.

A striking observation is the failure of generic distillation methods to preserve old knowledge under persistent annotation sparsity; by contrast, STAR-IOD's tailored approach maintains high mAP on both old and new categories, even after several incremental steps (Figure 7).

Figure 7: Visualization of detection for old (top) and new (bottom) classes on DOTA-IOD, showing reliable retention and plasticity.

Failure Modes and Theoretical Insights

Despite its performance, STAR-IOD is not immune to semantic confusion in classes with overlapping textual/visual representations, as evidenced by the severe AP collapse for the Helicopter category, which shares substantial text embedding similarity with Plane. This points to limitations in existing text encoders and emphasizes the importance of developing discriminative, context-aware embeddings for fine-grained open-vocabulary detection in RS-IOD (Figure 8).

Figure 8: Common Helicopter failure cases—misclassification, missed detection, and false positives—stem from both textual and visual ambiguity.

Implications and Future Directions

Practically, STAR-IOD provides a scalable solution for continual deployment of detection models in operational remote sensing environments, enhancing downstream tasks such as long-term surveillance, disaster monitoring, and infrastructure maintenance. The release of standardized RS-IOD splits, benchmarks, and code contributes to rigorous empirical assessment and reproducibility.

Theoretically, STAR-IOD substantiates the necessity of explicit scale-aware subspace modeling and data-driven pseudo-label refinement in the design of continual vision systems for remote sensing. The framework invites future advances in:

• Improved language-vision alignment, toward handling fine-grained class ambiguity.
• More expressive probabilistic topology modeling for long-sequence incremental detection.
• Generalized subspace partitioning strategies that adapt beyond bounding box area and leverage additional contextual features.

Conclusion

STAR-IOD establishes a new SOTA for RS-IOD, integrating scale-decoupled topology distillation with robust, clustering-based pseudo-label refinement. By explicitly addressing intra-class scale diversity and annotation sparsity, the framework demonstrates strong knowledge retention, adaptability, and balanced multi-scale performance. Opportunities for further research lie in more advanced cross-modal embedding and topology modeling, particularly under realistic, highly imbalanced, and long-horizon incremental scenarios (2605.20738).