DIOR-IOD Benchmark for Incremental Remote Sensing Detection

• DIOR-IOD is a remote sensing incremental object detection benchmark that reorganizes the DIOR dataset into sequential class-incremental learning stages.
• It defines two protocols (10+10 and 5+5+5+5) to simulate practical challenges such as severe scale variation, dense layouts, and unlabeled co-occurrences.
• Empirical evaluations using STAR-IOD demonstrate improved mAP metrics, underlining its value for assessing representation stability and mitigating catastrophic forgetting.

DIOR-IOD is a remote sensing incremental object detection benchmark introduced in “STAR-IOD: Scale-decoupled Topology Alignment with Pseudo-label Refinement for Remote Sensing Incremental Object Detection” (Zhang et al., 20 May 2026). Constructed from the existing DIOR dataset, it reorganizes remote sensing object detection into a sequential class-incremental regime in which classes are introduced stage by stage and, at each incremental stage, only the currently introduced classes are annotated. Its purpose is to standardize evaluation for Remote Sensing Incremental Object Detection (RS-IOD) under conditions that are characteristic of overhead imagery, notably severe intra-class scale variation, complex backgrounds, dense layouts, occlusions, and old/new class co-occurrence.

1. Definition and provenance

DIOR-IOD was introduced because, according to its authors, no standardized RS-IOD benchmark existed, previous work lacked unified dataset splits and evaluation protocols, and direct transfer of natural-image incremental detection methods was suboptimal for remote sensing scenes. The benchmark is explicitly designed to reflect a rehearsal-free setting in which data arrive in stages, old data are unavailable during later training, and current-stage images may contain unlabeled instances of previously learned categories. In that sense, DIOR-IOD is not merely “DIOR with a different split”; it is a benchmark tailored to the operational assumptions of class-incremental remote sensing detection (Zhang et al., 20 May 2026).

The benchmark is one of two datasets released in the same work, the other being DOTA-IOD. Within that paper, DIOR-IOD functions both as a benchmark contribution and as the principal empirical substrate for evaluating STAR-IOD, a Grounding DINO-based continual detector equipped with Subspace-decoupled Topology Distillation and a Clustering-driven Pseudo-label Generator. The code and dataset are released through the STAR-IOD project repository.

2. Dataset composition and incremental protocols

DIOR-IOD contains 20 classes and supports two standard protocols: a two-step setting with 10 base classes followed by 10 incremental classes, and a four-step setting with 5 classes introduced at each stage. The two-step protocol is the canonical setting; the four-step protocol is explicitly described as more challenging.

Protocol	Task structure	Classes
10 + 10	Task 1 / Task 2	Task 1: airplane, airport, bridge, service-area, toll-station, harbor, overpass, ship, trainstation, vehicle; Task 2: baseballfield, basketballcourt, chimney, dam, golffield, groundtrackfield, stadium, storagetank, tenniscourt, windmill
5 + 5 + 5 + 5	Task 1 / Task 2 / Task 3 / Task 4	Task 1: airplane, airport, bridge, service-area, toll-station; Task 2: baseball field, basketball court, golf field, chimney, dam; Task 3: ground track field, stadium, storage tank, tennis court, windmill; Task 4: harbor, overpass, ship, train station, vehicle

For the 10 + 10 protocol, the paper reports detailed per-class train and test instance counts. In Task 1, counts range from 501/509 for Trainstation to 27,351/35,186 for Ship. In Task 2, they range from 511/575 for Golffield to 4,898/7,343 for Tenniscourt. Summing the reported class counts gives approximately 69,073 training annotations and 124,445 test annotations. The paper does not report the total number of images for DIOR-IOD, does not specify a validation split, and does not provide a full image-level train/val/test partition (Zhang et al., 20 May 2026).

This organization has methodological significance. The benchmark preserves the class imbalance and multi-scale structure of the source remote sensing corpus while imposing an incremental annotation protocol, thereby combining ordinary detection difficulty with continual-learning difficulty.

3. Formal task definition and annotation regime

The RS-IOD setting on DIOR-IOD is formulated as sequential learning over disjoint class sets,

$$\mathcal{C} = \{\mathcal{C}_1, \mathcal{C}_2, \ldots, \mathcal{C}_n\},$$

with

$C_i \cap C_j = \emptyset,\ \forall i \neq j.$

At stage $t$, training uses only the current dataset

$\mathcal{D}_t = \{X_t, Y_t\},$

where $Y_t$ contains annotations only for the current class set $C_t$, even if the images contain objects from earlier stages. The learner must adapt $\mathcal{M}_{t-1}$ to $\mathcal{M}_t$ without revisiting previous task datasets and while preserving old-class performance (Zhang et al., 20 May 2026).

The paper decomposes current-stage images into three categories: Only Old, Only New, and Co-occurrence. Only images in Only New and Co-occurrence are available during incremental training, and in Co-occurrence images the old-class objects are present but unlabeled. This is a central source of forgetting, because the detector can receive background supervision on true old-class instances. On DIOR-IOD, old/new co-occurrence reaches 23.5% among the images available during incremental training, which makes missing annotations a substantive rather than incidental phenomenon.

The authors emphasize that this setting is especially difficult in remote sensing because the same category may appear at markedly different scales and because scenes often contain dense object arrangements and frequent category co-occurrence. A plausible implication is that DIOR-IOD stresses both representation stability and label-noise robustness more strongly than many natural-image class-incremental detection settings.

4. Evaluation protocol and baseline ecosystem

DIOR-IOD is evaluated with standard COCO-style metrics: $\mathit{mAP}$, $\mathit{mAP}_{50}$, $C_i \cap C_j = \emptyset,\ \forall i \neq j.$0, $C_i \cap C_j = \emptyset,\ \forall i \neq j.$1, $C_i \cap C_j = \emptyset,\ \forall i \neq j.$2, and $C_i \cap C_j = \emptyset,\ \forall i \neq j.$3. For incremental analysis, the paper additionally reports $C_i \cap C_j = \emptyset,\ \forall i \neq j.$4 for previously learned classes, $C_i \cap C_j = \emptyset,\ \forall i \neq j.$5 for current-task classes, and $C_i \cap C_j = \emptyset,\ \forall i \neq j.$6 for all classes seen so far. Forgetting is examined by comparing classwise performance before and after the incremental stage rather than through a separate closed-form forgetting metric (Zhang et al., 20 May 2026).

All compared methods are trained under a common setup: NVIDIA L20 GPU, AdamW optimizer, learning rate $C_i \cap C_j = \emptyset,\ \forall i \neq j.$7, weight decay $C_i \cap C_j = \emptyset,\ \forall i \neq j.$8, 25 epochs per task, and batch size 4. The compared methods include Fine-tuning, MD-DETR, CL-DETR, ERD, and GCD. In the authors’ implementation, STAR-IOD is architecturally grounded in Grounding DINO, and some baselines are likewise adapted to that detector family.

The benchmark is therefore not only a dataset definition but also an evaluation protocol with fixed metrics, fixed incremental schedules, and a common training recipe. That protocol is necessary because RS-IOD performance is sensitive to both catastrophic forgetting and new-class plasticity, and DIOR-IOD explicitly requires reporting both.

5. Empirical behavior on DIOR-IOD

In the two-step 10 + 10 setting, the base-stage performance for Grounding DINO-based methods is reported as $C_i \cap C_j = \emptyset,\ \forall i \neq j.$9 and $t$0. After the incremental stage, STAR-IOD achieves the best reported result on DIOR-IOD with $t$1, $t$2, $t$3, and $t$4. Against the strongest listed baseline, GCD, the gains are $t$5 on $t$6, $t$7 on $t$8, $t$9 on $\mathcal{D}_t = \{X_t, Y_t\},$0, and $\mathcal{D}_t = \{X_t, Y_t\},$1 on $\mathcal{D}_t = \{X_t, Y_t\},$2 (Zhang et al., 20 May 2026).

Setting	Method	Key result
Two-step 10 + 10	STAR-IOD	$\mathcal{D}_t = \{X_t, Y_t\},$3, $\mathcal{D}_t = \{X_t, Y_t\},$4
Two-step 10 + 10	GCD	$\mathcal{D}_t = \{X_t, Y_t\},$5, $\mathcal{D}_t = \{X_t, Y_t\},$6
Four-step 5 + 5 + 5 + 5 final stage	STAR-IOD	$\mathcal{D}_t = \{X_t, Y_t\},$7, $\mathcal{D}_t = \{X_t, Y_t\},$8
Four-step 5 + 5 + 5 + 5 final stage	GCD	$\mathcal{D}_t = \{X_t, Y_t\},$9, $Y_t$0

In the four-step 5 + 5 + 5 + 5 setting, the advantage grows at the final stage: STAR-IOD reports $Y_t$1, $Y_t$2, and $Y_t$3, compared with 34.5, 52.7, and 33.6 for GCD. The larger margin in the longer sequence suggests that DIOR-IOD becomes more discriminative as continual depth increases.

The ablations clarify what DIOR-IOD rewards. The Clustering-driven Pseudo-label Generator yields the largest old-class improvement, consistent with the importance of missing annotations. Scale partitioning through Scale-adaptive Instance Partitioning improves $Y_t$4 from 37.3 to 39.4 and yields gains of $Y_t$5, $Y_t$6, and $Y_t$7 on small, medium, and large objects, respectively. This supports the paper’s claim that intra-class scale variation is central on DIOR-IOD.

The forgetting analysis is likewise revealing. On the original Task 1 classes, Harbor drops in $Y_t$8 from 57.1 to 38.0 and Bridge from 54.5 to 43.0, whereas Toll-station remains at 82.5. The paper interprets this as evidence that initially weaker or less discriminative classes are harder to preserve than rigid, structurally distinctive ones.

6. Limitations, interpretation, and term ambiguity

The paper identifies DIOR-IOD as a benchmark for standardizing RS-IOD, but it also leaves several construction details under-specified. It does not fully document the number of images, image-level splits, any validation split, or an exact preprocessing and annotation-conversion pipeline. The benchmark is evaluated in a rehearsal-free setting only, so the comparative behavior of replay-based methods under equivalent constraints remains open. The scale thresholds used in STAR-IOD, $Y_t$9, are heuristic, and semantically similar classes remain difficult to separate (Zhang et al., 20 May 2026).

These limitations are important for interpretation. DIOR-IOD is best understood as a benchmark that foregrounds two remote-sensing-specific continual-learning pathologies: severe scale variation and unlabeled old-class instances in current-stage images. Its primary scientific value lies less in exhaustive dataset documentation than in making those pathologies measurable under fixed incremental protocols.

The term “DIOR-IOD” is also ambiguous across arXiv. In computer vision, “DIOR” can refer to a training-free conditional image embedding framework based on large vision-LLMs, but that paper explicitly “does not use the exact term ‘DIOR-IOD’” (Kawarada et al., 26 Dec 2025). In person re-identification, “DIOR” denotes “Dataset for Indoor-Outdoor Reidentification,” where “IOD” is an indoor-outdoor condition rather than a remote sensing incremental detection benchmark (Chen et al., 2023). In astrodynamics, the relevant acronym is “D-IOD,” meaning “Direct Initial Orbit Determination,” not DIOR-IOD (Chng et al., 2023). In medical imaging informatics, IODeep has been described as conceptually close to a “DIOR-IOD-like” DICOM AI-model object, but it is a research prototype rather than a formal DICOM standard (Contino et al., 2023). Within the literature surveyed here, the exact term DIOR-IOD refers specifically to the remote sensing incremental object detection benchmark introduced alongside STAR-IOD.