Domain RICO: Object Detection Incremental Learning

• Domain RICO (D-RICO) is a benchmark for domain-incremental learning in object detection, training models across 15 sequential tasks drawn from diverse real and synthetic datasets.
• It isolates realistic domain shifts—such as sensor types, viewpoints, and weather conditions—by keeping a fixed label set of person, bicycle, and vehicle.
• The benchmark employs COCO-style mAP and additional metrics like forgetting and forward transfer to highlight challenges in continual learning with single-model adaptations.

Searching arXiv for the main D-RICO paper and related benchmark context. Domain RICO (D-RICO) is a benchmark for domain-incremental learning in object detection in which a detector is trained over a long sequence of domain shifts while the label space is held fixed to three high-level classes: person, bicycle, and vehicle. Introduced in “RICO: Two Realistic Benchmarks and an In-Depth Analysis for Incremental Learning in Object Detection” (Neuwirth-Trapp et al., 19 Aug 2025), D-RICO is constructed from 15 sequential tasks drawn from 14 diverse real-world and synthetic datasets, and is intended to isolate the effects of domain change—sensor modality, viewpoint, weather, lighting, and annotation policy—without the confound of an expanding class vocabulary.

1. Definition and scope

D-RICO addresses domain-incremental learning (DIL) for object detection. Let $T_t$ be the $t$-th task with domain data $D_t = (X_t, Y_t)$, where $X_t = \{x_i^t\}_{i=1}^{n_t}$ are $n_t$ images and $Y_t = \{y_i^t\}_{i=1}^{n_t}$ are their label sets. Each label is defined as $y_i^t = \{(c_{ji}^t, b_{ji}^t)\}_{j=1}^{m_i^t}$, with class $c_{ji}^t \in C$ and bounding box $b_{ji}^t \in \mathbb{R}^4$, where the fixed class set is $$C=\{\text{person}, \text{bicycle}, \text{vehicle}\}$$. A model $t$0 trains sequentially on tasks $t$1; the model knows the task ID during training but not at inference (Neuwirth-Trapp et al., 19 Aug 2025).

The benchmark is designed to stress test incremental detectors under realistic domain shifts. The shifts explicitly include camera type such as fisheye, event, and drone; modality such as RGB, thermal, and fused RGB-thermal; scene context such as urban versus rural; weather such as clear versus fog, rain, or snow; time of day such as daylight versus nighttime; and simulation versus real data. The benchmark also preserves annotation heterogeneity, including differences in bounding-box tightness, amodal versus visible boxes, and small-object labeling policies.

Within the paper’s taxonomy, D-RICO is the domain-shift counterpart to Expanding-Classes RICO (EC-RICO). Its defining simplification is the fixed label space, which isolates continual adaptation to new domains rather than continual acquisition of new object categories.

2. Dataset composition and domain taxonomy

D-RICO comprises 15 tasks, each containing exactly 2,782 training, 423 validation, and 1,399 test images, for a total of 4,604 images per task. The full benchmark therefore contains $t$2 images and approximately 871,000 bounding-box annotations (Neuwirth-Trapp et al., 19 Aug 2025).

The 15 tasks are instantiated from 14 source datasets and cover a deliberately heterogeneous range of domain conditions. The task inventory is as follows.

Task	Domain name	Dataset
1	daytime (urban RGB)	nuImages
2	thermal	FLIR Thermal
3	fisheye (fix mount)	FishEye8K
4	drone (bird’s-eye)	VisDrone
5	synthetic (CARLA)	SHIFT
6	fisheye (car-mount)	WoodScape
7	RGB+thermal fusion	SMOD
8	video game (GTA V)	Sim10k
9	nighttime	BDD100K
10	fisheye (indoor)	LOAF
11	gated camera	DENSE
12	photorealistic synthetic	Synscapes
13	thermal fisheye indoor	TIMo
14	inclement weather	DENSE (fog/rain)
15	event camera	DSEC

Across all domains, the three-class taxonomy is fixed. In the benchmark definition, person denotes any human not on a bicycle or vehicle; bicycle denotes bicycle plus rider; and vehicle denotes all other wheeled vehicles, including car, truck, bus, and motorcycle without rider.

A central feature of D-RICO is that the domains differ not only in appearance but also in acquisition geometry and label semantics. Sensor modalities range from RGB and thermal to event-based and synthetic rendering. Viewing perspectives include car-mounted fisheye, stationary fisheye, drone’s-eye, and overhead surveillance. Environmental conditions vary between daytime and nighttime and between clear and adverse weather. Annotation policies vary as well. This makes D-RICO more than a photometric domain-shift benchmark; it is a benchmark for sequential adaptation under coupled covariate and annotation shifts.

3. Protocol and benchmark construction

The benchmark protocol fixes the number of tasks at $t$3. Task order is sampled with constraints: daytime is always first, back-to-back tasks from the same source are avoided, and consecutive tasks with similar sensors are avoided. The stated goal is to obtain a reasonably varied sequence (Neuwirth-Trapp et al., 19 Aug 2025).

At incremental step $t$4, the learner trains on task $t$5 only; previous tasks are not revisited except through replay. After learning $t$6, evaluation is performed on the test sets of tasks $t$7. The data splits for each task are scene-consistent, meaning that entire scene sequences remain within a single split. This detail matters because it limits leakage between training, validation, and testing in video-like or temporally correlated datasets.

The benchmark is positioned against earlier domain-incremental object-detection evaluations that use no more than 10 tasks and often a single dataset split. By contrast, D-RICO integrates 14 distinct datasets into a 15-step sequence. The paper’s stated objective is to move away from synthetic or simplified incremental-learning evaluations and toward conditions more representative of deployment, where the label schema may stay stable while sensors, environments, and acquisition policies change.

A plausible implication is that D-RICO tests several failure modes simultaneously: catastrophic forgetting of earlier domains, insufficient plasticity on newly introduced domains, and brittleness to task-order effects. The benchmark design does not remove these interactions; it makes them explicit.

4. Evaluation metrics

D-RICO adopts COCO-style mean Average Precision, $t$8, as the base detection measure and defines four incremental-learning metrics on top of it (Neuwirth-Trapp et al., 19 Aug 2025).

The first is the average performance after $t$9 tasks:

$D_t = (X_t, Y_t)$0

where $D_t = (X_t, Y_t)$1 is the performance on task $D_t = (X_t, Y_t)$2’s test set after training through $D_t = (X_t, Y_t)$3.

The second is the forgetting measure at step $D_t = (X_t, Y_t)$4:

$D_t = (X_t, Y_t)$5

A larger $D_t = (X_t, Y_t)$6 indicates greater catastrophic forgetting of earlier tasks.

The third is forward transfer after $D_t = (X_t, Y_t)$7 tasks:

$D_t = (X_t, Y_t)$8

where $D_t = (X_t, Y_t)$9 is the individual reference mAP when $X_t = \{x_i^t\}_{i=1}^{n_t}$0 is trained alone. Positive $X_t = \{x_i^t\}_{i=1}^{n_t}$1 means better new-task adaptation than isolated training.

The fourth is the intransigence measure after $X_t = \{x_i^t\}_{i=1}^{n_t}$2 tasks:

$X_t = \{x_i^t\}_{i=1}^{n_t}$3

where $X_t = \{x_i^t\}_{i=1}^{n_t}$4 is the performance obtained if all tasks $X_t = \{x_i^t\}_{i=1}^{n_t}$5 are trained jointly. In the benchmark description, positive $X_t = \{x_i^t\}_{i=1}^{n_t}$6 implies that the incremental learner approaches joint-training performance.

These metrics separate retention from adaptation. $X_t = \{x_i^t\}_{i=1}^{n_t}$7 summarizes cumulative accuracy; $X_t = \{x_i^t\}_{i=1}^{n_t}$8 quantifies forgetting; $X_t = \{x_i^t\}_{i=1}^{n_t}$9 measures whether prior experience helps the next task; and $n_t$0 situates incremental performance relative to a joint-training reference. In D-RICO, this separation is important because methods can be highly stable yet insufficiently plastic, or adaptable yet severely forgetful.

5. Experimental setup and baselines

The benchmark experiments use an EVA-02-L Vision Transformer backbone pretrained on Objects365 and MS COCO and frozen during incremental training, combined with a Cascade Faster R-CNN detection head. The implementation is a customized Detectron2 system with incremental-learning extensions, specifically a task sampler, IL evaluator, event storage, and replay buffer (Neuwirth-Trapp et al., 19 Aug 2025).

For training, each task is run for 700 iterations with batch size 20 and input resolution $n_t$1, using standard detection augmentations. Optimization uses AdamW with a cosine learning-rate schedule and 0.1 warm-up. Classification and regression loss are disabled for absent classes through loss masking.

The evaluated methods include Joint Training, described as an upper bound within the single-model constraint; Individual Training, described as unattainable in IL but used as a reference; Naïve Fine-Tuning; Replay with 1%, 10%, and 25% of past images; and four established continual-detection approaches: ABR, Meta-ILOD, BPF, and LDB.

This setup is significant for two reasons. First, the frozen backbone reduces one source of adaptation capacity and shifts the burden of continual learning toward the detector head and the method-specific anti-forgetting mechanism. Second, the inclusion of both Joint and Individual Training makes two distinct ceilings visible: the best one can expect from a single model trained across all domains, and the better performance attainable when each domain receives its own dedicated training run.

6. Empirical results, failure modes, and interpretation

After learning all 15 tasks, the reported final results are:

Method	$n_t$2	$n_t$3	$n_t$4
Joint Training	43.75±0.03	–	–
Individual	49.37±0.13	–	–
Naïve FT	30.60±0.35	19.43±0.30	-0.63±0.08
Replay 1 %	40.20±0.19	8.91±0.26	-0.79±0.15
Replay 10 %	43.06±0.14	4.40±0.10	-2.23±0.07
Replay 25 %	43.43±0.58	3.35±0.64	-2.88±0.06
ABR	32.81±0.66	15.26±0.71	-2.46±0.05
Meta-ILOD	38.51±0.17	9.26±0.25	-3.97±0.50
BPF	30.74±1.00	19.13±1.04	-0.76±0.03
LDB	42.49±0.17	1.21±0.13	-6.17±0.06

The full table in the paper also reports $n_t$5, with Naïve FT at $n_t$6, Replay 1% at $n_t$7, Replay 10% at $n_t$8, Replay 25% at $n_t$9, ABR at $Y_t = \{y_i^t\}_{i=1}^{n_t}$0, Meta-ILOD at $Y_t = \{y_i^t\}_{i=1}^{n_t}$1, BPF at $Y_t = \{y_i^t\}_{i=1}^{n_t}$2, and LDB at $Y_t = \{y_i^t\}_{i=1}^{n_t}$3 (Neuwirth-Trapp et al., 19 Aug 2025).

Several findings structure the benchmark’s interpretation. Naïve fine-tuning catastrophically forgets, with forgetting around 19 mAP points. Simple replay, even at 1% of past images, cuts forgetting by more than 50% and increases average mAP by roughly 10 points. Replay at 25% nearly matches joint training in average mAP, with 43.43 versus 43.75. Distillation-based methods—ABR, Meta-ILOD, and BPF—underperform relative to replay and, in some cases, even relative to Naïve FT on aggregate performance. LDB attains the lowest forgetting but has negative plasticity, reflected in $Y_t = \{y_i^t\}_{i=1}^{n_t}$4.

The paper also reports that “next-task transfer” mAP indicates that $Y_t = \{y_i^t\}_{i=1}^{n_t}$5 is a poor teacher for $Y_t = \{y_i^t\}_{i=1}^{n_t}$6 at approximately 28 mAP, which is used to explain why distillation is weak in this setting. It further notes that Joint Training at $Y_t = \{y_i^t\}_{i=1}^{n_t}$7 remains 5.6 mAP points below Individual Training at 49.4, indicating that a single model does not fully reconcile all 15 domain tasks. Task order matters for Naïve FT; reversing the order can improve it by about 8 mAP points, whereas strong replay methods are largely order-invariant. The Task Affinity matrix shows asymmetry and wide variation, with TA scores around 30–90%, but is not directly predictive of IL performance.

A common assumption in continual detection is that sophisticated anti-forgetting machinery will dominate simple rehearsal. D-RICO does not support that assumption. In this benchmark, random replay is a stronger baseline than several specialized methods, yet replay still fails to close the gap to Individual Training. The paper heuristically attributes this gap to weak teachers in distillation, single models’ inability to manage diverse tasks, and insufficient plasticity. This suggests that D-RICO is not merely a harder benchmark in a scalar sense; it exposes structural limitations in current single-model continual object detection under large domain shifts.