---
title: EC-RICO Incremental Object Detection Benchmark
url: https://www.emergentmind.com/topics/expanding-classes-rico-ec-rico
type: topic
---

# EC-RICO Incremental Object Detection Benchmark

Searching arXiv for the EC-RICO benchmark paper and baseline incremental object detection methods.
Expanding-Classes RICO (EC-RICO) is a realistic incremental object detection benchmark introduced in “RICO: Two Realistic Benchmarks and an In-Depth Analysis for Incremental Learning in Object Detection” [2508.13878]. It models a continual-learning setting in which each incremental learning step introduces both a new domain and a new object class. The benchmark is designed to evaluate object detectors under coupled distribution shift and label-space expansion, with an emphasis on the stability–plasticity trade-off: retaining performance on previously learned classes while adapting to newly introduced classes in new domains. In the formulation reported for EC-RICO, the benchmark comprises eight tasks drawn from distinct automotive or surveillance datasets, each standardized through a common preprocessing and evaluation protocol [2508.13878].

## 1. Concept and problem setting

EC-RICO simulates “a realistic continual-learning scenario in which, at each step, a new domain and a new object class are introduced” [2508.13878]. This distinguishes it from evaluations that isolate either domain shift or class expansion. The benchmark therefore targets a more demanding regime than fixed-class domain-incremental evaluation, because the detector must simultaneously absorb a new category and adapt to a new visual distribution.

At step $t$ with $t=1,\dots,8$, EC-RICO introduces one new object class alongside all previously seen classes. The training label set is defined as
$$
C_t = \{\text{classes introduced in } 1\ldots t\},
$$
and “all objects of future classes remain unlabeled (‘background’)” [2508.13878]. This labeling rule is central to the benchmark’s difficulty: future categories are present in images before they are officially introduced, but are treated as background until their designated step. A plausible implication is that EC-RICO stresses both open-world ambiguity and annotation incompleteness in a sequential setting.

The benchmark is positioned within incremental learning for object detection, where “Incremental Learning (IL) trains models sequentially on new data without full retraining” and must balance “adaptability to new data with retention of old knowledge” [2508.13878]. EC-RICO operationalizes this balance under real-world heterogeneity rather than synthetic simplifications.

## 2. Dataset composition and task sequence

Each EC-RICO task is drawn from “a distinct automotive or surveillance dataset,” and “all images are reformatted to RGB (3 channels) at 1536×1536 and annotated with axis-aligned bounding boxes for the classes in that step” [2508.13878]. The eight tasks span real and synthetic domains, multiple sensor modalities, multiple viewpoints, and heterogeneous annotation styles.

| Step | Dataset | New class |
|---|---|---|
| 1 | WoodScape | person |
| 2 | DENSE gated | car |
| 3 | nuImages daytime | bicycle |
| 4 | FishEye8K | motorcycle |
| 5 | SHIFT simulation | truck |
| 6 | VisDrone drone | bus |
| 7 | FLIR thermal | traffic light |
| 8 | BDD100K night | street sign |

The domain structure is explicitly cumulative. Step 1 begins with WoodScape and the class set $\{person\}$. Step 2 expands to $\{person, car\}$ and adds the gated domain. Step 3 adds bicycle and “+{multi-camera}.” Step 4 adds motorcycle and “+{surveillance}.” Step 5 adds truck and “+{synthetic}.” Step 6 adds bus and “+{aerial}.” Step 7 adds traffic light and “+{thermal}.” Step 8 adds street sign and “+{nighttime}” [2508.13878]. This organization makes domain complexity monotone in step index, rather than keeping environmental conditions fixed while classes accumulate.

The underlying datasets and domain characteristics are as follows: WoodScape is real, fisheye RGB, vehicle-mounted, with “segmentation→tight bounding box”; DENSE gated is real, with “gated RGB & Depth,” vehicle-mounted, and “instance boxes, remove ‘group’ classes”; nuImages is real, “multi-camera 6×RGB,” vehicle-mounted, with “tight, visible-only boxes”; FishEye8K is real, fisheye RGB, surveillance, with “loose overhead boxes”; SHIFT is synthetic, “RGB (CARLA),” vehicle-mounted, with “tight boxes from simulator”; VisDrone is real, “nadir RGB (drone),” aerial, with “amodal boxes”; FLIR thermal is real, thermal, vehicle-mounted, with “visible-only boxes, merged rider+bicycle”; and BDD100K night is real, RGB, vehicle-mounted, with “tight, visible-only boxes (night)” [2508.13878].

This diversity is not incidental. The benchmark is “built from 14 diverse datasets covering real and synthetic domains, varying conditions (e.g., weather, time of day), camera sensors, perspectives, and labeling policies,” although the EC-RICO task sequence itself is summarized through eight tasks/domains [2508.13878]. This suggests that the benchmark is intended as a stress test for robustness across acquisition conditions, geometry, and annotation conventions rather than a narrowly scoped class-incremental protocol.

## 3. Preprocessing and data protocol

EC-RICO uses a harmonization pipeline to reduce purely superficial incompatibilities while preserving substantive domain differences. The preprocessing protocol includes annotation alignment, image selection and cleaning, resizing and channel formatting, and training-time augmentation [2508.13878].

Under annotation alignment, the protocol specifies: “Merge rider + bicycle boxes when IoU > 0.25 (FLIR, VisDrone),” “Recompute boxes from segmentation masks (WoodScape, Synscapes),” and “Discard group annotations and very small boxes (< 7×7 px)” [2508.13878]. These operations standardize incompatible annotation policies across source datasets without collapsing their distinct semantics entirely.

Image selection and cleaning includes “Subsample high-frequency frames (every 50th in SHIFT, every 6th in TIMo),” “Filter by mean grayscale value to remove night/day as needed,” and “Remove near-duplicate or low-visibility images” [2508.13878]. The goal is not merely resizing or relabeling, but curation of a sequence appropriate for incremental evaluation.

Resizing and channel formatting impose a common representation: “Resize/pad/crop to 1536×1536,” “Duplicate grayscale→RGB for thermal tasks,” and “Fuse RGB+thermal via IFCNN network (SMOD)” [2508.13878]. Training-only augmentation further includes “Random horizontal flip; random brightness/contrast/saturation (0.6–1.4×); random lighting jitter (±0.1),” as well as “Random scale (0.1–2×) and fixed-size crop to 1536×1536” [2508.13878].

Dataset splits are fixed per task as “Training: 3040 images,” “Validation: 511 images,” and “Testing: 1417 images” [2508.13878]. The splits preserve “scene-level integrity (no scene overlap) in a 60 / 10 / 30 ratio via Monte Carlo assignment” [2508.13878]. For incremental object detection, scene-level separation is important because near-duplicate frames can otherwise inflate apparent retention and transfer.

## 4. Evaluation methodology and metrics

EC-RICO evaluates models after every incremental step. Let $mAP_{k,j}$ denote “mAP on task $j$’s test set after learning step $k$ ($j \le k$),” let $mAP'_j$ denote the “individual model mAP trained only on $D_j$,” and let $mAP^*_j$ denote the “joint-training mAP on $D_j$ (all tasks together)” [2508.13878].

The benchmark reports four metrics. Overall Accuracy, or Average Incremental mAP, is
$$
\overline{mAP}_k = \frac{1}{k}\sum_{j=1}^k mAP_{k,j}.
$$

The Forgetting Measure is
$$
FM_k = \frac{1}{k-1}\sum_{j=1}^{k-1}\max_{1\le l\le k-1}\bigl(mAP_{l,j} - mAP_{k,j}\bigr).
$$

Forward Transfer is
$$
FWT_k = \frac{1}{k-1}\sum_{j=2}^k \bigl(mAP_{j,j}-mAP'_j\bigr).
$$

The Intransigence Measure is
$$
IM_k = \frac{1}{k}\sum_{j=1}^k\bigl(mAP_{j,j}-mAP^*_j\bigr).
$$

The reported interpretation is: “Higher $\overline{mAP}$, FWT, IM indicate better performance; lower FM indicates less forgetting” [2508.13878]. The inclusion of both $mAP'_j$ and $mAP^*_j$ baselines is significant. The individual-model baseline isolates per-task learnability without interference, while the joint-training baseline estimates the attainable performance of a single model trained on the aggregate data. A plausible implication is that EC-RICO distinguishes three distinct failure modes: catastrophic forgetting, poor forward adaptation, and capacity-limited multi-domain compromise.

The recommended experimental setup uses “≥1 GPU with 24 GB VRAM,” an “EVA-02-L (frozen) + Cascade Faster R-CNN head,” “AdamW; lr=0.001; warm-up=10% iters; cosine LR decay to 0,” “Iterations per task: 700,” and “batch size=20” [2508.13878]. The replicable evaluation protocol requires: a joint-training run to collect $mAP^*_j$; individual-task runs to collect $mAP'_j$; an incremental run over $t=1\ldots8$ with checkpointing at iteration 700; evaluation on all test sets $D_j$ for $j\le t$ after each task; computation of $\overline{mAP}_8$, $FM_8$, $FWT_8$, and $IM_8$; and repetition “3× with different seeds; report mean ± std” [2508.13878].

## 5. Baselines and benchmark operating point

EC-RICO includes several baselines and incremental learning configurations. The baseline set comprises “Naïve Finetuning (FT): sequential fine-tuning without any anti-forgetting,” “Replay {1%, 10%, 25%}: random sample from all previous data in a growing buffer,” and four named methods: ABR, Meta-ILOD, BPF, and LDB [2508.13878].

The reported configurations are specific. ABR uses “Mixup+mosaic, distillation weights $\alpha_{class}=1$, $\alpha_{box}=1$, buffer=2000 images.” Meta-ILOD uses “Warp last conv layer, lr=1e-4, feature buffer=5000/class, distill weight=1.” BPF uses “Pseudo-labels disabled (DIL), distill $\alpha_{class}=0.15$, $\alpha_{box}=0.15$, buffer=2000.” LDB “Adds task-specific bias+output layers, batch=5, training iterations=2784” [2508.13878].

These baselines correspond to methods identified in the benchmark document as Liu et al. for ABR, Joseph et al. for Meta-ILOD, Leonardis-Trapp et al. for BPF, and Song et al. for LDB [2508.13878]. Within the benchmark narrative, they represent different anti-forgetting strategies: replay, distillation, architectural adjustment, and bias-balancing. The results reported for EC-RICO indicate that increasing methodological sophistication does not necessarily translate into superior performance under the combined domain-and-class incremental regime.

Because the benchmark standardizes backbone, detector head, input resolution, and training schedule, the operating point is intentionally demanding. A plausible implication is that the frozen EVA-02-L backbone and shared detection head expose limitations that might remain hidden in lower-resolution or less heterogeneous protocols.

## 6. Empirical findings and interpretation

The principal empirical result is that “all IL methods underperform in adaptability and retention, while replaying a small amount of previous data already outperforms all methods. However, individual training on the data remains superior” [2508.13878]. The benchmark quantifies the gap between upper baselines and incremental methods.

For joint versus individual training, the reported values are “Joint $\overline{mAP}=38.46\%$ vs. Individual $=45.54\%$ ⇒ $\Delta\approx7.1\%$ (single model limitation)” [2508.13878]. Among incremental methods, “Naïve FT: $\overline{mAP}=37.51\%$, $FM=8.62\%$, $FWT=-0.23$, $IM=6.52$”; “Replay 1%: $\overline{mAP}=38.09\%$, $FM=5.83\%$, $FWT=-2.17$, $IM=4.66$”; and “Best SOTA (LDB): $\overline{mAP}=28.88\%$, $FM=2.16\%$, $FWT=-16.6$, $IM=-7.76$” [2508.13878]. The summary conclusion is explicit: “All IL methods remain below Individual baseline; simple replay outperforms all complex methods” [2508.13878].

Three EC-RICO-specific challenges are identified. First, “Weak Distillation Teachers”: the “Previous model’s next-task mAP (average) ≈23.5%; distillation cannot transfer knowledge across diverse domains” [2508.13878]. Second, a pronounced “Stability-Plasticity Trade-off,” where “Strong anti-forgetting (low FM) often coincides with low plasticity (negative FWT)” [2508.13878]. Third, a “Single-model Bottleneck”: “The fixed backbone + shared head cannot adapt to diverging domain-class combinations (Δ7% joint vs. individual)” [2508.13878].

These findings directly shape the benchmark’s stated recommendations: future incremental detectors should “enhance plasticity (beyond mere forgetting minimization),” “incorporate domain-aware modules or parameter expansion,” and “explore smarter replay selection to maximize transfer per sample” [2508.13878]. This suggests that EC-RICO functions not only as an evaluation suite but also as an argument against assessing incremental object detection primarily through forgetting-oriented metrics on simplified benchmarks.

## 7. Relation to broader benchmark design and common misunderstandings

EC-RICO is one of two RICO benchmarks introduced in the same work, the other being Domain RICO (D-RICO), which “features domain shifts with a fixed class set,” whereas EC-RICO “integrates new domains and classes per IL step” [2508.13878]. The distinction is methodologically important. D-RICO isolates adaptation under a fixed ontology; EC-RICO adds label-space growth and unlabeled future classes. Confusing the two would obscure the specific source of difficulty in reported EC-RICO results.

A common misunderstanding is to interpret low forgetting alone as evidence of superior incremental learning. EC-RICO’s results explicitly counter that reading: the best reported “SOTA” result in terms of low $FM$ is LDB with $FM=2.16\%$, yet its $\overline{mAP}=28.88\%$ and $FWT=-16.6$ are substantially worse than replay and naïve fine-tuning on overall accuracy and forward adaptation [2508.13878]. In EC-RICO, low forgetting can coincide with severe intransigence.

A second misunderstanding is to treat joint training as an upper bound that incremental learning should approach closely if forgetting is solved. EC-RICO reports a nontrivial gap even between joint and individual training, $\Delta\approx7.1\%$, attributed to “single model limitation” [2508.13878]. This indicates that some of the challenge is not sequentiality per se, but the representational burden of compressing diverging domain-class combinations into one detector.

A third misunderstanding is that sophisticated distillation frameworks should dominate replay-based baselines. EC-RICO instead reports that “simple replay outperforms all complex methods” [2508.13878]. The accompanying interpretation is heuristic: the gap is attributed to “weak teachers in distillation, single models’ inability to manage diverse tasks, and insufficient plasticity” [2508.13878]. This suggests that the benchmark primarily exposes limitations of current anti-forgetting mechanisms when domain shift is entangled with semantic expansion.

In that sense, EC-RICO occupies a specific place in incremental object detection research. It is not merely a larger benchmark, but a benchmark whose construction forces evaluation under simultaneous domain drift, class accretion, heterogeneous sensors, heterogeneous annotation styles, and partially unlabeled future semantics. Its significance lies in demonstrating that methods that appear competitive in synthetic or simplified protocols may degrade sharply once these factors are combined [2508.13878].

Source: https://www.emergentmind.com/topics/expanding-classes-rico-ec-rico