DECERN: Active Learning for Fine-Grained Images

• DECERN is an active learning framework for fine-grained image classification that quantifies sample informativeness using discrepancy-confusion uncertainty and calibration diversity.
• The framework integrates local feature fusion, uncertainty-weighted k-means clustering, and dynamic thresholding to efficiently select informative samples under limited annotation budgets.
• Empirical results across multiple fine-grained datasets demonstrate that DECERN achieves competitive accuracy while maintaining balanced and diverse sample selection.

DECERN is an active learning framework for fine-grained image classification that combines discrepancy-confusion uncertainty and calibration diversity to evaluate the informativeness of unlabeled images under limited annotation budgets. It was introduced in "Combining Discrepancy-Confusion Uncertainty and Calibration Diversity for Active Fine-Grained Image Classification" (Jin et al., 29 Sep 2025). The method is designed for pool-based, batch active learning with a labeled set $$\mathcal{D}^\ell=\{(x_i^\ell, y_i^\ell)\}_{i=1}^{N_\ell}$$, an unlabeled pool $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$, and a budget $B$ images to query at each cycle. Its central claim is that, in fine-grained settings where classes differ only subtly and often in local regions, sample selection should jointly capture how unstable a representation is under class-conditioned local perturbations and how diverse the final queried batch is relative to both local cluster structure and labeled class prototypes.

1. Problem setting and motivation

DECERN is formulated for pool-based, batch active learning for classification. The backbone model is $\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$, where $\mathcal{F}_z$ is a feature encoder and $\mathcal{F}_c$ is a classifier head. The reported implementations use ResNet-50 or ViT-Small, both DINO-pretrained. The system runs for 8 active learning cycles, selecting $B = K\cdot N_c$ images per cycle with $K\in\{1,2\}$ and $N_c$ the number of classes.

The framework is motivated by the specific failure modes of active learning in fine-grained image classification. Fine-grained classes such as bird species, dog breeds, and flower types are visually very similar, and their deep features share many semantics while differing only subtly, often in local regions. Standard uncertainty-based methods such as entropy, margin, and confidence assume a clearer decision boundary than is available in this setting. Diversity-based methods such as k-means and CoreSet emphasize global spread, but they may miss subtle inter-class distinctions and can suffer when the batch size is small relative to the total number of categories. Feature-fusion active learning methods such as ALFA-Mix amplify differences by mixing features, but the paper argues that they mostly examine label variability and can miss samples whose ambiguity derives from cross-category similarity patterns rather than label instability alone (Jin et al., 29 Sep 2025).

DECERN therefore targets two requirements simultaneously. First, uncertainty should reflect how an image behaves when its representation is locally blended with class anchors, so that both category directionality and structural stability are probed. Second, diversity should preserve local representativeness while also exploring globally underrepresented regions that are far from the knowledge already captured by labeled data.

2. Multifaceted informativeness and the DECERN pipeline

DECERN defines a multifaceted informativeness measure with two components. The first is discrepancy-confusion uncertainty. The second is calibration diversity. Together they determine which unlabeled samples are queried at each active learning cycle.

Discrepancy-confusion uncertainty is built from local feature fusion between each unlabeled sample and each class anchor. The method measures how prediction probabilities change after fusion and how confused the fused prediction becomes. The paper characterizes these effects by two fine-grained properties. Category directionality measures how consistently a sample points toward a specific class under semantic-consistent perturbations. Structural stability measures how robust the sample’s semantic structure is under semantic-inconsistent perturbations. A sample is therefore not scored only by raw uncertainty, but by how its representation responds to class-conditioned local perturbations.

Calibration diversity is applied after high-uncertainty candidates are identified. DECERN first performs uncertainty-weighted k-means clustering on candidate features, with weights given by the uncertainty scores. Within each cluster, it then chooses one sample that is simultaneously close to the cluster centroid and far from all class anchors. The first criterion preserves local representativeness; the second maximizes global diversity relative to the current labeled set. The paper describes this as calibrating diversity by anchors so that selected samples are not merely nonredundant, but also globally novel in relation to already labeled prototypes (Jin et al., 29 Sep 2025).

The per-cycle procedure is fixed. The model is trained or updated on $\mathcal{D}^\ell$; class anchors are computed from labeled data; each unlabeled sample receives an instance-level uncertainty score; a dynamic threshold $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$0 filters high-uncertainty candidates; uncertainty-weighted k-means partitions those candidates into $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$1 clusters; calibration diversity selects one sample per cluster; the oracle provides labels for the selected batch; then $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$2 and $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$3 are updated and the next cycle begins.

3. Mathematical formulation

For an unlabeled sample $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$4, DECERN defines

$\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$5

For labeled data, class anchors in feature and probability space are

$\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$6

Local feature fusion is restricted to salient feature dimensions via a binary mask $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$7, obtained from gradients of the loss with respect to $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$8. If $\mathcal{D}^u=\{x_i^u\}_{i=1}^{N_u}$9 denotes the proportion of masked dimensions,

$B$0

For each class $B$1, the fusion strength is set to $B$2. The local fusion operator is

$B$3

DECERN then uses four distributions for each class anchor: the original prediction $B$4; the global theoretical mix

$B$5

the local theoretical mix

$B$6

and the actual fused prediction

$B$7

The direction-dependent weight is defined from cosine similarity:

$B$8

High similarity makes $B$9 small, which emphasizes confusion; low similarity makes $\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$0 large, which emphasizes discrepancy.

The category-level discrepancy-confusion score is

$\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$1

Here the first term is confusion uncertainty, the entropy of $\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$2, and the second is discrepancy uncertainty, the cross-entropy between a reference distribution and $\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$3.

The instance-level uncertainty aggregates across classes and references:

$\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$4

The paper notes that there is a small bracket typo in the original presentation and explicitly gives this as the intended structure (Jin et al., 29 Sep 2025).

Candidate filtering uses a dynamic threshold. If $\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$5 is the instance-level score of the $\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$6-th unlabeled sample, then with mean $\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$7 and standard deviation $\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$8,

$\mathcal{F} = \mathcal{F}_c \circ \mathcal{F}_z$9

The parameter $\mathcal{F}_z$0 is adapted dynamically based on the skewness of the score distribution, conceptually so that candidate selection changes as the model improves.

On the retained candidate set, uncertainty-weighted k-means forms $\mathcal{F}_z$1 clusters. Within each cluster $\mathcal{F}_z$2, DECERN selects

$\mathcal{F}_z$3

where $\mathcal{F}_z$4 is the cluster centroid and $\mathcal{F}_z$5 balances local representativeness against global diversity. The paper reports that moderate $\mathcal{F}_z$6, for example $\mathcal{F}_z$7, yields the best empirical performance (Jin et al., 29 Sep 2025).

4. Algorithmic loop, architecture, and experimental protocol

The algorithmic loop follows a standard active learning schedule with DECERN-specific scoring and selection stages. For each active learning cycle, the model is trained on the current labeled pool using standard cross-entropy; anchors $\mathcal{F}_z$8 and $\mathcal{F}_z$9 are computed for all classes; each unlabeled sample is processed through gradient-based masking, class-conditioned local fusion, and discrepancy-confusion scoring; uncertainty filtering retains candidates with $\mathcal{F}_c$0; weighted k-means partitions the candidates; calibration diversity chooses one representative per cluster; queried labels are added to the labeled set and removed from the unlabeled pool.

The reported implementation uses only a standard classifier head $\mathcal{F}_c$1, with no additional active-learning-specific head. Optimization uses Adam with learning rate $\mathcal{F}_c$2, cosine decay, batch size $\mathcal{F}_c$3, and standard cross-entropy over labeled data. The initial labeled data for the first cycle is obtained by random sampling, and no extra regularization beyond backbone pretraining and standard supervised training is used (Jin et al., 29 Sep 2025).

The evaluation spans 7 fine-grained datasets: Caltech101, BronzeDing, CUB-200-2011, Flowers102, Food101, OxfordIIITPet, and StanfordDogs. Caltech101 is listed as 100 object categories with train/test splits of 4,128 / 2,465. Budgets are defined by $\mathcal{F}_c$4 with $\mathcal{F}_c$5, except that Flowers102 uses only $\mathcal{F}_c$6. Combined with the two backbones, this yields 26 distinct experimental settings.

The comparison set includes Random sampling, K-Means, CoreSet, CoreGCN, ActiveFT, BALQUE, NoiseStability, and ALFA-Mix. All experiments are repeated with 5 seeds, and the reported metric is mean $\mathcal{F}_c$7 standard deviation of top-1 accuracy. The paper also reports that selection time is close to the fastest baselines and significantly faster than BALQUE and NoiseStability, despite the additional fusion computations, because the method uses simple k-means and per-sample operations rather than heavy secondary models.

5. Empirical results and ablation evidence

Across the 26 settings, DECERN is reported as best or tied-best in almost all settings after 8 active learning cycles (Jin et al., 29 Sep 2025). With ResNet-50 and budget $\mathcal{F}_c$8, reported examples include Caltech101 at 82.95% for DECERN versus 82.60% for ALFA-Mix and 81.86% for BALQUE; BronzeDing at 42.99% versus 42.35% for ALFA-Mix and 42.06% for ActiveFT; CUB at 39.15% versus 37.34% for the best baseline, K-Means; and StanfordDogs at 41.91% versus 40.49% for ALFA-Mix. With the larger budget $\mathcal{F}_c$9, gains remain visible: on CUB with ResNet-50, DECERN reaches 56.00% versus 55.30% for BALQUE and 55.26% for ALFA-Mix; on StanfordDogs with ResNet-50, DECERN reaches 53.41% versus 52.95% for ALFA-Mix.

The same pattern is reported for ViT-Small. On CUB with $B = K\cdot N_c$0, DECERN achieves 64.83% versus 63.39% for ALFA-Mix and 61.96% for BALQUE. On StanfordDogs with $B = K\cdot N_c$1, it reaches 72.07% versus 71.84% for ALFA-Mix. Time-accuracy plots further indicate near-best efficiency while maintaining superior accuracy.

The ablation study isolates both the uncertainty and diversity modules. Removing both reduces the method to random sampling. Diversity alone improves over this baseline but remains clearly below full DECERN. Using only confusion uncertainty $B = K\cdot N_c$2 or only discrepancy uncertainty $B = K\cdot N_c$3 degrades performance relative to the full discrepancy-confusion combination. Diversity ablations also show consistent losses when uncertainty-weighted clustering is replaced by equal weights, when clustering is removed, when calibration is removed by taking only cluster centroids with $B = K\cdot N_c$4, or when either the global diversity or local representativeness term is omitted. The reported interpretation is that discrepancy-confusion combination is better than single-view uncertainty, calibration diversity is better than either local or global diversity alone, and uncertainty-weighted clustering is better than unweighted clustering.

The paper also studies sampling imbalance via class distribution entropy. CoreSet, CoreGCN, ActiveFT, and NoiseStability are reported to create highly imbalanced labeled sets, especially with small budgets, which is associated with poorer performance. DECERN tends to keep better balance, which the authors attribute to combining uncertainty and calibrated diversity rather than greedily focusing on a few classes.

Qualitative analysis uses t-SNE visualizations on BronzeDing. ALFA-Mix is described as selecting many samples near ambiguous regions but sometimes missing truly discriminative ones when boundaries are blurred. NoiseStability and BALQUE often focus on tight local regions, producing local redundancy. ActiveFT matches global density but can oversample high-density easy regions. CoreGCN and CoreSet can pick outliers or non-representative points when the initial labeled distribution is poor. DECERN is reported to select samples spread across the feature manifold, covering high-uncertainty boundaries and including both prototypical points of uncertain regions and outliers far from anchors.

6. Interpretation, limitations, and related nomenclature

The authors’ interpretation of DECERN’s behavior emphasizes six mechanisms. First, uncertainty is aligned with fine-grained structure because class-anchor fusion probes whether the predicted class anchor stabilizes or destabilizes a representation and whether small shifts toward other classes induce large semantic changes. Second, gradient-based masks make local feature fusion target discriminative regions rather than the full feature vector. Third, confusion and discrepancy capture different aspects of sample value: “how unsure” and “how surprising” the fusion outcome is. Fourth, uncertainty-weighted clustering concentrates diversity sampling in informative regions rather than trivial ones. Fifth, calibration diversity balances local representativeness and global diversity from labeled prototypes. Sixth, dynamic thresholding adapts candidate selection as the uncertainty distribution changes over active learning cycles (Jin et al., 29 Sep 2025).

The reported limitations are equally specific. DECERN is evaluated only on image classification; its effectiveness on semantic segmentation or object detection remains untested. It depends on reliable class anchors, which can be fragile if the labeled set is extremely small or severely imbalanced. It also depends on backbone representations that already encode meaningful fine-grained distinctions; the reported experiments use DINO-pretrained models. Extension to non-image domains or to non-fine-grained tasks is described as conceptually possible but not demonstrated.

A recurrent source of ambiguity is nomenclature rather than method. In the supplied literature, "Discern" refers to a discourse-aware entailment reasoning network for conversational machine reading on ShARC (Gao et al., 2020), "DEC" refers to the Dynamic Enhancement Chain for multi-hop question answering (Ji et al., 21 Jun 2025), and the Deliberative Reasoning Network is a separate uncertainty-driven paradigm for belief-tracked inference with pretrained LLMs (Xu et al., 6 Aug 2025). DECERN, by contrast, denotes the active learning framework for fine-grained image classification described above. This distinction is important because the similarly named systems address different modalities, optimization targets, and evaluation regimes.