Soft Confident Learning in Anomaly Detection

• Soft Confident Learning is a training strategy where every sample is retained, with influence adjusted continuously by estimated data uncertainty via anomaly scores.
• It uses a saturated inverse mapping and IQR-based thresholding to modulate sample weights, ensuring high-confidence nominal data drive learning while boundary samples contribute weakly.
• Embedded within meta-learning frameworks like CoMet and CoZAD, the approach improves anomaly detection performance and model stability in contaminated and uncurated training environments.

Searching arXiv for papers on “Soft Confident Learning” and closely related formulations. Soft Confident Learning denotes a confidence-modulated training strategy in which uncertain training examples are not discarded but retained with reduced influence. In the 2025 anomaly-detection literature, the term is used most concretely in Confident Meta-learning (CoMet) and CoZAD, where anomaly scores serve as a proxy for data uncertainty, train/validation loss covariance serves as a proxy for model uncertainty, and both signals are injected directly into the training objective (Aqeel et al., 4 Aug 2025, Aqeel et al., 25 Aug 2025). The defining contrast is with hard confident-learning-style filtering: samples near the nominal–anomalous boundary, or samples suspected of contamination, are not removed, because they may still encode informative boundary structure.

1. Definition and conceptual boundaries

In CoZAD, Soft Confident Learning (SCL) is defined as a mechanism in which every training sample is retained, but its contribution is modulated continuously according to an estimated confidence that it belongs to the normal data manifold (Aqeel et al., 25 Aug 2025). Prototypical normal samples receive high weight; boundary or potentially anomalous samples receive low, but nonzero, weight. CoMet uses the same basic idea in a different setting: learning from uncurated data where nominal and anomalous samples may coexist, thereby relaxing the standard industrial-anomaly-detection assumption that all training data are nominal (Aqeel et al., 4 Aug 2025).

This usage is adjacent to, but distinct from, the original Confident Learning framework for noisy labels. Standard CL estimates the joint distribution between noisy labels and latent true labels, then uses thresholded counting, ranking, and pruning to identify label errors (Northcutt et al., 2019). By contrast, the anomaly-detection SCL formulations do not rely on class labels or a confident joint. They reuse the model’s own anomaly score as an endogenous confidence signal and soften the intervention from deletion to attenuation.

The term should also be distinguished from “Soft Confidence-Weighted learning”, which is a second-order online binary classification method that softens the hard update rule of Confidence-Weighted learning through a soft-margin objective (Wang et al., 2012). The lexical overlap is substantial, but the object of confidence is different: in SCW it is the Gaussian-distributed classifier parameters, whereas in SCL for anomaly detection it is the reliability of training samples and the stability of optimization.

2. Core mathematical formulation

The central SCL construction in CoMet and CoZAD has two components: data uncertainty and model uncertainty (Aqeel et al., 4 Aug 2025, Aqeel et al., 25 Aug 2025). Data uncertainty is estimated from the anomaly score itself. If $a_\theta(x_i)$ is the anomaly score of sample $x_i$, the per-sample confidence weight is defined by a saturated inverse map,

$w_i = \min(1, t/a_{\theta}(x_i))$

or, in CoZAD’s notation,

$w_i = \min(1, \tau/a_{\theta}(x_i)).$

When the score is below the threshold, the sample receives weight $1$; when the score exceeds the threshold, its weight decays proportionally. This makes the anomaly score simultaneously a detection statistic and a proxy for nominal confidence.

The threshold is estimated by interquartile-range (IQR) outlier detection: $$t = Q_3 + \kappa (Q_3 - Q_1), \qquad \tau = Q_3 + \kappa \cdot (Q_3 - Q_1).$$ Here $Q_1$ and $Q_3$ are the first and third quartiles of the anomaly-score distribution, and $\kappa$ is a tunable sensitivity parameter. The data-weighted anomaly-detection loss is then

$$\mathcal{L}_{data}(\theta) = \sum_{i=1}^{N} w_i \cdot \mathcal{L}_{AD}(x_i|\theta).$$

The interpretation stated in both papers is precise: high-confidence nominal samples dominate gradient updates, while suspicious or boundary samples still contribute weakly.

Model uncertainty is estimated globally through the covariance between training and validation losses. The intended object is a $x_i$0 covariance matrix over train and validation loss sequences,

$x_i$1

and its determinant is used as a scalar instability signal. The regularization coefficient is

$x_i$2

Large determinant implies unstable, dispersed train/validation behavior and therefore stronger $x_i$3 regularization.

The full SCL objective is

$x_i$4

This formulation is the central mathematical distinction of the recent anomaly-detection literature: sample-level trust is handled continuously, while optimization instability is handled through uncertainty-adaptive regularization rather than fixed weight decay.

3. Embedding in anomaly-detection systems

SCL is not presented as a standalone preprocessing trick. In both CoMet and CoZAD it is embedded in a larger training stack, especially meta-learning (Aqeel et al., 4 Aug 2025, Aqeel et al., 25 Aug 2025).

Framework	Setting	Role of SCL
CoMet	Uncurated industrial anomaly detection with contaminated training data	Downweights low-confidence samples and regularizes updates by train/validation loss covariance
CoZAD	Zero-shot anomaly detection trained on auxiliary nominal data and transferred across unseen domains	Retains all samples, emphasizes prototypical normal patterns, and regularizes meta-learning

CoMet is explicitly model-agnostic: it can be applied to any anomaly detector trainable end-to-end via gradient descent of a specific loss function, provided a per-sample anomaly score and a differentiable loss are available (Aqeel et al., 4 Aug 2025). The paper demonstrates this with DifferNet / Normalizing Flows and SimpleNet. Tasks are randomly partitioned, one task is used for inner-loop adaptation, the remaining tasks act as validation/meta-test data, and the SCL objective is used in both the inner and outer MAML-style updates. This means SCL is part of the bilevel optimization itself, not a data-cleaning stage executed once before training.

CoZAD integrates SCL with a frozen WideResNet-50 backbone, a SimpleNet-style anomaly detector, contrastive learning, and MAML-style meta-learning (Aqeel et al., 25 Aug 2025). The backbone extracts 1536-dimensional descriptors from the second and third intermediate layers, a learnable adaptor $x_i$5 maps them into a task-specific space,

$x_i$6

synthetic anomalies are created by

$x_i$7

with $x_i$8, and a discriminator $x_i$9 is trained with a margin-based anomaly loss. At inference, anomaly scoring becomes

$w_i = \min(1, t/a_{\theta}(x_i))$0

In CoZAD, contrastive learning is used to make normal embeddings more compact and discriminative, so that the anomaly score becomes a more meaningful proxy for confidence. The overall loss combines SCL with a batch contrastive term,

$w_i = \min(1, t/a_{\theta}(x_i))$1

The paper’s interpretation is that contrastive learning sharpens normal cluster structure, and SCL then biases updates toward reliable anchors while still preserving lower-confidence boundary regions.

4. Empirical behavior and ablation evidence

The most direct empirical validation of the “soft” part of Soft Confident Learning appears in the CoZAD ablations (Aqeel et al., 25 Aug 2025). On MVTec-AD and VisA, the direct comparison is:

• Hard CL (remove samples): 91.6 / 83.1
• Full soft weighting: 92.9 / 85.3

The paper treats this as the clearest evidence that preserving all samples, especially boundary ones, is preferable to sample deletion. In the same study, removing data uncertainty weighting reduces performance to 91.4 / 83.7, removing model uncertainty reduces it to 92.1 / 84.5, and removing confident learning entirely reduces it to 90.8 / 81.2. In the component study, adding “Confident Learning” to SimpleNet improves I-AUROC from 89.7 to 91.8 on MVTec-AD and from 80.0 to 81.8 on VisA.

CoMet supplies the complementary contaminated-training evidence (Aqeel et al., 4 Aug 2025). With MVTec-AD, the ablation on CoMet-NF moves from 94.9 without SCL and ML, to 96.8 without ML, to 99.2 for full CoMet; on CoMet-SN the same ablation ends at 99.7. The paper explicitly states that removing “Soft Confident Learning on Data” highlights its role in handling ambiguous samples. In a contamination study where anomalous samples are inserted into the training set at rates from 0% to 10%, CoMet sustains high AUROC with minimal degradation up to 10% contamination.

The precision–recall behavior reported for CoMet clarifies the effect of soft suppression. For DifferNet $w_i = \min(1, t/a_{\theta}(x_i))$2 CoMet-NF, recall rises from 76.4 to 93.4 on MVTec, from 70.3 to 90.8 on VIADUCT, and from 87.5 to 94.3 on KSDD2, while precision decreases slightly (Aqeel et al., 4 Aug 2025). The interpretation given in the paper is that the method reduces false negatives by preventing contaminated or boundary-adjacent samples from broadening the nominal manifold.

CoZAD extends the evidence to cross-domain zero-shot transfer. The paper reports image-level AUROC of 92.9 on MVTec-AD, 97.2 on BTAD, 94.7 on RSDD, 97.9 on DAGM, and 99.2 on DTD-Synthetic, with pixel-level AUROC of 96.3 on MVTec-AD and strong medical results including 96.8 on HeadCT, 98.7 on BrainMRI, and 99.3 on Br35H (Aqeel et al., 25 Aug 2025). The authors repeatedly connect these gains to SCL’s emphasis on prototypical normal samples while retaining boundary information.

5. Relation to adjacent confidence-modulated paradigms

The anomaly-detection literature uses “Soft Confident Learning” in a narrower sense than the broader soft-label literature. In “Learning with Confidence: Training Better Classifiers from Soft Labels”, the central object is a discrete probability distribution over class labels, and the main techniques are wrapper methods such as duplication, weighting, sampling, and ensembles (Vries et al., 2024). That work studies learning from explicit soft labels; CoMet and CoZAD instead derive confidence from anomaly scores and convert it into per-sample weights.

A second adjacent line is “Soft Augmentation for Image Classification”, where target confidence becomes a nonlinear function of augmentation severity (Liu et al., 2022). There, stronger crop or occlusion yields less confident supervision through

$w_i = \min(1, t/a_{\theta}(x_i))$3

and the softened target distributes the residual mass uniformly over non-ground-truth classes. This is confidence-modulated supervision, but the source of uncertainty is known input corruption rather than model-derived anomaly evidence.

A third adjacent concept is the original CL framework. Confident Learning estimates the joint distribution between noisy observed labels and latent true labels and then uses thresholded counting, ranking, and pruning (Northcutt et al., 2019). In that sense, standard CL is already probabilistic in its inputs and target object, but it becomes hard at the level of assignment and pruning. The recent SCL papers can therefore be read as replacing CL-style hard intervention with continuous attenuation in settings where labels are absent or contamination is only indirectly observable.

This suggests a useful technical distinction: some methods make the target soft, some make the sample weight soft, and some make the optimization trust soft. The anomaly-detection SCL formulations belong primarily to the third category.

6. Limitations, assumptions, and open directions

The recent SCL formulations are explicitly heuristic in several places. CoZAD notes that IQR-based thresholding may not adapt optimally to all score distributions and proposes learnable thresholding as future work (Aqeel et al., 25 Aug 2025). It also notes that confidence estimation remains heuristic because anomaly scores are converted to weights through a simple capped inverse transform, and uncertainty calibration is not deeply studied. Some SCL-specific hyperparameters are not fully specified, including $w_i = \min(1, t/a_{\theta}(x_i))$4, $w_i = \min(1, t/a_{\theta}(x_i))$5, the exact contrastive temperature, the precise task construction for MAML, the number of inner-loop steps, and the exact covariance estimation window.

CoMet makes the dependence on underlying assumptions especially explicit (Aqeel et al., 4 Aug 2025). The anomaly score must become meaningful early enough, because $w_i = \min(1, t/a_{\theta}(x_i))$6 is derived from $w_i = \min(1, t/a_{\theta}(x_i))$7. A majority of training samples should still be nominal; if contamination becomes dominant, quartile-based thresholding may break down. Boundary samples may include hard-but-valid nominal instances, and SCL intentionally suppresses them; the paper associates this with higher recall but slightly lower precision. The covariance determinant is also only a practical proxy for model uncertainty, not a Bayesian estimate.

Computational cost is another recurring constraint. In CoMet, SCL itself is relatively cheap, but the meta-learning wrapper is more expensive because it performs inner and outer updates across task splits (Aqeel et al., 4 Aug 2025). CoZAD similarly cites computational efficiency as a limitation for real-time deployment, especially on high-resolution imagery, even though the framework avoids dependence on vision-language alignments or model ensembles (Aqeel et al., 25 Aug 2025).

The forward-looking directions named in the literature are correspondingly concrete: learnable thresholding, explainability, few-shot adaptation, multimodal data, and federated learning (Aqeel et al., 25 Aug 2025). A plausible implication is that future SCL variants may move from heuristic confidence proxies toward explicitly calibrated, task-adaptive trust estimation, while preserving the central idea that uncertain samples should usually be attenuated rather than deleted.