Decoupled Confident Learning (DeCoLe)

• The paper introduces DeCoLe as a bias-aware method that decouples group-specific models to detect mislabeled instances and correct label bias.
• It formalizes label bias via group- and class-conditional error rates and uses group-specific thresholds to flag discrepancies between observed and predicted labels.
• Empirical evaluations across synthetic and real-world settings (e.g., hate-speech detection) show that DeCoLe markedly improves recall and precision over standard methods.

Searching arXiv for the specified DeCoLe papers to ground the article and confirm bibliographic details. Decoupled Confident Learning (DeCoLe) is a data-quality method for bias-aware mislabeling detection in datasets where label errors are not random, but systematically differ across social groups. It was introduced as a pruning method for mitigating label bias and later presented as a principled machine-learning framework for detecting mislabeled instances under group-dependent error processes. Its central idea is to extend confident-learning-style label error detection by decoupling the learning process across groups, so that each group has its own predictive model and confidence thresholds. DeCoLe is therefore directed at improving the quality of the data itself, rather than only making downstream predictors more robust to corrupted labels (Li et al., 9 Jul 2025, Li et al., 2023).

1. Emergence and problem scope

A 2023 paper introduced DeCoLe in the context of mitigating label bias, arguing that most fairness and noisy-label methods assume observed training labels are sufficiently correct even in domains where labels are produced by humans and may encode systematic societal bias. A 2025 paper reformulated the same core method as bias-aware mislabeling detection, emphasizing organizational data integrity and the need to identify likely label errors when label quality differs across social groups (Li et al., 2023, Li et al., 9 Jul 2025).

The motivating domains are those in which labels are consequential and are often derived from proxies, judgments, or instruments that may embed social or measurement bias. The papers explicitly identify healthcare, hiring, criminal justice, and content moderation / hate speech detection as settings in which biased labels are plausible and operationally significant. In these settings, the objective is not merely to train a model that is robust to noise, but to identify suspicious labels so that the dataset can be reviewed, pruned, or relabeled.

A key conceptual distinction is between generic label noise and label bias. Generic label noise is usually modeled as a class-conditional corruption process, where the probability of an incorrect observed label depends only on the latent class. DeCoLe targets a more structured regime in which mislabeling depends jointly on latent class and group membership. This distinction is central to the method’s rationale, because a single pooled detector may obscure the very subgroup-specific error patterns that require correction.

2. Formalization of label bias

DeCoLe is formulated for binary classification with an unobserved true label $y^* \in \{0,1\}$, an observed label $\tilde y \in \{0,1\}$, features $\mathbf{x}$, and group membership $g$. In the 2025 formulation, the observed dataset is written as $\mathbf D = (\mathbf x, g, \tilde y)^n$. The problem is to estimate which instances satisfy $\tilde y \neq y^*$ without access to gold labels for the full dataset (Li et al., 9 Jul 2025).

For a group $g_k$, the label-bias mechanism is expressed through group- and class-conditional error rates: $$\pi_{0,g_k} = p(\tilde y = 0 \mid y^* = 1, g=g_k), \qquad \pi_{1,g_k} = p(\tilde y = 1 \mid y^* = 0, g=g_k).$$ Label bias occurs when these rates differ across groups. This formalization makes the error process more specific than standard class-conditional noise: the same latent class may be mislabeled at different rates for different groups, and the dominant error type may itself vary by group (Li et al., 9 Jul 2025).

The papers also emphasize differential subgroup validity: the relationship between $\mathbf{x}$ and $y^*$ may differ across groups. This matters because a global model trained on pooled data may learn a compromise dominated by majority-group structure and thereby mis-handle minority-group instances. A plausible implication is that even if a pooled detector is well calibrated on average, it may be systematically miscalibrated for the groups most affected by label bias.

The hate-speech example used in the 2025 paper illustrates the point. Speech targeting some groups may be more likely to be mislabeled as non-hateful, yielding a group-dependent false negative pattern. In that setting, the observed label is not merely noisy; it is biased in a structured way that interacts with the target group.

3. Algorithmic mechanism

DeCoLe extends Confident Learning (CL) by performing confident-learning-style detection separately for each group. Instead of training a single classifier on pooled data, it trains one classifier $\tilde y \in \{0,1\}$0 for each group $\tilde y \in \{0,1\}$1, computes out-of-sample predicted probabilities within that group, derives group-specific thresholds, and flags contradictions between observed labels and confident predictions as likely mislabels (Li et al., 9 Jul 2025).

The algorithm is fully unsupervised with respect to gold labels. Its required inputs are the noisy dataset, the group indicator, and a classifier for each group. In the 2023 description, out-of-sample probabilities are obtained with cross-validated predictions, consistent with confident-learning practice; the method is described as model-agnostic, with logistic regression used in synthetic experiments and random forest in one hate-speech evaluation (Li et al., 2023).

For each group $\tilde y \in \{0,1\}$2, DeCoLe estimates

$\tilde y \in \{0,1\}$3

then computes two group-specific thresholds: $\tilde y \in \{0,1\}$4 The thresholds are the average predicted positive probability among observed positives and observed negatives within the same group.

In the 2025 notation, these thresholds define a Confident Positive Set and Confident Negative Set: $\tilde y \in \{0,1\}$5

$\tilde y \in \{0,1\}$6

An instance is flagged as potentially mislabeled if its observed label conflicts with the confident prediction, namely observed negatives in $\tilde y \in \{0,1\}$7 or observed positives in $\tilde y \in \{0,1\}$8. Equivalently, the procedure partitions the data into an estimated mislabeled subset $\tilde y \in \{0,1\}$9 and an estimated correctly labeled subset $\mathbf{x}$0 (Li et al., 9 Jul 2025).

The operational intuition is straightforward. If an example is labeled positive but appears less positive than the typical positive example in its own group, or if it is labeled negative but appears more positive than the typical negative example in that group, its observed label is treated as suspicious. Because the thresholds are group-specific, DeCoLe is designed to preserve subgroup-specific structure that a pooled model may wash out.

4. Theoretical characterization

The 2025 paper supplies a formal connection between DeCoLe and the confident-learning literature. Under an idealized condition in which each group-specific classifier returns correct predicted probabilities, the appendix gives a lemma expressing the group-specific bounds as mixtures over latent classes: $\mathbf{x}$1

$\mathbf{x}$2

These expressions formalize the claim that the empirical thresholds recover the group-specific noise structure when the probability model is ideal (Li et al., 9 Jul 2025).

The main guarantee is Theorem 1. If the group- and class-conditional noise rates are below $\mathbf{x}$3 and the group-specific classifiers produce ideal probabilities, then the detected mislabeled set $\mathbf{x}$4 is a consistent estimator of the true mislabeled set $\mathbf{x}$5. In plain terms, the method asymptotically identifies the mislabeled instances exactly under the ideal model.

The analysis is then relaxed to a non-ideal setting. Condition 2 allows the predicted probabilities to be “per-instance diffracted”: $\mathbf{x}$6 with bounded, group-specific noise. Theorem 2 states that DeCoLe remains a consistent estimator of the mislabeled set under this noisy-prediction regime, provided the group-conditional bias structure remains below the same $\mathbf{x}$7-type threshold. The proof argument described in the paper is that the noisy thresholds shift in a way that preserves the relevant decision inequalities, so the same instances are selected as in the ideal case.

These results do not eliminate the modeling assumptions. The stated guarantees rely on a setting in which mislabeling is governed primarily by group and true class. The papers explicitly note that if error depends strongly on additional covariates or contextual variables, then the guarantees may not fully apply.

5. Empirical evaluations

The empirical program consists of synthetic experiments and hate-speech detection studies, all designed to test whether group-wise decoupling improves mislabel detection when errors are subgroup-specific. Across these evaluations, DeCoLe is compared with standard Confident Learning (CL), Co-Teaching (CoT) in the 2025 paper, and random sampling or random pruning baselines (Li et al., 9 Jul 2025, Li et al., 2023).

In the 2023 synthetic study, the dataset contains $\mathbf{x}$8 examples with $\mathbf{x}$9, $g$0, $g$1, and $g$2. Group $g$3 is the majority group at 70% of the population. The construction includes differential subgroup validity through group-specific Gaussian class-conditional distributions with $g$4, and group- and class-conditional noise with high false negatives for the minority group $g$5, $g$6, and high false positives for the majority group $g$7, $g$8. Logistic regression is used, with 5 random seeds and 95% confidence intervals. That study reports that DeCoLe significantly outperforms CL and random sampling in pruning recall and precision, especially on the disadvantaged group, and that the cleaned dataset exhibits reduced false positives for $g$9 and reduced false negatives for $\mathbf D = (\mathbf x, g, \tilde y)^n$0 (Li et al., 2023).

The 2025 synthetic evaluation uses two groups, imbalanced group sizes, differential subgroup validity, and asymmetric group-conditional error patterns. It reports four metrics: overall recall of mislabeled instances, overall precision of the subset estimated as correctly labeled, recall of bias-inducing errors, and precision of the bias-dominant class in the subset estimated as correctly labeled. In the main simulation setting, DeCoLe achieves overall recall of about $\mathbf D = (\mathbf x, g, \tilde y)^n$1 for group $\mathbf D = (\mathbf x, g, \tilde y)^n$2 and $\mathbf D = (\mathbf x, g, \tilde y)^n$3 for $\mathbf D = (\mathbf x, g, \tilde y)^n$4, compared with roughly $\mathbf D = (\mathbf x, g, \tilde y)^n$5 and $\mathbf D = (\mathbf x, g, \tilde y)^n$6 for CL and $\mathbf D = (\mathbf x, g, \tilde y)^n$7 and $\mathbf D = (\mathbf x, g, \tilde y)^n$8 for CoT. On the precision side, it reaches about $\mathbf D = (\mathbf x, g, \tilde y)^n$9–$\tilde y \neq y^*$0, and the appendix reports robustness across balanced groups and scenarios in which both groups share the same error type but differ in error rates (Li et al., 9 Jul 2025).

The principal real-world application is hate-speech detection. The 2025 paper uses the dataset from Kennedy et al. (2020), notable because it includes both common noisy labels $\tilde y \neq y^*$1 and higher-quality theory-grounded labels $\tilde y \neq y^*$2 based on Rasch Measurement Theory and multiple hate-speech dimensions. The evaluation focuses on sexuality, race, and gender, including only groups with at least 3,000 instances so that group-specific neural models are feasible. The base model is a neural network over 768-dimensional DistilBERT embeddings, and results are reported over multiple train-test splits with confidence intervals. DeCoLe achieves the highest or near-highest recall of mislabeled instances, the highest precision among the instances retained as correctly labeled, and strong performance on the bias-aware metrics. One limitation noted in the paper is that for posts targeting men, DeCoLe ranks second rather than first on some metrics, though it remains strongest overall across the full evaluation suite (Li et al., 9 Jul 2025).

The earlier 2023 hate-speech study uses posts targeting young adults and seniors, observed labels from a single hate-speech survey item, and an RMT-based measure as the more bias-reduced ground truth. With a random forest base model, 5 random seeds, and 95% confidence bounds, DeCoLe substantially improves recall over CL and random pruning while maintaining comparable precision, and the cleaned dataset shows significantly reduced false negatives for both target groups (Li et al., 2023).

6. Relation to adjacent methods, uses, and limitations

DeCoLe is best understood as a data auditing and cleaning method, not as a fairness post-processing method for model outputs. The papers explicitly distinguish it from approaches such as equalized odds that assume observed labels are correct and then attempt to equalize performance relative to those labels. If the labels themselves are biased, fairness optimization against those labels can be misleading; DeCoLe instead targets the upstream label-quality problem (Li et al., 2023, Li et al., 9 Jul 2025).

It is also distinct from generic noisy-label methods. Reweighting, loss correction, and co-teaching are described as approaches that modify training, architecture, or loss to improve robustness to generic noise, whereas DeCoLe is data-centric and explicitly designed for group-conditioned noise. Its key hypothesis is that once one conditions on group, the remaining noise behaves more like standard class-conditional noise, allowing confident-learning logic to operate more effectively within each subgroup.

In practical use, the method is appropriate when observed labels may be biased differently across groups and group identifiers are available, but gold-standard labels are not available for all instances. The 2025 paper suggests a workflow in which DeCoLe is run first to flag high-risk instances, after which human experts review those instances and feed the results into active relabeling or data-cleaning pipelines. This suggests an organizational role for DeCoLe in prioritizing limited annotation or audit resources toward the subset most likely to be wrong.

The limitations are likewise explicit. DeCoLe requires known group labels, depends on the quality of within-group classifiers and their out-of-sample probability estimates, and assumes that mislabeling is governed primarily by group and true class. If group definitions are incomplete or erroneous, or if error depends strongly on additional covariates or contextual structure, then performance may degrade and the formal guarantees may not transfer. Within its intended setting, however, the method provides a principled way to detect mislabeled instances under label bias and to improve data reliability at the source (Li et al., 9 Jul 2025).