Complementary-Label Learning (CLL)

• Complementary-Label Learning (CLL) is a weakly supervised multi-class classification paradigm where instances are labeled with classes they do not belong to.
• It employs transition matrices, corrected loss functions, and candidate-label duality to convert complementary labels into effective ordinary-label predictions.
• Recent advancements integrate multiple complementary labels, unlabeled data, and tailored data augmentation to overcome challenges like noise, imbalance, and large label spaces.

Searching arXiv for recent and foundational papers on Complementary-Label Learning to ground the article. Complementary-Label Learning (CLL) is a weakly supervised multi-class classification paradigm in which each training instance is annotated not with its true class, but with one or more classes that it does not belong to. In the standard single-complement setting, the learner observes pairs $(x_i,\bar{y}_i)$ with $\bar{y}_i \neq y_i$, trains only on complementary supervision, and must output an ordinary-label classifier at test time. Across the literature, CLL has developed from unbiased risk estimation under uniform complementary-label generation to broader frameworks based on transition matrices, probability estimation, candidate-label duality, multi-complement and unlabeled extensions, data augmentation, and adaptations to semi-supervised, domain-shifted, and large-class regimes (Mai et al., 22 Sep 2025, Lin et al., 2022, Katsura et al., 2020).

1. Formal setting and probabilistic structure

A standard formulation assumes $K$ classes, input space $\mathcal{X}\subseteq \mathbb{R}^d$, and observed training data $(x_i,\bar{y}_i)$, where $\bar{y}_i \in \{1,\dots,K\}$ is a complementary label satisfying $\bar{y}_i \neq y_i$ and $K>2$. A scoring function $g_\theta:\mathbb{R}^d\to\mathbb{R}^K$ produces logits or probabilities $p=\mathrm{softmax}(g_\theta(x))$, and prediction is $\bar{y}_i \neq y_i$0. Many CLL methods optimize a complementary empirical objective of the form

$\bar{y}_i \neq y_i$1

The key distinction from ordinary supervision is that each label is a negative constraint rather than a positive class assertion (Mai et al., 22 Sep 2025).

A central probabilistic device is the complementary-label transition matrix. In the class-conditional model, complementary labels satisfy

$\bar{y}_i \neq y_i$2

with $\bar{y}_i \neq y_i$3 in the noiseless case, and the ordinary and complementary posteriors are linked by

$\bar{y}_i \neq y_i$4

Under uniform complementary-label generation,

$\bar{y}_i \neq y_i$5

This identity underlies unbiased risk estimation, forward correction, and decoding-based formulations, and it clarifies why CLL is a structured label-transformation problem rather than merely a variant of noisy-label learning (Lin et al., 2022).

The literature also generalizes single complementary labels to set-valued supervision. Multiple complementary labels can be represented as $\bar{y}_i \neq y_i$6 of size $\bar{y}_i \neq y_i$7, and in candidate-label formulations the complement of the candidate set is treated as the observed supervision. This establishes an explicit equivalence between “multiple complementary labels” and “a reduced candidate set” (Katsura et al., 2020, Cao et al., 2020).

2. Risk estimation, reduction, and decoding

Foundational CLL theory derived unbiased estimators of ordinary classification risk from complementarily labeled data. In the one-versus-all and pairwise-comparison constructions, the binary surrogate $\bar{y}_i \neq y_i$8 is required to satisfy the symmetric condition

$\bar{y}_i \neq y_i$9

Under the uniform complementary-label model, the ordinary risk admits an unbiased expression in terms of a complementary loss, and corresponding estimation error bounds achieve the optimal parametric convergence rate (Ishida et al., 2017).

A later generalization removed the restriction to specific losses and models by introducing the corrected complementary loss

$K$0

which is unbiased under uniform complementary labeling. For a general known transition matrix $K$1, the corrected loss vector is obtained through

$K$2

This framework also introduced class-wise non-negative correction and a gradient-ascent trick to mitigate the negative-risk pathology that appears when flexible models drive the empirical complementary risk below zero (Ishida et al., 2018).

A distinct line of work recast CLL as probability estimation rather than direct risk correction. In that view, one first estimates $K$3, then decodes back to $K$4 either by inversion when $K$5 is well-conditioned,

$K$6

or by nearest-row decoding when $K$7 is ill-conditioned or non-invertible,

$K$8

This same framework motivates validation via held-out complementary log-loss, or SCEL, without access to ordinary labels (Lin et al., 2022).

The relationship between ordinary-label learning and CLL was further formalized through additivity and duality of one-versus-all and pairwise losses. In that formulation, providing $K$9 complementary labels is equivalent to providing $\mathcal{X}\subseteq \mathbb{R}^d$0 candidate labels, and the candidate-label loss can be written in both ordinary-label and complementary-label forms: $\mathcal{X}\subseteq \mathbb{R}^d$1 This bridge places CLL within a broader family of weak-supervision schemes rather than as an isolated special case (Katsura et al., 2020).

3. Multiple complements, unlabeled data, and neighboring paradigms

Allowing multiple complementary labels per instance changes both the estimator and the information content of supervision. One direct unbiased estimator for a set $\mathcal{X}\subseteq \mathbb{R}^d$2 of complementary labels uses

$\mathcal{X}\subseteq \mathbb{R}^d$3

Because the negative term can destabilize optimization with unbounded losses, upper-bound surrogates based on

$\mathcal{X}\subseteq \mathbb{R}^d$4

were proposed as more stable alternatives. The same work also described wrapper methods that decompose multi-complementary supervision into multiple single-complement examples, though this dilutes supervision purity (Feng et al., 2019).

An alternative extension derived unbiased risk estimators for any number of complementary labels together with unlabeled samples. Defining the cumulative multi-class loss

$\mathcal{X}\subseteq \mathbb{R}^d$5

the multi-complementary loss is

$\mathcal{X}\subseteq \mathbb{R}^d$6

and unlabeled data can be incorporated through a trade-off coefficient $\mathcal{X}\subseteq \mathbb{R}^d$7 without losing unbiasedness. The resulting MCUL estimator preserves the standard $\mathcal{X}\subseteq \mathbb{R}^d$8 generalization rate (Cao et al., 2020).

Complementary supervision has also been embedded into adjacent weakly supervised settings. In semi-supervised learning, low-confidence unlabeled samples can contribute “trusted non-labels” rather than discarded pseudo-labels: CCL constructs negative pairs for contrastive learning from bottom-$\mathcal{X}\subseteq \mathbb{R}^d$9 low-probability classes and improved FixMatch on CIFAR-10 from $(x_i,\bar{y}_i)$0 to $(x_i,\bar{y}_i)$1 with 40 labels and on STL-10 from $(x_i,\bar{y}_i)$2 to $(x_i,\bar{y}_i)$3 with 40 labels (Deng et al., 2022). In test-time adaptation, complementary labels are drawn from “less probable categories,” yielding a risk-consistent complementary loss and state-of-the-art results on CIFAR-10-C and CIFAR-100-C under both one-at-a-time and continual corruption protocols (Han et al., 2023). Complementary labels have likewise been used in unsupervised domain adaptation from a complementary-labeled source domain, in partial-label learning through an adversarial complementary classifier, and in open-world settings with augmented classes, where unlabeled target-distributed samples identify an aggregated unseen class (Zhang et al., 2020, Jia et al., 2023, Li et al., 2022).

4. Non-uniformity, imbalance, realism, and large label spaces

A persistent misconception in early CLL benchmarking was that uniform, noiseless complementary-label generation is a reasonable stand-in for practice. Human-annotated benchmarks contradict that assumption. CLImage introduced CLCIFAR10, CLCIFAR20, CLMicroImageNet10, and CLMicroImageNet20, and found that the empirical transition matrices are non-uniform, class-dependent, and noisy, with noise rates $(x_i,\bar{y}_i)$4, $(x_i,\bar{y}_i)$5, $(x_i,\bar{y}_i)$6, and $(x_i,\bar{y}_i)$7, respectively. Performance drops from synthetic to real complementary labels are correspondingly large: for example, FWD-U on CIFAR10 falls from $(x_i,\bar{y}_i)$8 to $(x_i,\bar{y}_i)$9, and SCL-NL from $\bar{y}_i \in \{1,\dots,K\}$0 to $\bar{y}_i \in \{1,\dots,K\}$1. The dataset-level ablation identifies annotation noise as the most influential factor, while also showing that validation with only complementary labels remains materially weaker than ordinary-label validation (Wang et al., 2023).

Several methods explicitly target this realism gap. SCARCE replaces the uniform-generation assumption with a Selected-Completely-at-Random assumption analogous to SCAR in PU learning and rewrites multi-class CLL as a family of negative-unlabeled binary problems under one-vs-rest. It introduces a corrected risk

$\bar{y}_i \in \{1,\dots,K\}$2

where $\bar{y}_i \in \{1,\dots,K\}$3 is a non-negative wrapper, and reports superiority over prior methods on both synthetic and real-world benchmarks (Wang et al., 2023). For class-imbalanced complementary data, WCLL defines class weights

$\bar{y}_i \in \{1,\dots,K\}$4

and a weighted complementary loss

$\bar{y}_i \in \{1,\dots,K\}$5

with estimation error bounds under Rademacher complexity assumptions. The reported results show strong gains on MNIST, CIFAR-10, Tiny-Imagenet, and DDSM under multiple imbalance scenarios (Wei et al., 2022).

Large class spaces expose a different failure mode: under uniform complementary-label generation, each observed complement removes only one of many classes, and the learning signal becomes extremely weak. BICL addresses this by deliberately designing biased, sparse transition matrices in which each true class only maps to a small subset of complementary labels. The framework argues, via a Fano-style lower bound involving $\bar{y}_i \in \{1,\dots,K\}$6, that biased constrained labeling reduces conditional entropy and improves learnability. Empirically, this shifts CLL into regimes that had previously been nearly unusable: on CIFAR-100, FWD rises from $\bar{y}_i \in \{1,\dots,K\}$7 under uniform generation to $\bar{y}_i \in \{1,\dots,K\}$8 under BICL, and on TinyImageNet-200 from $\bar{y}_i \in \{1,\dots,K\}$9 to $\bar{y}_i \neq y_i$0 (Mai et al., 15 May 2026).

These developments jointly indicate that robustness in CLL is inseparable from assumptions about the complementary-label mechanism. Real complementary labels are biased and noisy; uniformity is convenient analytically but often practically inaccurate; and deliberately biased mechanisms can even be advantageous when designed to strengthen the supervision signal (Wang et al., 2023, Lin et al., 2022, Mai et al., 15 May 2026).

5. Data augmentation, representation, and optimization dynamics

For several years, most CLL work concentrated on loss design, while the role of augmentation remained underexplored. A first major analysis showed that complementary-label learning relies on implicit sharing of complementary information among nearby instances in representation space. That work introduced complementary-label augmentation, which propagates additional soft complementary labels through an affinity matrix: $\bar{y}_i \neq y_i$1 or iteratively,

$\bar{y}_i \neq y_i$2

The resulting soft targets are used in

$\bar{y}_i \neq y_i$3

Distance-weighted multi-step propagation produced large gains; for example, SCL-NL on CIFAR-10 improved from $\bar{y}_i \neq y_i$4 to $\bar{y}_i \neq y_i$5, and on CLCIFAR-10 from $\bar{y}_i \neq y_i$6 to $\bar{y}_i \neq y_i$7 (Lin et al., 2023).

A later augmentation study examined Mixup directly in CLL and found that it is ineffective when naively applied because mixed complementary labels can conflict with the semantic content of the interpolated sample. The analysis defines complementary-label noise for a model $\bar{y}_i \neq y_i$8 as

$\bar{y}_i \neq y_i$9

and shows that Mixup introduces extra noise terms in the complementary risk. On CIFAR-10 with SCL-NL and ResNet-18, original Mixup yields $K>2$0 complementary-label noise, and a noise-free protocol improves accuracy by 7 percentage points. The proposed Intra-Cluster Mixup (ICM) restricts mixing to nearby examples in a SimSiam embedding space, after $K>2$1-means clustering, and optimizes

$K>2$2

ICM reduces the complementary-label noise ratio on MNIST from $K>2$3 with Mixup to $K>2$4, and the full experiments report improvements of $K>2$5 and $K>2$6 on MNIST and CIFAR datasets, respectively, across balanced and imbalanced settings (Mai et al., 22 Sep 2025).

These results make a broader methodological point. In CLL, augmentation is not neutral: it changes the semantics of negative supervision. Methods that preserve locality or label-sharing structure can improve optimization, whereas augmentations that indiscriminately mix complementary constraints can inject contradictions into the training signal (Lin et al., 2023, Mai et al., 22 Sep 2025).

6. Empirical landscape, evaluation practice, and software infrastructure

The empirical evaluation of CLL is now split between synthetic complementary-label benchmarks and real human-annotated datasets, and the gap between them is itself a substantive research result. Synthetic experiments remain useful for controlled studies of transition matrices, imbalance, and estimator properties, but CLImage shows that methods performing strongly under uniform synthetic assumptions can degrade sharply under realistic bias and annotation noise. This makes evaluation protocols, validation criteria, and generation assumptions part of the scientific content of CLL rather than mere implementation detail (Wang et al., 2023).

Standardization efforts have therefore become important. The libcll toolkit provides a universal interface spanning synthetic and real-world datasets, multiple generation assumptions, and major algorithmic families including URE, CPE, FWD, SCL, DM, and MCL variants. It supports uniform, biased, noisy, SCAR-style, and multiple-complement settings; includes datasets such as MNIST, KMNIST, FMNIST, CIFAR10, CIFAR20, Yeast, Texture, Dermatology, Synthetic Control, MicroImageNet10, MicroImageNet20, CLCIFAR10, CLCIFAR20, CLMicroImageNet10, and CLMicroImageNet20; and reports that CPE and FWD are strong defaults across many scenarios, with CPE-T particularly robust when the provided transition matrix is mismatched or noisy (Ye et al., 2024).

A concise reading of the present empirical landscape is therefore possible. Uniform-noiseless CLL is now relatively well understood; complementary probability estimation and forward correction are strong baselines when the transition mechanism is reliable; multi-complement and unlabeled variants expand the usable supervision formats; data augmentation has become a first-class design axis; and real-world robustness remains the central unresolved challenge, especially under noisy, biased, or difficult-to-validate complementary-label distributions (Lin et al., 2022, Wang et al., 2023, Ye et al., 2024).