---
title: Complementary-Label Learning (CLL)
url: https://www.emergentmind.com/topics/complementary-label-learning-cll
type: topic
---

# Complementary-Label Learning (CLL)

Searching arXiv for recent and foundational papers on Complementary-Label Learning to ground the article.
Complementary-Label Learning (CLL) is a weakly supervised multi-class classification paradigm in which each training instance is annotated not with its true class, but with one or more classes that it does **not** belong to. In the standard single-complement setting, the learner observes pairs \((x_i,\bar{y}_i)\) with \(\bar{y}_i \neq y_i\), trains only on complementary supervision, and must output an ordinary-label classifier at test time. Across the literature, CLL has developed from unbiased risk estimation under uniform complementary-label generation to broader frameworks based on transition matrices, probability estimation, candidate-label duality, multi-complement and unlabeled extensions, data augmentation, and adaptations to semi-supervised, domain-shifted, and large-class regimes [2509.17971, 2209.09500, 2002.02158].

## 1. Formal setting and probabilistic structure

A standard formulation assumes \(K\) classes, input space \(\mathcal{X}\subseteq \mathbb{R}^d\), and observed training data \((x_i,\bar{y}_i)\), where \(\bar{y}_i \in \{1,\dots,K\}\) is a complementary label satisfying \(\bar{y}_i \neq y_i\) and \(K>2\). A scoring function \(g_\theta:\mathbb{R}^d\to\mathbb{R}^K\) produces logits or probabilities \(p=\mathrm{softmax}(g_\theta(x))\), and prediction is \(f(x)=\arg\max_k g_\theta(x)_k\). Many CLL methods optimize a complementary empirical objective of the form
\[
\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar{y}_i,g_\theta(x_i)\big).
\]
The key distinction from ordinary supervision is that each label is a negative constraint rather than a positive class assertion [2509.17971].

A central probabilistic device is the complementary-label transition matrix. In the class-conditional model, complementary labels satisfy
\[
R_{kj}=P(\bar{Y}=k\mid Y=j),
\]
with \(R_{jj}=0\) in the noiseless case, and the ordinary and complementary posteriors are linked by
\[
\mathbf{p}_{\bar{Y}\mid x}=R\,\mathbf{p}_{Y\mid x}.
\]
Under uniform complementary-label generation,
\[
P(\bar{Y}=k\mid X)=\frac{1-P(Y=k\mid X)}{K-1}.
\]
This identity underlies unbiased risk estimation, forward correction, and decoding-based formulations, and it clarifies why CLL is a structured label-transformation problem rather than merely a variant of noisy-label learning [2209.09500].

The literature also generalizes single complementary labels to set-valued supervision. Multiple complementary labels can be represented as \(\bar{Y}\subset Y\) of size \(c\), and in candidate-label formulations the complement of the candidate set is treated as the observed supervision. This establishes an explicit equivalence between “multiple complementary labels” and “a reduced candidate set” [2002.02158, 2001.04243].

## 2. Risk estimation, reduction, and decoding

Foundational CLL theory derived unbiased estimators of ordinary classification risk from complementarily labeled data. In the one-versus-all and pairwise-comparison constructions, the binary surrogate \(\ell\) is required to satisfy the symmetric condition
\[
\ell(z)+\ell(-z)=1.
\]
Under the uniform complementary-label model, the ordinary risk admits an unbiased expression in terms of a complementary loss, and corresponding estimation error bounds achieve the optimal parametric convergence rate [1705.07541].

A later generalization removed the restriction to specific losses and models by introducing the corrected complementary loss
\[
\tilde{\ell}(f(x),\bar{y})=-(K-1)\,\ell(f(x),\bar{y})+\sum_{j=1}^K \ell(f(x),j),
\]
which is unbiased under uniform complementary labeling. For a general known transition matrix \(T\), the corrected loss vector is obtained through
\[
\tilde{L}(f(x))=(T^{-\,\top})\,L(f(x)).
\]
This framework also introduced class-wise non-negative correction and a gradient-ascent trick to mitigate the negative-risk pathology that appears when flexible models drive the empirical complementary risk below zero [1810.04327].

A distinct line of work recast CLL as probability estimation rather than direct risk correction. In that view, one first estimates \(P(\bar{Y}\mid X)\), then decodes back to \(P(Y\mid X)\) either by inversion when \(R\) is well-conditioned,
\[
\mathbf{p}_{Y\mid x}=R^{-1}\mathbf{p}_{\bar{Y}\mid x},
\]
or by nearest-row decoding when \(R\) is ill-conditioned or non-invertible,
\[
\hat{y}(x)=\arg\min_k d\!\left(\hat{\mathbf{p}}_{\bar{Y}\mid x},R_k\right).
\]
This same framework motivates validation via held-out complementary log-loss, or SCEL, without access to ordinary labels [2209.09500].

The relationship between ordinary-label learning and CLL was further formalized through additivity and duality of one-versus-all and pairwise losses. In that formulation, providing \(M\) complementary labels is equivalent to providing \(N=K-M\) candidate labels, and the candidate-label loss can be written in both ordinary-label and complementary-label forms:
\[
\CandLoss{f(x), Y}
= \xi_1\sum_{y\in Y}\ordLoss{f(x), y} + \xi_2
= \xi_1\sum_{\bar{y}\in \bar{Y}}\compLoss{f(x), \bar{y}} + \bar{\xi}_2.
\]
This bridge places CLL within a broader family of weak-supervision schemes rather than as an isolated special case [2002.02158].

## 3. Multiple complements, unlabeled data, and neighboring paradigms

Allowing multiple complementary labels per instance changes both the estimator and the information content of supervision. One direct unbiased estimator for a set \(\tilde{Y}\) of complementary labels uses
\[
\hat{R}(f)=\frac{1}{n}\sum_{i=1}^n\left[\sum_{y\notin \tilde{Y}_i}\mathcal{L}\big(f(x_i),y\big)-\frac{K-1-|\tilde{Y}_i|}{|\tilde{Y}_i|}\sum_{y'\in \tilde{Y}_i}\mathcal{L}\big(f(x_i),y'\big)\right].
\]
Because the negative term can destabilize optimization with unbounded losses, upper-bound surrogates based on
\[
-\log\Big(\sum_{y\notin \tilde{Y}_i}p_\theta(y\mid x_i)\Big)
\quad\text{or}\quad
\exp\Big(-\sum_{y\notin \tilde{Y}_i}p_\theta(y\mid x_i)\Big)
\]
were proposed as more stable alternatives. The same work also described wrapper methods that decompose multi-complementary supervision into multiple single-complement examples, though this dilutes supervision purity [1912.12927].

An alternative extension derived unbiased risk estimators for any number of complementary labels together with unlabeled samples. Defining the cumulative multi-class loss
\[
\mathcal{L}(g(x)):=\sum_{y=1}^K \ell(g(x),y),
\]
the multi-complementary loss is
\[
\overline{\ell}(g(x),\overline{Y})
=\mathcal{L}(g(x))-\frac{K-1}{|\overline{Y}|}\sum_{y\in\overline{Y}}\ell(g(x),y),
\]
and unlabeled data can be incorporated through a trade-off coefficient \(\gamma\) without losing unbiasedness. The resulting MCUL estimator preserves the standard \(\mathcal{O}(1/\sqrt{n})\) generalization rate [2001.04243].

Complementary supervision has also been embedded into adjacent weakly supervised settings. In semi-supervised learning, low-confidence unlabeled samples can contribute “trusted non-labels” rather than discarded pseudo-labels: CCL constructs negative pairs for contrastive learning from bottom-\(k\) low-probability classes and improved FixMatch on CIFAR-10 from \(92.53\%\) to \(94.96\%\) with 40 labels and on STL-10 from \(64.03\%\) to \(70.38\%\) with 40 labels [2212.06643]. In test-time adaptation, complementary labels are drawn from “less probable categories,” yielding a risk-consistent complementary loss and state-of-the-art results on CIFAR-10-C and CIFAR-100-C under both one-at-a-time and continual corruption protocols [2301.06013]. Complementary labels have likewise been used in unsupervised domain adaptation from a complementary-labeled source domain, in partial-label learning through an adversarial complementary classifier, and in open-world settings with augmented classes, where unlabeled target-distributed samples identify an aggregated unseen class [2008.01454, 2305.09897, 2211.10701].

## 4. Non-uniformity, imbalance, realism, and large label spaces

A persistent misconception in early CLL benchmarking was that uniform, noiseless complementary-label generation is a reasonable stand-in for practice. Human-annotated benchmarks contradict that assumption. CLImage introduced CLCIFAR10, CLCIFAR20, CLMicroImageNet10, and CLMicroImageNet20, and found that the empirical transition matrices are non-uniform, class-dependent, and noisy, with noise rates \(3.93\%\), \(2.80\%\), \(5.19\%\), and \(3.21\%\), respectively. Performance drops from synthetic to real complementary labels are correspondingly large: for example, FWD-U on CIFAR10 falls from \(64.19\pm0.57\) to \(34.83\pm0.50\), and SCL-NL from \(63.76\pm0.09\) to \(34.77\pm0.60\). The dataset-level ablation identifies annotation noise as the most influential factor, while also showing that validation with only complementary labels remains materially weaker than ordinary-label validation [2305.08295].

Several methods explicitly target this realism gap. SCARCE replaces the uniform-generation assumption with a Selected-Completely-at-Random assumption analogous to SCAR in PU learning and rewrites multi-class CLL as a family of negative-unlabeled binary problems under one-vs-rest. It introduces a corrected risk
\[
\tilde{R}_k(f_k)=g(RR_k(f_k))+(1-\pi_k)\frac{1}{n_k^N}\sum_{x\in D_k^N}\ell(-f_k(x)),
\]
where \(g\) is a non-negative wrapper, and reports superiority over prior methods on both synthetic and real-world benchmarks [2311.15502]. For class-imbalanced complementary data, WCLL defines class weights
\[
\omega_j=\frac{1/\pi_j}{\sum_{i=1}^K 1/\pi_i}
\]
and a weighted complementary loss
\[
\hat{\ell}(f(x),j)=-(K-1)\omega_j\ell(f(x),j)+\sum_{i=1}^K\omega_i\ell(f(x),i),
\]
with estimation error bounds under Rademacher complexity assumptions. The reported results show strong gains on MNIST, CIFAR-10, Tiny-Imagenet, and DDSM under multiple imbalance scenarios [2209.14189].

Large class spaces expose a different failure mode: under uniform complementary-label generation, each observed complement removes only one of many classes, and the learning signal becomes extremely weak. BICL addresses this by deliberately designing biased, sparse transition matrices in which each true class only maps to a small subset of complementary labels. The framework argues, via a Fano-style lower bound involving \(H^Q(Y\mid \bar{Y})\), that biased constrained labeling reduces conditional entropy and improves learnability. Empirically, this shifts CLL into regimes that had previously been nearly unusable: on CIFAR-100, FWD rises from \(5.53\%\) under uniform generation to \(46.70\%\) under BICL, and on TinyImageNet-200 from \(4.00\%\) to \(32.15\%\) [2605.15586].

These developments jointly indicate that robustness in CLL is inseparable from assumptions about the complementary-label mechanism. Real complementary labels are biased and noisy; uniformity is convenient analytically but often practically inaccurate; and deliberately biased mechanisms can even be advantageous when designed to strengthen the supervision signal [2305.08295, 2209.09500, 2605.15586].

## 5. Data augmentation, representation, and optimization dynamics

For several years, most CLL work concentrated on loss design, while the role of augmentation remained underexplored. A first major analysis showed that complementary-label learning relies on implicit sharing of complementary information among nearby instances in representation space. That work introduced complementary-label augmentation, which propagates additional soft complementary labels through an affinity matrix:
\[
Z=\alpha \bar{Y}+(1-\alpha)W\bar{Y},
\]
or iteratively,
\[
Z\leftarrow \alpha \bar{Y}+(1-\alpha)WZ.
\]
The resulting soft targets are used in
\[
R_{LA}(g;\ell)=\frac{1}{N}\sum_{i=1}^N \sum_{k=1}^K Z_{i,k}\,\ell(k,g(x_i)).
\]
Distance-weighted multi-step propagation produced large gains; for example, SCL-NL on CIFAR-10 improved from \(73.95\pm0.86\) to \(88.72\pm0.15\), and on CLCIFAR-10 from \(44.12\pm1.49\) to \(74.87\pm2.25\) [2305.08344].

A later augmentation study examined Mixup directly in CLL and found that it is ineffective when naively applied because mixed complementary labels can conflict with the semantic content of the interpolated sample. The analysis defines complementary-label noise for a model \(f\) as
\[
\varepsilon=\mathbb{E}_{(x,\bar{y})\sim \bar{D}}\big[\mathbb{I}\{\bar{y}=f(x)\}\big],
\]
and shows that Mixup introduces extra noise terms in the complementary risk. On CIFAR-10 with SCL-NL and ResNet-18, original Mixup yields \(15.81\%\) complementary-label noise, and a noise-free protocol improves accuracy by 7 percentage points. The proposed Intra-Cluster Mixup (ICM) restricts mixing to nearby examples in a SimSiam embedding space, after \(k\)-means clustering, and optimizes
\[
\mathcal{L}'(g_\theta;\ell)
=\frac{1}{N}\sum_{i=1}^{N}\Big[\lambda\,\ell\big(\bar{y}_i,g_\theta(\tilde{x}_{i,j})\big)+(1-\lambda)\,\ell\big(\bar{y}_j,g_\theta(\tilde{x}_{i,j})\big)\Big].
\]
ICM reduces the complementary-label noise ratio on MNIST from \(16.24\%\) with Mixup to \(0.95\%\), and the full experiments report improvements of \(30\%\) and \(10\%\) on MNIST and CIFAR datasets, respectively, across balanced and imbalanced settings [2509.17971].

These results make a broader methodological point. In CLL, augmentation is not neutral: it changes the semantics of negative supervision. Methods that preserve locality or label-sharing structure can improve optimization, whereas augmentations that indiscriminately mix complementary constraints can inject contradictions into the training signal [2305.08344, 2509.17971].

## 6. Empirical landscape, evaluation practice, and software infrastructure

The empirical evaluation of CLL is now split between synthetic complementary-label benchmarks and real human-annotated datasets, and the gap between them is itself a substantive research result. Synthetic experiments remain useful for controlled studies of transition matrices, imbalance, and estimator properties, but CLImage shows that methods performing strongly under uniform synthetic assumptions can degrade sharply under realistic bias and annotation noise. This makes evaluation protocols, validation criteria, and generation assumptions part of the scientific content of CLL rather than mere implementation detail [2305.08295].

Standardization efforts have therefore become important. The `libcll` toolkit provides a universal interface spanning synthetic and real-world datasets, multiple generation assumptions, and major algorithmic families including URE, CPE, FWD, SCL, DM, and MCL variants. It supports uniform, biased, noisy, SCAR-style, and multiple-complement settings; includes datasets such as MNIST, KMNIST, FMNIST, CIFAR10, CIFAR20, Yeast, Texture, Dermatology, Synthetic Control, MicroImageNet10, MicroImageNet20, CLCIFAR10, CLCIFAR20, CLMicroImageNet10, and CLMicroImageNet20; and reports that CPE and FWD are strong defaults across many scenarios, with CPE-T particularly robust when the provided transition matrix is mismatched or noisy [2411.12276].

A concise reading of the present empirical landscape is therefore possible. Uniform-noiseless CLL is now relatively well understood; complementary probability estimation and forward correction are strong baselines when the transition mechanism is reliable; multi-complement and unlabeled variants expand the usable supervision formats; data augmentation has become a first-class design axis; and real-world robustness remains the central unresolved challenge, especially under noisy, biased, or difficult-to-validate complementary-label distributions [2209.09500, 2305.08295, 2411.12276].

Source: https://www.emergentmind.com/topics/complementary-label-learning-cll