Intra-Cluster Mixup (ICM) is a data augmentation framework for complementary-label learning that leverages intra-cluster mixing in a learned feature space to reduce complementary-label noise.
It employs a pretrained SimSiam encoder and k-means clustering to restrict mixup operations to semantically similar samples, preserving the complementary-label constraint.
Empirical results on benchmarks like MNIST and CIFAR show significant accuracy improvements, with noise reduction leading to gains up to 35.7% in certain settings.
Intra-Cluster Mixup (ICM) is a data-augmentation framework tailored to complementary-label learning (CLL) that preserves the CLL constraint while leveraging label sharing, thereby reducing the complementary-label noise that makes vanilla Mixup ineffective in CLL. In CLL, training labels specify classes to which an instance does not belong rather than its ordinary class. The formulation introduced in "Intra-Cluster Mixup: An Effective Data Augmentation Technique for Complementary-Label Learning" treats the core obstacle for Mixup in this regime as a label-mismatch problem: naively mixing complementary labels can allocate probability mass to a true class of a mixed input, violating the complementary-label constraint. ICM addresses this mismatch by restricting mixing to nearby examples in a learned feature space, and is reported to improve balanced and imbalanced CLL across synthetic and real-world complementary-label datasets, with substantial gains on MNIST- and CIFAR-family benchmarks (Mai et al., 22 Sep 2025).
1. Complementary-label learning setting
CLL is a specialized form of weakly-supervised learning in which the learner observes only complementary labels. Let K≥3 denote the number of classes, and let the training set be
Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},
where yi is the unknown ordinary label. The goal is still ordinary classification at test time, with classifier
f(x)=argmaxkg(x)k.
A standard loss-based CLL objective minimizes a surrogate complementary loss over the complementary dataset:
L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).
The framework surrounding ICM is situated within an established family of CLL risk estimators and surrogate losses. Unbiased Risk Estimators (URE) were shown to exist under a uniform transition matrix, meaning complementary labels are uniformly drawn from classes other than the ordinary label, but these estimators can be unstable and can suffer from negative empirical risks. To avoid that instability, surrogate complementary loss methods optimize convex surrogates tied directly to complementary labels. Two instantiations used with ICM are the negative-learning variant
ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),
with γ∈(0,1), and the exponential variant
ℓEXP(yˉ,p)=exp(pyˉ).
For non-uniform complementary-label generation, Forward Correction (FWD) introduces a transition layer,
fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).
The distribution-matching loss DM is also used as a representative CLL baseline (Mai et al., 22 Sep 2025).
This formulation is important because ICM is not a standalone classifier. It is an augmentation procedure designed to plug into loss-based CLL training without changing the underlying task definition.
2. Why vanilla Mixup fails in CLL
Vanilla Mixup in ordinary supervised learning forms a synthetic input-target pair from samples (xi,yi) and Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},0 using Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},1:
Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},2
In the ordinary-label setting, the soft target is consistent with the mixed input. In CLL, however, the ordinary labels are unavailable. If one naively substitutes complementary labels,
Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},3
the resulting soft target may assign positive mass to a true class of one of the mixed inputs. The paper identifies this as complementary-label noise (Mai et al., 22 Sep 2025).
The risk decomposition used in the paper formalizes this failure mode. Let the hard complementary-label risk and the soft risk under Mixup be
Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},4
An error term due to complementary-label noise is defined as
Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},5
For a synthetic pair Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},6, the proposition in the paper shows that the mixed complementary risk includes a clean term plus Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},7, which captures additional error arising when complementary labels coincide with true classes under mixing (Mai et al., 22 Sep 2025).
The empirical evidence reported is specific and substantial. On CIFAR-10 with ResNet18 and SCL-NL, naive Mixup induces approximately Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},8 complementary-label noise, and removing that noise improves accuracy by approximately Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},9. A gradient-estimation analysis further compares the ordinary gradient yi0 with the complementary gradient yi1 through
yi2
where
yi3
The “Mixup Noise-Free” setting shows significantly lower MSE, bias, and variance than naive Mixup on MNIST and CIFAR-10. This suggests that the central deficiency of vanilla Mixup in CLL is not interpolation itself, but interpolation across semantically incompatible complementary labels (Mai et al., 22 Sep 2025).
3. ICM algorithm and learning objective
ICM restricts Mixup to nearby examples that are likely to share the same ordinary class. The method first extracts embeddings with a pretrained SimSiam encoder and then performs yi4-means clustering on the embedding set yi5. Each sample receives a cluster label yi6 as a preprocessing step. Euclidean distance is used in embedding space, and the clustering objective minimizes within-cluster sum of squares (Mai et al., 22 Sep 2025).
Within training, each minibatch yi7 is partitioned into cluster-specific sub-batches yi8. For each cluster yi9 with f(x)=argmaxkg(x)k.0, distinct pairs f(x)=argmaxkg(x)k.1 and f(x)=argmaxkg(x)k.2 are sampled. Using f(x)=argmaxkg(x)k.3 with f(x)=argmaxkg(x)k.4, ICM forms
f(x)=argmaxkg(x)k.5
For stability, f(x)=argmaxkg(x)k.6 is reported to work well in practice, and the implementation sets f(x)=argmaxkg(x)k.7 to synchronize input and label mixing.
In this formulation, complementary labels remain one-hot over incorrect classes before mixing; the convex interpolation is intended to keep the mixed target unlikely to allocate probability to a true class when both examples come from the same latent class cluster (Mai et al., 22 Sep 2025).
The significance of ICM lies in how it reframes augmentation for complementary supervision. Standard Mixup assumes label interpolation is semantically valid across random pairs. ICM replaces that assumption with a locality prior in a learned representation space, using cluster consistency as a surrogate for latent class consistency.
4. Integration with CLL losses and implementation details
A key characteristic of ICM is that it is designed to slot directly into existing CLL objectives. For SCL-NL and SCL-EXP, hard complementary labels are replaced with soft f(x)=argmaxkg(x)k.9 and passed into the chosen loss:
L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).0
L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).1
For FWD, the same construction is used after the transition transform:
L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).2
For DM, ICM provides the synthetic pairs and the loss is computed on the mixed inputs and mixed complementary targets without further changes. The paper states that no modification is needed beyond replacing random cross-class Mixup with within-cluster sampling and using soft complementary targets (Mai et al., 22 Sep 2025).
The implementation protocol is fixed at the dataset level. Embeddings are extracted once with a pretrained SimSiam encoder, clustering is performed once with L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).3-means, and cluster IDs are stored. Typical experiments use L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).4, selected via validation, with an ablation over L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).5 recommending L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).6 as a good accuracy-efficiency trade-off. The training loop then repeatedly samples minibatches, partitions them by cluster, forms intra-cluster mixed examples, accumulates the selected CLL loss on the synthetic pairs, and updates parameters by gradient descent (Mai et al., 22 Sep 2025).
Recommended settings reported for reproducibility include a ResNet18 backbone, Adam, initial learning rate L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).7, weight decayL(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).8, batch size L(gθ;ℓ)=N1i=1∑Nℓ(yˉi,gθ(xi)).9, and ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),0 epochs. Dataset-specific ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),1 values are also reported: CIFAR-10 approximately ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),2, CIFAR-20 approximately ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),3, MNIST approximately ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),4, KMNIST approximately ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),5, and FMNIST approximately ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),6 (Mai et al., 22 Sep 2025).
5. Empirical behavior across balanced and imbalanced CLL
The evaluation spans synthetic complementary labels on CIFAR-10, CIFAR-20, MNIST, KMNIST, and FMNIST, together with real human-annotated complementary labels on CLCIFAR10 and CLCIFAR20. Balanced and long-tailed imbalanced settings are considered, including ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),7, ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),8, and ℓNL(yˉ,p)=−log(1−pyˉ+γ),p=softmax(g(x)),9 up to γ∈(0,1)0, under three setups: imbalanced ordinary labels with uniform transition, balanced ordinary labels with biased transition, and both imbalanced ordinary labels and biased transition (Mai et al., 22 Sep 2025).
Representative reported results are summarized below.
Setting
Baseline vs. +ICM
Change
Balanced CLCIFAR10
FWD γ∈(0,1)1 vs. FWD+ICM γ∈(0,1)2
γ∈(0,1)3
Balanced CLCIFAR10
S-NL γ∈(0,1)4 vs. S-NL+ICM γ∈(0,1)5
γ∈(0,1)6
Balanced CLCIFAR20
FWD γ∈(0,1)7 vs. FWD+ICM γ∈(0,1)8
γ∈(0,1)9
CLCIFAR10, ℓEXP(yˉ,p)=exp(pyˉ).0
S-NL ℓEXP(yˉ,p)=exp(pyˉ).1 vs. S-NL+ICM ℓEXP(yˉ,p)=exp(pyˉ).2
ℓEXP(yˉ,p)=exp(pyˉ).3
CIFAR-10, ℓEXP(yˉ,p)=exp(pyˉ).4
S-NL ℓEXP(yˉ,p)=exp(pyˉ).5 vs. S-NL+ICM ℓEXP(yˉ,p)=exp(pyˉ).6
ℓEXP(yˉ,p)=exp(pyˉ).7
CIFAR-10, ℓEXP(yˉ,p)=exp(pyˉ).8
FWD ℓEXP(yˉ,p)=exp(pyˉ).9 vs. FWD+ICM fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).0
fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).1
CIFAR-20, fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).2
FWD fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).3 vs. FWD+ICM fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).4
fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).5
MNIST, fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).6
S-NL fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).7 vs. S-NL+ICM fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).8
fˉ(x)=T⊤f(x),ℓFWD(yˉ,p)=ℓ(yˉ,T⊤p).9
Across methods, naive “+Mix” provides only limited benefit and is consistently worse than “+ICM”; one reported example is CIFAR-10 with (xi,yi)0, where S-EXP+Mix obtains (xi,yi)1 while S-EXP+ICM reaches (xi,yi)2, a gain of (xi,yi)3. The paper also reports a marked drop in complementary-label noise ratio relative to Mixup, such as MNIST decreasing from (xi,yi)4 to (xi,yi)5 under ICM. Combining ICM with Flip, Cutout, AutoAugment, and RandAug yields accuracy near (xi,yi)6 on CIFAR-10, and ICM consistently boosts performance over the same augmentations without ICM (Mai et al., 22 Sep 2025).
These results support two linked conclusions. First, ICM is not tied to one particular CLL loss, since gains are reported for SCL-NL, SCL-EXP, FWD, and DM. Second, the effect is strongest where complementary-label noise is especially damaging, which is consistent with the paper’s broader summary that ICM improves MNIST-family accuracy by approximately (xi,yi)7 and CIFAR-family performance by approximately (xi,yi)8–(xi,yi)9.
6. Computational profile, limitations, and related formulations
The computational overhead of ICM is concentrated in preprocessing. SimSiam embedding extraction requires one pass over the dataset, and Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},00-means clustering has complexity Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},01 with Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},02 clusters and Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},03 iterations. Both are performed once offline. During training, sampling intra-cluster pairs within minibatches is reported to add negligible overhead relative to forward and backward passes. For large Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},04, approximate Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},05-means or mini-batch Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},06-means can be used, and cluster IDs Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},07 impose minimal memory overhead (Mai et al., 22 Sep 2025).
The method has several explicit limitations. If embeddings do not reflect class structure, as can occur with weak features or insufficient pretraining, clusters may overlap and intra-cluster mixing may not reduce noise. In datasets with heavy inter-class overlap, intra-cluster mixing may still induce noise. Performance depends on the choice of Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},08, Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},09, and embedding quality; too small a Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},10 yields coarse clusters, while too large a Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},11 produces tiny clusters. ICM also creates more complex synthetic structures, and simpler models such as linear models or small MLPs can underperform with ICM on simple datasets such as the MNIST family. The multi-complementary-label setting was not evaluated and is left as future work (Mai et al., 22 Sep 2025).
For severe imbalance, the paper reports an extension called MICM that mixes three samples inside clusters using Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},12 for inputs and Inverse Distance Weighting for label coefficients:
Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},13
Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},14
The weighting uses
Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},15
with Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},16 used when Dˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},17 to avoid division by zero and control anchor weight. MICM is reported to improve minority-class learning on complex datasets such as CIFAR, while ICM suffices on simpler datasets such as the MNIST family (Mai et al., 22 Sep 2025).
A related but distinct formulation appears in "SynerMix: Synergistic Mixup Solution for Enhanced Intra-Class Cohesion and Inter-Class Separability in Image Classification", where SynerMix-Intra is described as “ICM at the class granularity” in an ordinary supervised setting. There, features from the same class are combined with normalized UniformDˉ={(xi,yˉi)}i=1N,xi∈Rd,yˉi∈[K]∖{yi},18 weights, labels remain hard one-hot targets, and the method is combined with inter-class MixUp or Manifold MixUp through a weighted objective. That paper states that “Intra-Cluster Mixup (ICM) is intra-class mixing when the ‘cluster’ is the entire class,” and suggests adaptation to true sub-cluster ICM by replacing whole-class sets with sub-clusters defined within a class (Xu et al., 2024). This suggests that the term “intra-cluster mixup” has a broader geometric interpretation beyond CLL: local convex combinations are used to promote label-consistent interpolation, but the label semantics differ substantially between ordinary supervision and complementary-label supervision.
Within the CLL literature, ICM’s novelty is therefore not merely that it performs local Mixup, but that it operationalizes locality specifically to respect the complementary-label constraint. That places it alongside URE, SCL, FWD, and DM as a method for complementary supervision, while addressing a part of the training pipeline—data augmentation—that earlier CLL work had largely left underexplored (Mai et al., 22 Sep 2025).