Papers
Topics
Authors
Recent
Search
2000 character limit reached

Intra-Cluster Mixup (ICM): Augmentation for CLL

Updated 12 July 2026
  • Intra-Cluster Mixup (ICM) is a data augmentation framework for complementary-label learning that leverages intra-cluster mixing in a learned feature space to reduce complementary-label noise.
  • It employs a pretrained SimSiam encoder and k-means clustering to restrict mixup operations to semantically similar samples, preserving the complementary-label constraint.
  • Empirical results on benchmarks like MNIST and CIFAR show significant accuracy improvements, with noise reduction leading to gains up to 35.7% in certain settings.

Intra-Cluster Mixup (ICM) is a data-augmentation framework tailored to complementary-label learning (CLL) that preserves the CLL constraint while leveraging label sharing, thereby reducing the complementary-label noise that makes vanilla Mixup ineffective in CLL. In CLL, training labels specify classes to which an instance does not belong rather than its ordinary class. The formulation introduced in "Intra-Cluster Mixup: An Effective Data Augmentation Technique for Complementary-Label Learning" treats the core obstacle for Mixup in this regime as a label-mismatch problem: naively mixing complementary labels can allocate probability mass to a true class of a mixed input, violating the complementary-label constraint. ICM addresses this mismatch by restricting mixing to nearby examples in a learned feature space, and is reported to improve balanced and imbalanced CLL across synthetic and real-world complementary-label datasets, with substantial gains on MNIST- and CIFAR-family benchmarks (Mai et al., 22 Sep 2025).

1. Complementary-label learning setting

CLL is a specialized form of weakly-supervised learning in which the learner observes only complementary labels. Let K3K \ge 3 denote the number of classes, and let the training set be

Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},

where yiy_i is the unknown ordinary label. The goal is still ordinary classification at test time, with classifier

f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.

A standard loss-based CLL objective minimizes a surrogate complementary loss over the complementary dataset:

L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).

The framework surrounding ICM is situated within an established family of CLL risk estimators and surrogate losses. Unbiased Risk Estimators (URE) were shown to exist under a uniform transition matrix, meaning complementary labels are uniformly drawn from classes other than the ordinary label, but these estimators can be unstable and can suffer from negative empirical risks. To avoid that instability, surrogate complementary loss methods optimize convex surrogates tied directly to complementary labels. Two instantiations used with ICM are the negative-learning variant

NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),

with γ(0,1)\gamma\in(0,1), and the exponential variant

EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).

For non-uniform complementary-label generation, Forward Correction (FWD) introduces a transition layer,

fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).

The distribution-matching loss DM is also used as a representative CLL baseline (Mai et al., 22 Sep 2025).

This formulation is important because ICM is not a standalone classifier. It is an augmentation procedure designed to plug into loss-based CLL training without changing the underlying task definition.

2. Why vanilla Mixup fails in CLL

Vanilla Mixup in ordinary supervised learning forms a synthetic input-target pair from samples (xi,yi)(x_i,y_i) and Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},0 using Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},1:

Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},2

In the ordinary-label setting, the soft target is consistent with the mixed input. In CLL, however, the ordinary labels are unavailable. If one naively substitutes complementary labels,

Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},3

the resulting soft target may assign positive mass to a true class of one of the mixed inputs. The paper identifies this as complementary-label noise (Mai et al., 22 Sep 2025).

The risk decomposition used in the paper formalizes this failure mode. Let the hard complementary-label risk and the soft risk under Mixup be

Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},4

An error term due to complementary-label noise is defined as

Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},5

For a synthetic pair Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},6, the proposition in the paper shows that the mixed complementary risk includes a clean term plus Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},7, which captures additional error arising when complementary labels coincide with true classes under mixing (Mai et al., 22 Sep 2025).

The empirical evidence reported is specific and substantial. On CIFAR-10 with ResNet18 and SCL-NL, naive Mixup induces approximately Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},8 complementary-label noise, and removing that noise improves accuracy by approximately Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},9. A gradient-estimation analysis further compares the ordinary gradient yiy_i0 with the complementary gradient yiy_i1 through

yiy_i2

where

yiy_i3

The “Mixup Noise-Free” setting shows significantly lower MSE, bias, and variance than naive Mixup on MNIST and CIFAR-10. This suggests that the central deficiency of vanilla Mixup in CLL is not interpolation itself, but interpolation across semantically incompatible complementary labels (Mai et al., 22 Sep 2025).

3. ICM algorithm and learning objective

ICM restricts Mixup to nearby examples that are likely to share the same ordinary class. The method first extracts embeddings with a pretrained SimSiam encoder and then performs yiy_i4-means clustering on the embedding set yiy_i5. Each sample receives a cluster label yiy_i6 as a preprocessing step. Euclidean distance is used in embedding space, and the clustering objective minimizes within-cluster sum of squares (Mai et al., 22 Sep 2025).

Within training, each minibatch yiy_i7 is partitioned into cluster-specific sub-batches yiy_i8. For each cluster yiy_i9 with f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.0, distinct pairs f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.1 and f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.2 are sampled. Using f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.3 with f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.4, ICM forms

f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.5

For stability, f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.6 is reported to work well in practice, and the implementation sets f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.7 to synchronize input and label mixing.

The training objective is the augmented CLL loss

f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.8

In this formulation, complementary labels remain one-hot over incorrect classes before mixing; the convex interpolation is intended to keep the mixed target unlikely to allocate probability to a true class when both examples come from the same latent class cluster (Mai et al., 22 Sep 2025).

The significance of ICM lies in how it reframes augmentation for complementary supervision. Standard Mixup assumes label interpolation is semantically valid across random pairs. ICM replaces that assumption with a locality prior in a learned representation space, using cluster consistency as a surrogate for latent class consistency.

4. Integration with CLL losses and implementation details

A key characteristic of ICM is that it is designed to slot directly into existing CLL objectives. For SCL-NL and SCL-EXP, hard complementary labels are replaced with soft f(x)=argmaxkg(x)k.f(x)=\arg\max_k g(x)_k.9 and passed into the chosen loss:

L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).0

L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).1

For FWD, the same construction is used after the transition transform:

L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).2

For DM, ICM provides the synthetic pairs and the loss is computed on the mixed inputs and mixed complementary targets without further changes. The paper states that no modification is needed beyond replacing random cross-class Mixup with within-cluster sampling and using soft complementary targets (Mai et al., 22 Sep 2025).

The implementation protocol is fixed at the dataset level. Embeddings are extracted once with a pretrained SimSiam encoder, clustering is performed once with L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).3-means, and cluster IDs are stored. Typical experiments use L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).4, selected via validation, with an ablation over L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).5 recommending L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).6 as a good accuracy-efficiency trade-off. The training loop then repeatedly samples minibatches, partitions them by cluster, forms intra-cluster mixed examples, accumulates the selected CLL loss on the synthetic pairs, and updates parameters by gradient descent (Mai et al., 22 Sep 2025).

Recommended settings reported for reproducibility include a ResNet18 backbone, Adam, initial learning rate L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).7, weight decay L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).8, batch size L(gθ;)=1Ni=1N(yˉi,gθ(xi)).\mathcal{L}(g_\theta;\ell)=\frac{1}{N}\sum_{i=1}^N \ell\big(\bar y_i,g_\theta(x_i)\big).9, and NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),0 epochs. Dataset-specific NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),1 values are also reported: CIFAR-10 approximately NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),2, CIFAR-20 approximately NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),3, MNIST approximately NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),4, KMNIST approximately NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),5, and FMNIST approximately NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),6 (Mai et al., 22 Sep 2025).

5. Empirical behavior across balanced and imbalanced CLL

The evaluation spans synthetic complementary labels on CIFAR-10, CIFAR-20, MNIST, KMNIST, and FMNIST, together with real human-annotated complementary labels on CLCIFAR10 and CLCIFAR20. Balanced and long-tailed imbalanced settings are considered, including NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),7, NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),8, and NL(yˉ,p)=log(1pyˉ+γ),p=softmax(g(x)),\ell_{\mathrm{NL}}(\bar y,p)=-\log\big(1-p_{\bar y}+\gamma\big), \qquad p=\mathrm{softmax}(g(x)),9 up to γ(0,1)\gamma\in(0,1)0, under three setups: imbalanced ordinary labels with uniform transition, balanced ordinary labels with biased transition, and both imbalanced ordinary labels and biased transition (Mai et al., 22 Sep 2025).

Representative reported results are summarized below.

Setting Baseline vs. +ICM Change
Balanced CLCIFAR10 FWD γ(0,1)\gamma\in(0,1)1 vs. FWD+ICM γ(0,1)\gamma\in(0,1)2 γ(0,1)\gamma\in(0,1)3
Balanced CLCIFAR10 S-NL γ(0,1)\gamma\in(0,1)4 vs. S-NL+ICM γ(0,1)\gamma\in(0,1)5 γ(0,1)\gamma\in(0,1)6
Balanced CLCIFAR20 FWD γ(0,1)\gamma\in(0,1)7 vs. FWD+ICM γ(0,1)\gamma\in(0,1)8 γ(0,1)\gamma\in(0,1)9
CLCIFAR10, EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).0 S-NL EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).1 vs. S-NL+ICM EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).2 EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).3
CIFAR-10, EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).4 S-NL EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).5 vs. S-NL+ICM EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).6 EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).7
CIFAR-10, EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).8 FWD EXP(yˉ,p)=exp(pyˉ).\ell_{\mathrm{EXP}}(\bar y,p)=\exp(p_{\bar y}).9 vs. FWD+ICM fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).0 fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).1
CIFAR-20, fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).2 FWD fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).3 vs. FWD+ICM fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).4 fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).5
MNIST, fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).6 S-NL fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).7 vs. S-NL+ICM fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).8 fˉ(x)=Tf(x),FWD(yˉ,p)=(yˉ,Tp).\bar f(x)=T^\top f(x),\qquad \ell_{\mathrm{FWD}}(\bar y,p)=\ell\big(\bar y,T^\top p\big).9

Across methods, naive “+Mix” provides only limited benefit and is consistently worse than “+ICM”; one reported example is CIFAR-10 with (xi,yi)(x_i,y_i)0, where S-EXP+Mix obtains (xi,yi)(x_i,y_i)1 while S-EXP+ICM reaches (xi,yi)(x_i,y_i)2, a gain of (xi,yi)(x_i,y_i)3. The paper also reports a marked drop in complementary-label noise ratio relative to Mixup, such as MNIST decreasing from (xi,yi)(x_i,y_i)4 to (xi,yi)(x_i,y_i)5 under ICM. Combining ICM with Flip, Cutout, AutoAugment, and RandAug yields accuracy near (xi,yi)(x_i,y_i)6 on CIFAR-10, and ICM consistently boosts performance over the same augmentations without ICM (Mai et al., 22 Sep 2025).

These results support two linked conclusions. First, ICM is not tied to one particular CLL loss, since gains are reported for SCL-NL, SCL-EXP, FWD, and DM. Second, the effect is strongest where complementary-label noise is especially damaging, which is consistent with the paper’s broader summary that ICM improves MNIST-family accuracy by approximately (xi,yi)(x_i,y_i)7 and CIFAR-family performance by approximately (xi,yi)(x_i,y_i)8–(xi,yi)(x_i,y_i)9.

The computational overhead of ICM is concentrated in preprocessing. SimSiam embedding extraction requires one pass over the dataset, and Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},00-means clustering has complexity Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},01 with Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},02 clusters and Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},03 iterations. Both are performed once offline. During training, sampling intra-cluster pairs within minibatches is reported to add negligible overhead relative to forward and backward passes. For large Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},04, approximate Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},05-means or mini-batch Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},06-means can be used, and cluster IDs Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},07 impose minimal memory overhead (Mai et al., 22 Sep 2025).

The method has several explicit limitations. If embeddings do not reflect class structure, as can occur with weak features or insufficient pretraining, clusters may overlap and intra-cluster mixing may not reduce noise. In datasets with heavy inter-class overlap, intra-cluster mixing may still induce noise. Performance depends on the choice of Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},08, Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},09, and embedding quality; too small a Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},10 yields coarse clusters, while too large a Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},11 produces tiny clusters. ICM also creates more complex synthetic structures, and simpler models such as linear models or small MLPs can underperform with ICM on simple datasets such as the MNIST family. The multi-complementary-label setting was not evaluated and is left as future work (Mai et al., 22 Sep 2025).

For severe imbalance, the paper reports an extension called MICM that mixes three samples inside clusters using Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},12 for inputs and Inverse Distance Weighting for label coefficients:

Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},13

Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},14

The weighting uses

Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},15

with Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},16 used when Dˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},17 to avoid division by zero and control anchor weight. MICM is reported to improve minority-class learning on complex datasets such as CIFAR, while ICM suffices on simpler datasets such as the MNIST family (Mai et al., 22 Sep 2025).

A related but distinct formulation appears in "SynerMix: Synergistic Mixup Solution for Enhanced Intra-Class Cohesion and Inter-Class Separability in Image Classification", where SynerMix-Intra is described as “ICM at the class granularity” in an ordinary supervised setting. There, features from the same class are combined with normalized UniformDˉ={(xi,yˉi)}i=1N,xiRd,yˉi[K]{yi},\bar D = \{(x_i,\bar y_i)\}_{i=1}^N,\qquad x_i \in \mathbb{R}^d,\quad \bar y_i \in [K]\setminus\{y_i\},18 weights, labels remain hard one-hot targets, and the method is combined with inter-class MixUp or Manifold MixUp through a weighted objective. That paper states that “Intra-Cluster Mixup (ICM) is intra-class mixing when the ‘cluster’ is the entire class,” and suggests adaptation to true sub-cluster ICM by replacing whole-class sets with sub-clusters defined within a class (Xu et al., 2024). This suggests that the term “intra-cluster mixup” has a broader geometric interpretation beyond CLL: local convex combinations are used to promote label-consistent interpolation, but the label semantics differ substantially between ordinary supervision and complementary-label supervision.

Within the CLL literature, ICM’s novelty is therefore not merely that it performs local Mixup, but that it operationalizes locality specifically to respect the complementary-label constraint. That places it alongside URE, SCL, FWD, and DM as a method for complementary supervision, while addressing a part of the training pipeline—data augmentation—that earlier CLL work had largely left underexplored (Mai et al., 22 Sep 2025).

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Intra-Cluster Mixup (ICM).