Multi-Category Contrastive Learning

• Multi-Category Contrastive Learning (MCCL) is a framework of contrastive methods designed to learn representations for samples with overlapping or hierarchical labels.
• It employs multiple projection heads, category-conditioned embeddings, and prototype-based strategies to capture varied similarity notions and prevent over- or under-clustering.
• Empirical studies show MCCL improves performance across tasks by leveraging multi-relation loss functions, particularly in low-data and complex label regimes.

Multi-Category Contrastive Learning (MCCL) denotes a family of contrastive representation-learning methods for settings in which a sample participates in more than one semantic category, relation, or granularity. In contrast to standard contrastive learning with a single projection head and a single notion of “same class vs. different class,” MCCL-style methods define multiple positive relations, category-conditioned embeddings, or category-level proxies so that instances can be pulled together and separated according to label overlap, hierarchy level, aspect identity, category centers, or foreground/background category structure. Across recent work, this design is motivated by the observation that single-head supervised contrastive learning can disregard multiple aspects of similarity and may induce over-clustering or under-clustering in multi-label and hierarchical regimes (Ghanooni et al., 4 Feb 2025), while multi-label settings additionally require positive/negative definitions that do not collapse partially overlapping labels into a binary same/different partition (Liu et al., 2022, Audibert et al., 2024).

1. Scope, terminology, and research lineage

The term “Multi-Category Contrastive Learning” is used across several adjacent research lines rather than as a single canonical algorithm. In multi-label few-shot aspect category detection, MCCL appears as a supervised contrastive component inside a label-enhanced prototypical network (Liu et al., 2022). In multi-label image learning with missing labels, a contrastive formulation based on class-wise and global nuclear norms is introduced as CLML (Ma et al., 2022). In end-to-end multilabel learning, KMCL defines a kernel-based contrastive objective in a Gaussian RKHS (Sajedi et al., 2023). A broader empirical study of multi-label contrastive learning analyzes several multi-category formulations and their behavior across vision and NLP datasets (Audibert et al., 2024). In parallel, multilevel contrastive learning extends supervised contrastive learning with multiple projection heads for multi-label and hierarchical classification (Ghanooni et al., 4 Feb 2025), while later work applies MCCL to open-vocabulary semantic segmentation (Zhang et al., 8 Aug 2025) and long-tailed dynamic facial expression recognition (Cui et al., 4 Sep 2025).

Variant	Setting	Defining mechanism
LPN with MCCL (Liu et al., 2022)	Multi-label few-shot aspect category detection	Per-aspect label-specific embeddings and supervised contrastive loss
CLML (Ma et al., 2022)	Multi-label learning with missing labels	Label correction, low-rank local structure, high-rank global structure
KMCL (Sajedi et al., 2023)	End-to-end multilabel classification	Mixture of exponential kernels in Gaussian RKHS
MLCL (Ghanooni et al., 4 Feb 2025)	Multi-label and hierarchical classification	Multiple projection heads, one per label or hierarchy level
SynSeg MCCL (Zhang et al., 8 Aug 2025)	Open-vocabulary semantic segmentation	Intra/inter-category alignment and separation
MICACL MCCL (Cui et al., 4 Sep 2025)	Long-tailed DFER	Multiscale category-aware weighting and dynamic temperature

This suggests that MCCL is best understood as an umbrella for contrastive objectives that preserve multiple category relations, rather than a uniquely fixed loss.

2. Mathematical structure of multi-category contrast

A generic multi-label supervised contrastive formulation treats each example $i$ as carrying a binary multi-hot label vector $y_i\in\{0,1\}^L$, with feature representation $z_i\in\mathbb{R}^d$, positive set $P(i)\subset A(i)=B\setminus\{i\}$, and positive weights $\lambda_j^i\ge 0$. In the formulation summarized in the comprehensive study, the per-anchor loss is

$$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$

This template subsumes Jaccard-weighted, prototype-based, and label-instance-based variants, and it already departs from single-label SupCon by letting two instances be attracted with a strength determined by shared-label structure rather than a hard class identity (Audibert et al., 2024).

Multilevel contrastive learning makes this multi-relation structure explicit by defining $K$ similarity relations among samples, one relation per label or hierarchy level. For head $k$, $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$, $P_i^k$ is the positive set under relation $y_i\in\{0,1\}^L$0, and the supervised contrastive loss is

$y_i\in\{0,1\}^L$1

The aggregate objective is

$y_i\in\{0,1\}^L$2

For hierarchical tasks, positives are $y_i\in\{0,1\}^L$3; for multi-label tasks, $y_i\in\{0,1\}^L$4; and a global head can be built from thresholded Jaccard overlap $y_i\in\{0,1\}^L$5 (Ghanooni et al., 4 Feb 2025).

In episodic multi-label few-shot learning, MCCL is instantiated through label-specific embeddings. For aspect $y_i\in\{0,1\}^L$6, the prototype $y_i\in\{0,1\}^L$7 and description embedding $y_i\in\{0,1\}^L$8 are concatenated as $y_i\in\{0,1\}^L$9, attention is computed by

$z_i\in\mathbb{R}^d$0

and the label-specific embedding is

$z_i\in\mathbb{R}^d$1

Let $z_i\in\mathbb{R}^d$2. For anchor $z_i\in\mathbb{R}^d$3, the positive set is $z_i\in\mathbb{R}^d$4, the negative set is $z_i\in\mathbb{R}^d$5, and the supervised contrastive loss is averaged over anchors to form $z_i\in\mathbb{R}^d$6 (Liu et al., 2022).

3. Principal design patterns

One major MCCL pattern is the use of multiple projection spaces. In MLCL, the encoder backbone $z_i\in\mathbb{R}^d$7 maps input $z_i\in\mathbb{R}^d$8 to embedding $z_i\in\mathbb{R}^d$9, while multiple projection heads $P(i)\subset A(i)=B\setminus\{i\}$0 learn focused projection spaces for distinct category dimensions. This architecture is motivated by the claim that a single projection head fails to capture the full complexity of different aspects of a sample, especially in multi-label and hierarchical settings, and that multiple heads disentangle different similarity notions. At inference time all projection heads are discarded and the final representation is $P(i)\subset A(i)=B\setminus\{i\}$1 itself, or an aggregate thereof (Ghanooni et al., 4 Feb 2025).

A second pattern is the use of prototypes or category centers instead of pure instance-instance pairing. The prototype-only loss in multi-label contrastive learning treats learnable class prototypes $P(i)\subset A(i)=B\setminus\{i\}$2 as fixed instances in the denominator (Audibert et al., 2024). Center Contrastive Loss makes this proxy-based idea class-wise and explicit by maintaining a center bank $P(i)\subset A(i)=B\setminus\{i\}$3 for each class and contrasting each normalized query $P(i)\subset A(i)=B\setminus\{i\}$4 against all class centers through

$P(i)\subset A(i)=B\setminus\{i\}$5

combined with the center-pull term

$P(i)\subset A(i)=B\setminus\{i\}$6

Centers are updated by standard back-propagation and renormalized each iteration. This removes the need for well-designed sample mining and re-balances the supervisory signal of each class (Cai et al., 2023).

A third pattern models uncertainty and label interaction in a kernel space. KMCL’s Kernel Mixture Module projects a deterministic feature $P(i)\subset A(i)=B\setminus\{i\}$7 to per-class mixture coefficients $P(i)\subset A(i)=B\setminus\{i\}$8, means $P(i)\subset A(i)=B\setminus\{i\}$9, and variances $\lambda_j^i\ge 0$0, then defines per-class exponential kernels

$\lambda_j^i\ge 0$1

The full objective is

$\lambda_j^i\ge 0$2

where the reconstruction loss reconstructs the positive-label mixture, the asymmetric classification loss addresses positive/negative imbalance, and the kernel-based contrastive loss pulls together kernel components for images that share labels via a Bhattacharyya-based InfoNCE (Sajedi et al., 2023).

A fourth pattern reformulates “contrast” without explicit pairwise InfoNCE. CLML defines corrected labels $\lambda_j^i\ge 0$3 by flipping certain missing negatives to positive once network confidence exceeds a threshold after a warm-up epoch, then constructs the loss

$\lambda_j^i\ge 0$4

The first term enforces low-rank local structure for each class, while the second term enforces high-rank global structure across the batch. This design is explicitly intended to avoid the false-negative problem that arises when multi-class positive/negative definitions are borrowed naively for multi-label learning with missing labels (Ma et al., 2022).

4. Task-specific realizations

In multi-label few-shot aspect category detection, MCCL is integrated into a prototypical meta-learning pipeline. Review sentences and label descriptions are encoded with BERT, where the lower 6 layers are frozen and the top 6 are fine-tuned; multi-head self-attention yields the sentence embedding $\lambda_j^i\ge 0$5, label-enhanced prototypes are computed by bilinear relevance weighting against label-description embeddings, and query classification uses negative squared Euclidean distance to prototypes. MCCL then refines the sentence embeddings so that instances sharing an aspect are pulled together while those that do not are pushed apart. Training combines $\lambda_j^i\ge 0$6, $\lambda_j^i\ge 0$7, and an aspect-count prediction loss $\lambda_j^i\ge 0$8, and inference first predicts the aspect count and then selects the top-$\lambda_j^i\ge 0$9 aspects by $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$0 (Liu et al., 2022).

In weakly supervised open-vocabulary semantic segmentation, SynSeg defines MCCL over per-image category sets $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$1. For each category $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$2, the model obtains a foreground synergy feature $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$3, a background synergy feature $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$4, and a CLIP text embedding $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$5. It then combines four losses: intra-category alignment $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$6, intra-category separation $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$7, inter-category alignment of backgrounds $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$8, and inter-category separation of foregrounds $$\ell(z_i) =-\frac{1}{\sum_{p\in P(i)}\lambda_p^i}\sum_{j\in P(i)} \lambda_j^i \;\log \frac{\exp(z_i\!\cdot\!z_j/\tau)}{ \sum_{k\in A(i)}\exp(z_i\!\cdot\!z_k/\tau)}.$$9. These are produced by a Feature Synergy Structure in which FiLM fusion generates a conditional visual tensor, a transformer decoder outputs a semantic activation map, and projected visual tokens are aggregated into foreground and background synergy features. During training, only FiLM, the projector, and the decoder are updated; CLIP weights stay frozen (Zhang et al., 8 Aug 2025).

In long-tailed dynamic facial expression recognition, MICACL defines a Multiscale Category-aware Contrastive Learning strategy on top of a multi-instance backbone formed by the Graph-Enhanced Instance Interaction Module and the Weighted Instance Aggregation Network. The category-aware sample weight is

$K$0

so that tail classes and low-confidence examples receive larger weights. At each scale $K$1, pooled features are projected back to the embedding dimension, cosine similarities are computed with a dynamic temperature

$K$2

and the multiscale loss averages the category-aware contrastive objective across scales. A Classification Enhancement Term is added, yielding $K$3 (Cui et al., 4 Sep 2025).

5. Empirical behavior across domains

Reported empirical results consistently tie MCCL-style methods to regimes in which single-label or single-head contrastive learning is structurally mismatched to the data. In multilevel contrastive learning, top-1 linear-evaluation accuracy on frozen encoders improves from SupCon 76.5% to MLCL 77.7% on CIFAR-100 full, from SupCon 72.8% to MLCL 73.9% on DeepFashion, and from SupCon 46.2% to MLCL 56.0% on low-data CIFAR-100 with 5 K samples. On the TripAdvisor text task with 180 samples, performance is reported as CE 77.0% → MLCL w/o global 77.3% → MLCL 77.9%. Ablations show $K$4 versus $K$5 versus $K$6, a $K$7 low-data gain from adding a superclass head, $K$8–$K$9 gains from per-head temperature tuning with $k$0, and $k$1–$k$2 robustness improvements from the global-head weight in noisy-label or low-data regimes (Ghanooni et al., 4 Feb 2025).

In few-shot aspect category detection, the full LPN configuration with label enhancement and contrastive learning consistently outperforms Proto-AWATT by AUC gains of roughly $k$3–$k$4 and Macro-F1 gains of $k$5–$k$6 across FewAsp single-aspect, multi-aspect, and full settings. Ablations indicate that adding label texts alone yields AUC gains of $k$7–$k$8, while the contrastive module adds another $k$9–$z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$0. The reported t-SNE visualizations show that MCCL makes prototypes far better separated (Liu et al., 2022).

In missing-label multi-label image classification, CLML improves the ResNet101 model by margins of $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$1, $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$2, and $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$3 respectively on MSCOCO, VOC, and NUS-WIDE. On COCO with $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$4 labels retained, the reported improvements are BCE: mAP 76.8% → +1.2% → 78.0%, Focal: 77.0% → +1.3% → 78.3%, Hill: 78.8% → +0.8% → 79.6%, and SPLC: 78.4% → +2.0% → 80.4%. On VOC, adding only the local low-rank term yields $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$5 mAP, only label-correction yields $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$6 mAP, and full CLML yields $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$7 mAP over BCE (Ma et al., 2022).

In category-center-based metric learning, Center Contrastive Loss reports Recall@1 improvements from 80.8 to 83.1 on SOP, from 71.1 to 73.5 on CUB-200-2011, from 87.7 to 91.0 on Cars196, and from 90.2 to 92.3 on InShop Clothes. The paper further reports that one reaches 80% Recall@1 on SOP in $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$8 epochs, whereas most pairwise or listwise methods require $z_i^k=g_k(f(\tilde x_i))\in\mathbb{R}^d$9 epochs, and that noisy-label tests on CUB and Cars196 improve by 3–5 points (Cai et al., 2023).

In end-to-end multilabel learning, KMCL reports VOC2007 mAP 96.2% versus 95.8%, MS-COCO mAP 88.6% versus 88.4% with $P_i^k$0 OF1/CF1, ADP mAP 96.5% versus 96.1% with $P_i^k$1 overall Recall, and ChestX-ray14 mAP 31.5% versus 26.3% and 25.5%, with $P_i^k$2 overall Recall and $P_i^k$3 per-class Recall. Complexity on ADP is reported as KMCL-M 29.4 M params/5.7 GMAC versus ML-GCN 44.9 M/31.4 GMAC (Sajedi et al., 2023).

The broad comparative study of multi-label contrastive learning qualifies these gains. It reports that contrastive methods are particularly strong in Macro-F1 on datasets with a large number of labels, that performance advantages are amplified in limited-data regimes, and that ranking metrics such as mAP may still favor ZLPR. It also reports that on small label spaces such as PASCAL with 20 labels, BCE or ASY remain slightly better, whereas on large-label datasets such as MS-COCO, NUS-WIDE, RCV1, and BGC, the regularized contrastive loss matches or beats the best BCE-based losses in Macro-F1 (Audibert et al., 2024).

In dense prediction and long-tailed sequence recognition, the reported gains are likewise task-specific. SynSeg reports zero-shot mIoU of 62.2 on VOC, 41.8 on Context, 34.9 on Object, 23.6 on Stuff, and 30.9 on City, with an average mIoU of 38.7; removing any of the four MCCL terms degrades performance, and dropping $P_i^k$4 yields the largest drop (Zhang et al., 8 Aug 2025). MICACL reports on DFEW fold 5 that the configuration without MCCL attains 68.62% / 56.26% WAR / UAR, while adding MCCL raises this to 69.91% / 64.34%, with the largest benefit on tail categories (Cui et al., 4 Sep 2025).

6. Interpretation, misconceptions, and open directions

A common source of confusion is to treat MCCL as synonymous with ordinary supervised contrastive learning on multi-label data. The literature does not support that simplification. Some formulations create one projection head per label or hierarchy level (Ghanooni et al., 4 Feb 2025); some construct per-label token-level embeddings inside an episodic prototypical learner (Liu et al., 2022); some preserve label structure through nuclear-norm geometry under missing labels (Ma et al., 2022); some replace pair mining with class-wise centers (Cai et al., 2023); some move the objective into a Gaussian RKHS and model epistemic uncertainty (Sajedi et al., 2023); and some apply category-conditioned contrast to dense foreground/background decomposition or long-tailed multiscale sequence features (Zhang et al., 8 Aug 2025, Cui et al., 4 Sep 2025). This suggests that the acronym identifies a problem class and design philosophy more reliably than a fixed formula.

The practical guidance reported in the literature is correspondingly conditional. For MLCL, one projection head per label or hierarchy level is recommended; if $P_i^k$5 is large, rare labels may be grouped into a “global” head; $P_i^k$6 should be set inversely proportional to expected class granularity; $P_i^k$7 should reflect task importance; new categories can be handled by adding heads or fine-tuning existing ones; and projection heads are discarded at deployment (Ghanooni et al., 4 Feb 2025). For the broader multi-label setting, the comprehensive study recommends MCCL especially for large label spaces and data-scarce scenarios, while noting that standard BCE, ASY, or ZLPR often suffice when the label space is small and that ranking metrics may still favor ranking-oriented objectives (Audibert et al., 2024).

Several limitations and future directions are explicit in the current literature. KMCL notes an isotropic-versus-anisotropic trade-off, the possibility of embedding hierarchical or textual priors into $P_i^k$8, extensions beyond classification to detection, segmentation, or semi-supervised regimes, and exploration of other RKHS kernel families (Sajedi et al., 2023). More generally, the coexistence of pairwise, proxy-based, low-rank, category-conditioned dense, and multiscale long-tail formulations indicates that future work will likely continue to specialize MCCL to the structural constraints of each regime rather than converge immediately on a universal objective.