Conditional Contrastive Regularization

• Conditional Contrastive Regularization is a family of contrastive objectives that incorporate side information, such as pseudo-labels or sensitive attributes, to guide sample attraction and repulsion.
• It adapts contrastive pair construction and loss weighting by conditioning on variables like class labels, continuous targets, or auxiliary information, enhancing learning efficiency.
• Empirical results demonstrate that CCR improves performance, fairness, and robustness across semi-supervised, fairness-oriented, and imbalanced data tasks by refining representation geometry.

Searching arXiv for recent and foundational papers relevant to Conditional Contrastive Regularization. Conditional Contrastive Regularization (CCR) is best understood as a family of contrastive objectives in which attraction and repulsion are conditioned on side information such as class labels, pseudo-labels, sensitive attributes, auxiliary variables, continuous targets, or paired clean/noisy states. In the literature summarized here, closely related formulations appear as pseudo-label-conditioned contrastive regularization for semi-supervised learning (Lee et al., 2022), conditional supervised contrastive learning for equalized-odds fair text classification (Chi et al., 2022), Conditional Contrastive Learning for fair self-supervised learning (Ma et al., 2021), kernelized conditional contrastive objectives (Tsai et al., 2022), and target-conditioned regularization for deep imbalanced regression (Keramati et al., 2023). The acronym is also overloaded: “CCR” may denote Contrast-Consistent Ranking rather than Conditional Contrastive Regularization (Stoehr et al., 2023), Cross-lingual Cross-modal Retrieval (Nie et al., 2024), or SA-CCR/RSA-CCR in counterparty credit risk (Berrahoui et al., 2019).

1. Terminology and scope

The literature does not present Conditional Contrastive Regularization as a single canonical objective with a fixed notation. Instead, related papers use terms such as “contrastive regularization,” “Conditional Contrastive Learning,” “conditional supervised contrastive learning,” and task-specific names such as ConR and VeCoR (Lee et al., 2022, Ma et al., 2021, Chi et al., 2022, Keramati et al., 2023, Koo et al., 28 Jun 2026). This suggests that CCR is more accurately treated as a methodological pattern than as a single named algorithm.

Across these works, the common operation is to make the contrastive relation graph depend on a condition variable. In semi-supervised learning, the condition is a hard pseudo-label $\hat{q}_u=\arg\max q_u$ (Lee et al., 2022). In fairness-oriented text classification, the condition is the joint pair $(A,Y)$, where $A$ is a sensitive attribute and $Y$ the task label (Chi et al., 2022). In fair self-supervised learning, the condition is the sensitive attribute $Z$, and both positive and negative pairs are sampled within the same sensitive group (Ma et al., 2021). In conditional contrastive learning with kernels, the condition is a generic variable $Z$, which may be auxiliary information, sensitive information, or an embedding feature (Tsai et al., 2022). In deep imbalanced regression, the condition is the continuous target itself, operationalized through target similarity and prediction similarity (Keramati et al., 2023).

A persistent source of confusion is acronym collision. “CCR” in “Unsupervised Contrast-Consistent Ranking with LLMs” means Contrast-Consistent Ranking and is explicitly described as “not” Conditional Contrastive Regularization (Stoehr et al., 2023). “CCR” in multilingual retrieval means Cross-lingual Cross-modal Retrieval (Nie et al., 2024). “SA-CCR” and “RSA-CCR” belong to Basel counterparty credit risk methodology rather than contrastive representation learning (Berrahoui et al., 2019). Any encyclopedic treatment of CCR in machine learning therefore has to separate the conditional-contrastive family from these unrelated usages.

2. Common mathematical structure

Despite terminological variation, the representative formulations share a small set of ingredients: a condition variable, a rule for constructing positives and negatives or their weighted surrogates, and an auxiliary or replacement objective that reshapes representation geometry.

In pseudo-label-conditioned semi-supervised learning, the base objective is

$$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$

with $\mathcal{R}_{CR}$ defined over strongly augmented unlabeled samples and positives drawn from the same pseudo-label cluster (Lee et al., 2022). In equalized-odds fair text classification, the practical conditional supervised InfoNCE term is

$$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$

where the denominator is restricted to examples with the same $(a_i,y_i)$ (Chi et al., 2022). In fair self-supervised learning, the conditional contrastive objective replaces the unconditional product of marginals $(A,Y)$0 with the conditional product $(A,Y)$1, yielding a lower bound related to $(A,Y)$2 (Ma et al., 2021). In kernelized conditional contrastive learning, exact conditional sampling is replaced by

$(A,Y)$3

so that conditional similarity is estimated by a kernel-weighted use of all samples (Tsai et al., 2022). In continuous-label regression, ConR augments a supervised regression loss by

$(A,Y)$4

where positives are target-similar pairs and negatives are label-dissimilar but prediction-similar pairs (Keramati et al., 2023).

Representative paper	Condition variable	Conditional mechanism
(Lee et al., 2022)	Pseudo-label $(A,Y)$5	Same-pseudo-label positives, different-pseudo-label negatives
(Chi et al., 2022)	Joint $(A,Y)$6	Conditional supervised InfoNCE within the same $(A,Y)$7 subgroup
(Ma et al., 2021)	Sensitive attribute $(A,Y)$8	Positive and negative sampling within the same sensitive group
(Tsai et al., 2022)	Auxiliary/sensitive/embedding $(A,Y)$9	Kernel-weighted conditionalization of all samples
(Keramati et al., 2023)	Continuous target $A$0	Target-similar positives; target-dissimilar but prediction-similar negatives
(Koo et al., 28 Jun 2026)	Noisy-condition input with clean/noisy targets	Attraction to clean velocity and repulsion from noisy velocity

The main distinction across formulations is whether the condition changes pair construction, loss weighting, or the objective itself. Some methods are additive regularizers; others replace the conditional sampling rule inside InfoNCE; still others, such as VeCoR, use a difference of squared distances rather than an NT-Xent-style softmax (Koo et al., 28 Jun 2026). A common misconception is therefore to treat CCR as necessarily an additive penalty of the form $A$1. That description is accurate for some instances, but not for kernelized conditional contrastive learning, where the core intervention is a reformulation of the conditional objective rather than a standalone penalty (Tsai et al., 2022).

3. Semi-supervised and pseudo-label-conditioned formulations

The most direct semi-supervised example is “Contrastive Regularization for Semi-Supervised Learning,” which analyzes a limitation of confidence-thresholded consistency regularization: only examples satisfying $A$2 contribute to the unsupervised update, so label information is propagated only through confident pseudo-labels (Lee et al., 2022). The proposed contrastive regularization uses hard pseudo-labels as the condition variable, assigns strongly augmented samples to clusters by $A$3, uses same-pseudo-label samples as positives, and different-pseudo-label samples as negatives. An important asymmetry is that only anchors with confidence above $A$4 contribute a contrastive loss, while their positive and negative partners can include both high- and low-confidence examples (Lee et al., 2022).

The paper interprets the resulting geometry through gradients: same pseudo-label implies attraction and different pseudo-label implies repulsion. This was motivated as a way to propagate the information of confident pseudo-labels into more unlabeled samples during training by well-clustered features. The method uses a 2-layer MLP projection head, no momentum encoder, no queue or memory bank, and minibatch-level contrastive learning only (Lee et al., 2022).

Empirically, the method improves both efficiency and accuracy of consistency regularization. On CIFAR-100, a FixMatch+CR iteration is about $A$5 slower than FixMatch, but the total time to best performance is only $A$6 of FixMatch’s total time, and about $A$7 of FixMatch’s training time is enough for FixMatch+CR to reach FixMatch’s best performance (Lee et al., 2022). Representative gains reported in the data include SVHN with 20 labels, where FixMatch improves from $A$8 to $A$9, and CIFAR-10 with 20 labels, where it improves from $Y$0 to $Y$1 (Lee et al., 2022).

A related but structurally distinct formulation appears in multi-source UDA. CMSDA combines an interpolated supervised contrastive loss $Y$2 on labeled source data with confidence-thresholded target consistency regularization $Y$3 based on weak/strong augmentations and hard pseudo-labeling (Scalbert et al., 2021). The contrastive term is class-conditional on source labels, while the target branch uses a FixMatch-style mask $Y$4. The paper does not define CCR explicitly, but it is methodologically adjacent because it splits conditional structure into source-side contrastive alignment and target-side confident consistency regularization (Scalbert et al., 2021).

4. Fairness-oriented CCR

Fairness is the domain in which conditional contrastive regularization is most explicitly tied to a target invariance criterion. In “Conditional Supervised Contrastive Learning for Fair Text Classification,” the representation $Y$5 is trained so that equalized odds is approximated by reducing $Y$6, with $Y$7 the sensitive attribute and $Y$8 the task label (Chi et al., 2022). The paper proves the relation

$Y$9

then uses ordinary supervised contrastive learning to control the first term and conditional supervised InfoNCE to control the second (Chi et al., 2022). The resulting one-stage objective,

$Z$0

is one of the clearest CCR-style objectives in the supplied literature (Chi et al., 2022).

A closely related fairness formulation is “Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learning,” which conditions the contrastive task on the sensitive attribute $Z$1 itself (Ma et al., 2021). The key intervention is sampling both positive and negative pairs from the same gender, the same race, or the same age group. The paper shows that

$Z$2

and proposes a tighter variant, Tight-CCL, in which the critic also takes $Z$3 as input (Ma et al., 2021). In operational terms, this converts standard SimCLR into a condition-aware sampling scheme rather than an additive regularizer.

The empirical record in these fairness papers is unusually detailed. On Adult, SimCLR reports $Z$4 accuracy with $Z$5, $Z$6, and $Z$7, while CCL reports $Z$8, $Z$9, $Z$0, and $Z$1, respectively (Ma et al., 2021). Across the five tabular fairness datasets, the paper summarizes average improvement from SimCLR to CCL as $Z$2 on $Z$3, $Z$4 on $Z$5, and $Z$6 on $Z$7 (Ma et al., 2021). In fair text classification, two-stage CL and INLP are reported as generally providing the best fairness/performance trade-offs, while adversarial approaches are described as less stable and generally worse (Chi et al., 2022).

A recurring misconception is that fairness-oriented CCR necessarily removes the sensitive attribute completely. The cited papers are more precise: the objective is to reduce dependence on $Z$8 given $Z$9, or to make the sensitive attribute much less useful for solving the contrastive task, not to establish a universal guarantee of complete invariance in every deployment setting (Chi et al., 2022, Ma et al., 2021).

5. Kernelized, continuous-target, and task-specific extensions

When exact conditional sampling is sparse or ill-defined, the condition can be handled by smoothing rather than hard grouping. “Conditional Contrastive Learning with Kernel” replaces sampling from $$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$0 by a finite-sample kernel conditional embedding operator,

$$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$1

which induces the conditionalized similarity matrix $$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$2 (Tsai et al., 2022). The method is applied to weakly supervised contrastive learning, fair contrastive learning, and hard negative contrastive learning. Reported gains include UT-Zappos/CUB/ImageNet-100 top-1 accuracy improving from $$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$3 under WeaklySup$$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$4 to $$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$5 under WeaklySup$$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$6, and ColorMNIST accuracy improving from $$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$7 to $$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$8 while maintaining nearly identical fairness suppression (Tsai et al., 2022).

For continuous labels, ConR supplies a regression-specific answer to the fact that class-conditioned SupCon does not transfer naturally to non-discrete targets (Keramati et al., 2023). Its positive set $$\mathcal{L}(\mathcal{B}) = \mathcal{L}_{L}(\mathcal{X}) + \lambda_{CS} \mathcal{R}_{CS} (\mathcal{U}) + \lambda_{CR} \mathcal{R}_{CR} (\mathcal{U}),$$9 contains target-similar samples, while its negative set $\mathcal{R}_{CR}$0 contains samples with dissimilar labels but similar predictions. The negative weight $\mathcal{R}_{CR}$1 is proportional to a density-based pushing power and inversely related to target similarity, so minority targets are protected against collapse into majority neighborhoods (Keramati et al., 2023). Representative results include AgeDB-DIR few-shot MAE improvements of $\mathcal{R}_{CR}$2 over LDS, $\mathcal{R}_{CR}$3 over FDS, and $\mathcal{R}_{CR}$4 over RankSim; on NYUD2-DIR, average improvements in few-shot RMSE are $\mathcal{R}_{CR}$5 over LDS, $\mathcal{R}_{CR}$6 over FDS, and $\mathcal{R}_{CR}$7 over Balanced MSE (Keramati et al., 2023).

Several papers instantiate CCR-like ideas in specialized architectures without adopting the exact name. VeRe-Flow introduces velocity contrastive regularization,

$\mathcal{R}_{CR}$8

which attracts the predicted velocity toward the clean direction while repelling it from the noisy direction in conditional flow matching for noise-robust bandwidth expansion (Koo et al., 28 Jun 2026). InfoSCC-GAN uses a pretrained contrastive encoder and a pretrained classifier as a semantic regularizer for conditional generation, with generator regularization using external or internal attributes every $\mathcal{R}_{CR}$9-th iteration (Kinakh et al., 2021). In graph contrastive learning, Contrast-Reg is a calibration-motivated plug-in regularizer rather than a strict conditional method, but it addresses the same problem of semantically incorrect contrastive pairs by ensuring that decreasing the contrastive loss leads to better performance in downstream tasks (Ma et al., 2021).

6. Empirical profile, limitations, and recurrent misunderstandings

Across application areas, conditional contrastive regularization is typically reported as a training-time mechanism for shaping feature geometry rather than an inference-time modification. ConR is used during training only and adds no additional inference-time regularization computation (Keramati et al., 2023). The semi-supervised contrastive regularizer of (Lee et al., 2022) acts on normalized projection embeddings $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$0, whereas classification continues to use the usual penultimate features and classifier head. Fairness-oriented methods likewise learn a representation and then evaluate downstream performance and fairness after fine-tuning or linear probing (Chi et al., 2022, Ma et al., 2021).

The main empirical pattern is that conditioning improves robustness when the unconditional contrastive task has an identifiable shortcut. In SSL this shortcut is pseudo-label sparsity or confidence masking (Lee et al., 2022); in fair SSL it is the use of group membership to distinguish positives from negatives (Ma et al., 2021); in fair text classification it is residual dependence of $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$1 on $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$2 given $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$3 (Chi et al., 2022); in sparse conditional sampling it is the brittleness of exact matching on $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$4 (Tsai et al., 2022); and in imbalanced regression it is the collapse of minority-label examples toward majority-label neighbors in feature space (Keramati et al., 2023). This suggests that CCR is especially effective when the condition variable identifies a nuisance factor that standard contrastive training would otherwise exploit.

The limitations are equally consistent. Most methods require access to the condition variable during training. Fair self-supervised learning explicitly requires sensitive attributes during pretraining (Ma et al., 2021). Conditional supervised contrastive learning for fairness depends on having enough samples with the same $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$5 in a batch, and the paper notes that memory banks may be needed for intersectional or heavily imbalanced groups (Chi et al., 2022). Kernelized conditional objectives depend on kernel choice, bandwidth, and batchwise matrix inversion (Tsai et al., 2022). Pseudo-label-conditioned SSL uses hard $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$6-based conditioning and is vulnerable to pseudo-label quality (Lee et al., 2022). ConR depends on a task-appropriate similarity function $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$7 and a threshold $$L_{\text{CS-InfoNCE}} = -\sum_{i=1}^{2N} \frac{1}{N_{a_i,y_i}-1} \log(\ell_i),$$8, and for depth estimation it uses average depth as a practical but coarse proxy (Keramati et al., 2023).

A final misunderstanding is to equate all uses of the acronym “CCR.” In the machine-learning literature assembled here, the term “Conditional Contrastive Regularization” is best reserved for condition-aware contrastive geometry; it should not be conflated with Contrast-Consistent Ranking in language-model probing (Stoehr et al., 2023), Cross-lingual Cross-modal Retrieval (Nie et al., 2024), or SA-CCR/RSA-CCR in regulatory exposure measurement (Berrahoui et al., 2019). Within its proper domain, CCR denotes a broad design principle: use conditioning information to decide which similarities should be preserved, which discrepancies should be repelled, and which shortcuts should be denied to the representation learner.