De-bias Contrastive Learning Methods

• De-bias contrastive learning is a representation method that adjusts pair construction and sampling to reduce spurious bias while preserving task-relevant semantics.
• It employs techniques such as denominator correction, inverse-propensity sampling, and decoupled objectives to explicitly mitigate false negatives and other biases.
• Applications span computer vision, NLP, graph learning, and recommendation systems, enhancing fairness without compromising discriminative power.

De-bias contrastive learning denotes a family of representation-learning methods that modifies contrastive objectives, pair construction, or sampling so that embeddings become less sensitive to spurious correlations, sensitive attributes, sampling artifacts, or exposure effects while remaining discriminative for the target task. In the literature, the same basic contrastive template has been adapted to background bias in vision, word-frequency and sentence-length bias in sentence embedding, social bias in pre-trained LLMs, group fairness in tabular learning, degree bias in graph neural networks, exposure and popularity bias in recommender systems, and subgroup disparities in speech models (Chuang et al., 2020, Wang et al., 2022, Miao et al., 2023, Nielsen et al., 2024).

1. Problem setting and bias taxonomy

The central problem is that standard contrastive learning often treats pair construction as if it were neutral, when in fact the positive and negative sets can encode systematic bias. In self-supervised settings, randomly sampled negatives may include false negatives: semantically similar examples that should not be pushed apart. In supervised settings, noisy labels contaminate the positive set, and those mislabeled samples can appear as easy positives, reinforcing spurious attraction rather than correcting it. In domain-specific settings, the same issue appears through other nuisance factors: background context in images, lexical shortcuts in NLI, popularity in recommendation, or node degree in graphs (Chuang et al., 2020, Long et al., 2023, Lyu et al., 2022, Hu et al., 5 Jun 2025).

In computer vision, one prominent spurious correlation is background bias: convolutional networks can classify surprisingly well even when the foreground is removed, purely from background context, and suffer large drops when backgrounds are altered. CLAD was designed precisely for this setting, treating foreground content as semantics and background as the spurious factor to be neutralized (Wang et al., 2022). Related vision work extends this logic to contradicting-pair mixup, label-noise debiasing in supervised contrastive learning, and explicit spatial guidance from bias-contrastive pairs, all of which target the same failure mode: the encoder overfits to easier, non-causal cues (Hwang et al., 2022, Long et al., 2023, Park et al., 2024).

In NLP, bias enters through both data construction and social stereotypes. DebCSE diagnoses word frequency bias, sentence length bias, false negative sample bias, and a broader training-data distribution bias in unsupervised sentence embedding; DCT targets overlap bias, subsequence bias, and negwords bias in NLU; Co$^2$PT and CCPA use counterfactual pairs to reduce downstream and representational social bias; TriCon-Fair targets negative–positive coupling in social debiasing of pre-trained LLMs (Miao et al., 2023, Lyu et al., 2022, Dong et al., 2023, Li et al., 2023, Lyu et al., 2 Nov 2025).

In fairness-oriented settings, the bias variable is often explicit. FairContrast uses the sensitive attribute in tabular data to construct fairness-aware positives, and an attention-based fair contrastive framework conditions negative aggregation on sensitive attributes rather than predefined kernels. Earlier work on fair representations uses the same class label as an attraction signal and the same protected attribute as a repulsion signal (Tayebi et al., 2 Oct 2025, Nielsen et al., 2024, Shen et al., 2021).

Structured domains add further variants. SHARP and the HAR loss address degree bias in graph contrastive learning, where low-degree nodes receive insufficient and noisy information. In recommender systems, CLRec, Multi-CLRec, and DCLMDB treat exposure, popularity, conformity, and the Matthew effect as contrastive debiasing problems rather than purely ranking or reweighting problems (Hu et al., 5 Jun 2025, Zhou et al., 2020, Huang et al., 2024). In speech, CLUES targets underperforming subgroups by imposing task-level, subgroup-level, and error-within-subgroup contrastive structure (Koudounas et al., 2024).

2. Core mechanisms of debiased contrastive design

Across these works, de-bias contrastive learning is less a single loss than a design principle: positives are constructed to preserve task-relevant semantics while perturbing the biased factor, and negatives are constructed to preserve the biased factor while changing the semantics. CLAD states this principle explicitly: identify the spurious factor, construct positives that preserve semantic content while perturbing the spurious factor, construct negatives that preserve the spurious factor while changing semantic content, use InfoNCE to pull semantic-consistent pairs together and push spurious-consistent pairs apart, combine with supervised classification, and scale negatives with memory-efficient queues keyed by the spurious factor (Wang et al., 2022).

A second mechanism is explicit bias correction inside the denominator of the contrastive objective. Debiased Contrastive Learning formalizes false-negative correction by subtracting the expected same-label contribution from the negative pool. Its finite-sample estimator replaces the standard negative denominator with

$$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$

thereby correcting sampling bias without access to labels (Chuang et al., 2020). D-SCL applies the same logic to supervised contrastive learning under label noise through Positive–Unlabeled mixture decontamination and hard-aware importance sampling, because false positives dominate the harm in human label noise regimes (Long et al., 2023).

A third mechanism is propensity- or hardness-aware sampling. DebCSE uses inverse-propensity-inspired probabilities to sample positives that are low in surface similarity and high in semantic similarity, and negatives that are high in surface similarity and low in semantic similarity. HAR reweights positives and negatives according to learning hardness, explicitly enlarging the positive set for low-degree nodes. Attention-based fair contrastive learning uses learned attention over sensitive attributes to identify bias-reducing negatives, replacing predefined kernel conditioning (Miao et al., 2023, Hu et al., 5 Jun 2025, Nielsen et al., 2024).

A fourth mechanism is decoupling. TriCon-Fair argues that standard InfoNCE entangles attraction to positives and repulsion from negatives through a shared denominator, creating negative–positive coupling. It therefore uses separate logistic terms with independent margins for the positive and negative parts, combined with a language modeling objective to preserve utility (Lyu et al., 2 Nov 2025). A related decomposition appears in FairContrast, where the positive-pair policy induces an information bottleneck objective

$$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$

with $\lambda = 1-\pi$ determined by the cross-group pairing rate (Tayebi et al., 2 Oct 2025).

Method	Bias target	Core mechanism
DCL	False negatives in self-supervised CL	Correct the denominator using $\tau^+$ and $\tau^-$ (Chuang et al., 2020)
CLAD	Background bias in CNNs	Same-foreground positives, same-background negatives (Wang et al., 2022)
DebCSE	Sentence length and false-negative bias	Inverse-propensity-inspired positive/negative sampling (Miao et al., 2023)
FairContrast	Group fairness in tabular data	Cross-group positives for privileged favorable outcomes (Tayebi et al., 2 Oct 2025)
SHARP / HAR	Degree bias in GNNs	Label-expanded positives and hardness-adaptive reweighting (Hu et al., 5 Jun 2025)
CLRec	Exposure bias in recommendation	Queue-based contrastive learning equivalent to IPW (Zhou et al., 2020)

3. Vision and multimodal instantiations

In vision, CLAD is a canonical example because it operationalizes debiasing directly at the foreground–background interface. For image $x_i$ with foreground mask $M_i$, CLAD constructs the positive

$x_i^+ = M_i \odot x_i + (1 - M_i) \odot b_k,$

so that the same foreground object is placed on a different-class background, while negatives are sampled from a background-keyed dictionary so that they share the anchor’s background class but have varying foregrounds. The total objective is

$$L_{\mathrm{CLAD}} = L_{\mathrm{class}}(x_i) + \lambda L_{\mathrm{con}}(i),$$

or, in CLAD$$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$0,

$$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$1

On Background Challenge (ImageNet-9), CLAD$$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$2 achieved Original 95.6, Only-FG 94.6, Mixed-Rand 89.3, Mixed-Same 90.5, Only-BG-T 22.6, and BG-Gap 1.2, while CLAD achieved Original 95.9, Only-FG 93.8, Mixed-Rand 87.5, Mixed-Same 90.1, Only-BG-T 31.3, and BG-Gap 2.6. CLAD$$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$3 outperformed the prior best Mixed-Rand accuracy by 4.1% and maintained Original accuracy with at most a 0.4% drop versus base (Wang et al., 2022).

Other vision methods modify pair construction differently. SelecMix trains an auxiliary contrastive model to amplify biased features, then applies mixup to contradicting pairs: same label but dissimilar biased features, or different labels but similar biased features. On Colored MNIST at $$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$4, V+Ours reached 83.55% unbiased accuracy and L+Ours 82.80%, compared with 50.51% for Vanilla, 78.64% for LfF, and 80.20% for DFA. On BFFHQ at $$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$5, V+Ours reached 71.60% versus 56.20% for Vanilla and 65.60% for LfF (Hwang et al., 2022).

D-SCL addresses label noise within supervised contrastive learning rather than spurious context, but the mechanism remains contrastive debiasing. Its analysis shows that human labeling errors manifest as easy positive samples in around 99% of cases. On ImageNet-1k with BEiT-3 training from scratch, CE achieved 77.91, SupCon 82.57, and D-SCL 84.21. On CIFAR-100, CE achieved 59.67, SupCon 60.77, and D-SCL 64.47 (Long et al., 2023).

Multimodal sarcasm detection extends the same principle to text–image inputs. DMSD-CL constructs opposite-label counterfactuals with similar word bias and same-label counterfactuals with dissimilar word bias, then weights positives and negatives by a representation-level bias similarity proxy. On the OOD test with shifted word-label associations, DMSD-CL achieved Accuracy 70.25% and F1 67.03%, while the best baseline CMGCN achieved Accuracy 34.25% and F1 32.39%. Removing the adapted weighting reduced OOD Accuracy from 70.25% to 64.75%, and removing contrastive learning reduced it to 25.75% (Jia et al., 2023).

A related but distinct image-classification approach uses bias-contrastive pairs not to define an InfoNCE loss but to derive spatial guidance. It pairs a bias-aligned sample with a same-class bias-negative sample, identifies class-discerning common attributes, and amplifies under-exploited intrinsic regions. On Waterbirds, the method reported 63.64, 65.22, 65.23, and 66.33 across bias severities 0.5, 1.0, 2.0, and 5.0; on BFFHQ it reported 71.68, 77.56, 83.08, and 87.60; on BAR it reported 75.14 and 85.03 at 1.0 and 5.0 (Park et al., 2024).

4. Language, speech, and language-model alignment

In sentence representation learning, debiasing frequently means changing how unsupervised positives and negatives approximate supervised semantic structure. DCLR starts from SimCSE-style dropout positives, then punishes suspected false negatives using a complementary model and adds VAT-inspired noise-based negatives to improve uniformity. On seven STS tasks, DCLR improved average Spearman over SimCSE on all reported backbones: from 78.41 to 78.90 on BERT-base, from 76.57 to 77.87 on RoBERTa-base, and from 78.90 to 79.30 on BERT-large (Zhou et al., 2022).

DebCSE sharpens this diagnosis by treating unsupervised sentence embedding as a debiasing problem. It reports that supervised positives and negatives have surface similarity approximately 0.38 and 0.28, whereas standard contrastive positives and negatives are approximately 1.00 and 0.09. DebCSE uses inverse-propensity-weighted sampling over candidate pools defined by edit distance and cosine similarity, together with an alternative-normalization contrastive loss. On BERT-base it achieved an average of 80.33% across STS12–16, STS-B, and SICK-R, improving over unsupervised SimCSE at 76.25% by +4.08 points (Miao et al., 2023).

In NLU, DCT frames debiasing as feature-level contrastive selection. A bias-only TinyBERT identifies biased samples and dynamically selects the most similar biased negatives each epoch, while positives are chosen as least similar biased samples that share the label. On MNLI→HANS, BERT-base achieved 61.22 and DCT 68.30; on SNLI→Scramble, BERT-base achieved 72.74 and DCT 86.40; on FEVER→Symm v1, BERT-base achieved 56.53 and DCT 63.27. At the same time, ID performance remained essentially unchanged: MNLI dev 84.16 for BERT-base versus 84.19 for DCT (Lyu et al., 2022).

For social bias in LLMs, counterfactual contrastive strategies are common. Co$$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$6PT freezes the PLM and learns deep prompts so that original and counterfactual demographic variants are aligned during downstream prompt tuning. On Bias-STS-B, vanilla prompt tuning produced Diff. = 0.321, frac $$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$7, and frac $$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$8, whereas Co$$g(x, \{u_i\}_{i=1}^N, \{v_i\}_{i=1}^M) = \max \left\{ \frac{1}{\tau^-}\left(\frac{1}{N}\sum_{i=1}^N e^{f(x)^\top f(u_i)} - \tau^+\,\frac{1}{M}\sum_{i=1}^M e^{f(x)^\top f(v_i)} \right), e^{-1/t} \right\},$$9PT achieved Diff. = 0.058, frac $$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$0, and frac $$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$1, with Pearson/Spearman 0.884/0.880 (Dong et al., 2023). CCPA adopts a two-stage recipe—continuous prompt tuning to push counterfactual pairs farther apart, followed by contrastive learning to pull them closer—and reduced BERT SEAT average $$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$2 from 0.621 to 0.249 while improving StereoSet ICAT from 66.86 to 73.28 (Li et al., 2023).

Recent work has extended this logic to alignment tax and explicit social debiasing. The contrastive framework in “Paying Alignment Tax with Contrastive Learning” builds faithful positives and toxic, low-confidence, or entity-manipulated negatives, focuses the contrastive representation on named entities, and combines this with dynamic loss scaling. On Reddit TL;DR, GPT-2 moved from baseline to toxicity 0.041 ± 0.151 with a reduction of −0.007 and faithfulness 0.136 ± 0.225 with an increase of +0.018; Phi-2 achieved toxicity 0.053 ± 0.174 with reduction −0.014 and faithfulness 0.239 ± 0.307 with increase +0.195; Llama2-7B achieved toxicity 0.068 ± 0.195 with reduction −0.013 and faithfulness 0.325 ± 0.358 with increase +0.285 (Korkmaz et al., 25 May 2025). TriCon-Fair addresses the same general social-bias setting by replacing InfoNCE with a decoupled triplet objective and an LM loss. On GPT-2, it reported Gender SS 55.43, Race SS 57.33, Religion SS 58.31, LM Score 90.58, and ICAT 77.86, improving ICAT by +7.96 over the original 69.90 while keeping LM Score nearly unchanged (Lyu et al., 2 Nov 2025).

In speech, CLUES demonstrates that contrastive debiasing can target subgroup disparities directly rather than relying only on predefined protected groups. It applies Multi-Similarity loss at task, subgroup, and error-within-subgroup levels. On FSC, the baseline wav2vec 2.0 model had Acc 93.419 ± 0.169, F1 93.110 ± 0.168, and $$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$3, whereas CLUES with DivExplorer achieved Acc 98.789 ± 0.104, F1 98.761 ± 0.095, and $$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$4. On ITALIC, XLS-R-300 moved from F1 73.218 ± 0.329 and $$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$5 to F1 76.721 ± 0.201 and $$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$6 (Koudounas et al., 2024).

5. Fairness, graphs, and recommendation as debiasing by representation geometry

Fairness-oriented contrastive learning often makes the sensitive attribute explicit in pair selection. FairContrast is a representative tabular formulation: within subgroup pairing is used for most samples, but for anchors belonging to the privileged group with a favorable outcome, positives are drawn from the unprivileged group within the same outcome. The paper shows that minimizing InfoNCE with this pair policy is equivalent to maximizing

$$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$7

with $$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$8. On Adult, supervised FairContrast achieved accuracy 84.4% with DP = 0.0255, compared with Unfair MLP DP = 0.1855, SCARF DP = 0.1848, and CVIB DP = 0.1350. On German Credit, supervised FairContrast achieved accuracy 78% with DP = 0.0099 (Tayebi et al., 2 Oct 2025).

The attention-based fair contrastive framework FARE generalizes fairness conditioning to high-cardinality and high-dimensional sensitive information. Instead of hand-designed kernels, it uses queries and keys derived from sensitive attributes to reweight negative similarities:

$$\arg\max_{\theta} \left\{ I(Z;Y) - \lambda\,I(Z;S\mid Y) \right\},$$9

On ColorMNIST, SparseFARE achieved 86.4 ± 1.3 accuracy and 74.0 ± 3.8 bias removal, compared with InfoNCE at 84.1 ± 1.8 and 48.8 ± 4.5. On CelebA, SparseFARE achieved Acc 70.4 and EO 18.7, the best fairness result in the reported table (Nielsen et al., 2024). Earlier work on fair representations uses the simpler geometry of class cohesion and protected-attribute repulsion. On Moji, Con$\lambda = 1-\pi$0 achieved Accuracy 75.84%, GAP 13.92, h-leak 55.75, and logits-leak 55.32, compared with CE at Accuracy 72.09%, GAP 40.21, h-leak 85.75, and logits-leak 70.96 (Shen et al., 2021).

In graphs, the debiasing target is structural unfairness rather than demographic disparity. HAR augments the positive set with same-label nodes and reweights both positives and negatives by hardness. SHARP extends the method to semi-supervised settings via pseudo-labeling. With a GCN backbone at $\lambda = 1-\pi$1, SHARP reached 0.8700 on Cora, 0.7252 on CiteSeer, 0.8942 on PubMed, and 0.8193 on Wiki-CS. Degree-level gains were strongest on low-degree nodes: on Cora with GAT at $\lambda = 1-\pi$2, degree = 1 increased by +11.92% over the GAT baseline (Hu et al., 5 Jun 2025).

Recommendation systems provide a different but closely related interpretation. CLRec proves that an InfoNCE-style objective is equivalent to inverse propensity weighting when negatives are sampled from the exposure distribution, yielding a contrastive explanation of exposure debiasing. In Taobao deployment, aggregate diversity increased from 10,780,111 to 21,905,318, popularity index decreased from 0.658 to 0.224, and average dwell time increased by +11.9% (Zhou et al., 2020). DCLMDB extends contrastive debiasing in recommendation to dual biases—popularity and conformity—using a causal graph and two hinge triplet losses,

$\lambda = 1-\pi$3

combined with BPR. On Movielens-10M with an MF backbone, Recall@20 improved from 0.1276 to 0.1724 and Recall@50 from 0.2332 to 0.2948; on Netflix with LightGCN, Recall@20 improved from 0.1149 to 0.1442 and Recall@50 from 0.2009 to 0.2383 (Huang et al., 2024).

6. Evaluation criteria, trade-offs, and limitations

The evaluation of de-bias contrastive learning is heterogeneous because the bias variable changes by domain. Background debiasing uses Mixed-Rand accuracy and BG-Gap, with smaller BG-Gap indicating less reliance on background. Fairness-oriented tabular methods report Demographic Parity, Equalized Odds, Equal Opportunity, or disparate impact. Social-bias work on LLMs uses StereoSet SS and ICAT, CrowS-Pairs, Bias-NLI neutrality metrics, Bias-in-Bios TPR gaps, and leakage probes. Graph methods report global and degree-stratified F1. Recommendation methods emphasize Recall/HR/NDCG together with popularity index, aggregate diversity, IOU with popular items, or exposure disparity. Speech uses worst negative subgroup divergence $\lambda = 1-\pi$4 together with silhouette metrics for subgroup structure (Wang et al., 2022, Tayebi et al., 2 Oct 2025, Lyu et al., 2 Nov 2025, Hu et al., 5 Jun 2025, Zhou et al., 2020, Koudounas et al., 2024).

A recurrent empirical theme is that debiasing succeeds when the model is forced to separate semantic content from the nuisance factor without destroying task signal. CLAD reports feature cosine similarity 0.920 and decision consistency 0.969 for CLAD$\lambda = 1-\pi$5 between Original and Mixed-Rand pairs that share foreground but differ in background, compared with 0.795 and 0.800 for the IN9 baseline. DebCSE moves training-pair distributions toward supervised references and improves Uniformity and Alignment relative to unsupervised SimCSE. CLUES raises both subgroup silhouette $\lambda = 1-\pi$6 and correctness-partition silhouette $\lambda = 1-\pi$7 alongside fairness improvements (Wang et al., 2022, Miao et al., 2023, Koudounas et al., 2024).

The main trade-offs are also consistent across domains. Stronger contrastive weighting can over-regularize: in the alignment-tax study, $\lambda = 1-\pi$8 degrades both toxicity and faithfulness, and the best balance was at $\lambda = 1-\pi$9 (Korkmaz et al., 25 May 2025). In CLAD, $\tau^+$0 gave strong robustness with no drop on original accuracy, whereas larger $\tau^+$1 can degrade performance on original images (Wang et al., 2022). DebCSE reports that semantic-only or surface-only sampling is suboptimal, with the best average at $\tau^+$2 (Miao et al., 2023). D-SCL is designed for realistic human label noise regimes and is surpassed by highly specialized methods only at extreme synthetic noise such as 40% (Long et al., 2023).

Limitations are typically tied to the quality of the bias model or counterfactual generator. CLAD depends on approximate masks and can degrade under severe mask errors or when classes legitimately benefit from scene context. DebCSE depends on the quality of back-translation or summarization candidates and on surface/semantic features matching supervision. Co$\tau^+$3PT and CCPA depend on lexical counterfactual quality and are primarily demonstrated for binary gender. TriCon-Fair depends on curated counterfactual positives and generated hard biased negatives with quality filters. FARE requires access to sensitive attributes at training time. SHARP can be affected by pseudo-label noise under sparse labels. CLUES depends on subgroup assignment quality and incurs per-epoch clustering overhead (Wang et al., 2022, Miao et al., 2023, Dong et al., 2023, Lyu et al., 2 Nov 2025, Nielsen et al., 2024, Hu et al., 5 Jun 2025, Koudounas et al., 2024).

Taken together, these results establish de-bias contrastive learning as a unifying methodology rather than a single algorithm. The common operation is to redesign similarity structure so that the encoder is rewarded for invariance to the biased factor and penalized for encoding it as task-discriminative. The concrete instantiation varies—mixture correction for false negatives, counterfactual prompts, foreground–background swaps, attention over sensitive attributes, subgroup-aware Multi-Similarity losses, or queue-based propensity correction—but the underlying representation goal is the same: preserve $\tau^+$4 while suppressing the biased component that would otherwise dominate prediction (Chuang et al., 2020, Tayebi et al., 2 Oct 2025).