---
title: Fair Supervised Contrastive Loss (FSCL)
url: https://www.emergentmind.com/topics/fair-supervised-contrastive-loss-fscl
type: topic
---

# Fair Supervised Contrastive Loss (FSCL)

Fair Supervised Contrastive Loss (FSCL) is a family of loss formulations and training paradigms designed to address fairness concerns in supervised contrastive learning. FSCL explicitly modifies the sampling, metric structure, and/or regularization in supervised contrastive loss to prevent models from encoding or amplifying sensitive-attribute information and data biases in the learned representations. These losses have been instantiated across computer vision and graph learning domains, with several distinct but related formalizations. FSCL achieves both controlling for statistical/probabilistic fairness criteria (e.g., statistical parity, equal opportunity, group-conditional E/O) and maintaining discriminative accuracy across groups and classes.

## 1. Theoretical Underpinnings and Rationale

Standard supervised contrastive learning (SupCon, InfoNCE) pulls together representations of samples that share a target class, while pushing apart those from different classes. In the presence of data biases—spurious correlations between sensitive attributes (e.g., gender, age, color) and the target label—SupCon can minimize its objective by encoding sensitive-attribute information, inadvertently causing unfairness. If the dataset is demographically imbalanced, majority groups attain more compact and better-separated clusters than minority groups, further exacerbating group-wise disparities [2203.16209].

FSCL addresses these pathologies by explicitly constructing similarity comparisons and/or regularization such that encoding sensitive-attribute information does not reduce the loss, and by calibrating contributions across demographic groups to avoid unfair over-weighting. Some variants introduce additional margin-based or distribution-matching constraints to robustify against spurious bias signals [2211.05568].

## 2. Formal Definitions and Loss Formulations

Multiple formalizations of FSCL exist across tasks. Three representative formulations are:

| Paper / Domain                | Core FSCL Formulation                     | Key Mechanism                     |
|-------------------------------|--------------------------------------------|-----------------------------------|
| Image (vision) [2203.16209]   | Partition positives/negatives by (target, sensitive) label, restrict negatives to same-sensitive negatives | Penalizes encoding of sensitive attributes; group-wise normalization for fairness |
| Margin-based [2211.05568]     | $\epsilon$-SupInfoNCE with explicit margin; FairKL distance-distribution regularizer | Large positive/negative margin; match bias-aligned/conflicting distributions |
| Graph neural networks [2404.06090] | Contrastive loss on the "content" subspace, pulling same-label nodes together irrespective of sensitive group | Drives all nodes with the same label to cluster, erasing sensitive attribute from representations |

In vision tasks [2203.16209], for anchor embedding $z_i$ (target $y_i$, sensitive $s_i$):

\[
L_{\mathrm{FSCL}} = -\sum_{i=1}^{2N} \frac{1}{|Z_p(i)|} \sum_{p\in Z_p(i)}
\log
\frac{
    \exp(z_i \cdot z_p / \tau)
}{
    \sum_{n \in Z_{tg}(i)} \exp(z_i \cdot z_n / \tau)
}
\]

where positives $Z_p(i)$ include all (target, any-sensitive) matches, negatives $Z_{tg}(i)$ are different target but same-sensitive.

For margin-based FSCL [2211.05568], for anchor $x$, positives $x_i^+$, negatives $x_j^-$:

\[
\mathcal{L}_{\epsilon\text{-SupInfoNCE}} =
-\sum_{i=1}^P
\log
\frac{
    \exp(s_i^+)
}{
    \exp(s_i^+ - \epsilon) + \sum_{j=1}^N \exp(s_j^-)
}
\]
where $s_i^+ = f(x)^\top f(x_i^+)$, $s_j^- = f(x)^\top f(x_j^-)$.

A FairKL regularizer matches the empirical distributions (via KL divergence) of bias-aligned and bias-conflicting pairs, further neutralizing group disparities in distance.

In graph settings [2404.06090], the FSCL formulation only uses the classification-related projection ("content" subspace) and groups by current (pseudo-)labels only, thus actively discarding sensitive-attribute signals from this space.

## 3. Training Procedure and Hyperparameterization

The FSCL training workflow generally comprises:

1. **Batch construction**: For each anchor, construct positives (by target and/or sensitive labels) and precisely filtered negatives (e.g., same-sensitive, different-target).
2. **Contrastive computation**: Evaluate the FSCL loss as detailed above, possibly alongside margin or distribution matching terms.
3. **FairKL regularization (optional)**: Compute KL-divergence between bias-aligned/bias-conflicting distances for both positive and negative pairs; add as an explicit penalty [2211.05568].
4. **Group-wise normalization (FSCL+)**: Normalize contributions to the loss by group cardinalities over $(\text{class}, \text{sensitive})$ tuples, balancing intra-group compactness across demographics [2203.16209].
5. **Joint objective**: Sum FSCL with standard task losses (e.g., cross-entropy for classification) and other regularizers (e.g., invariance, environmental loss in GNNs [2404.06090]).
6. **Optimization**: Hyperparameters such as margin $\epsilon$ (vision: $[0.1, 0.5]$), weights for contrastive/fairness terms ($\alpha, \lambda$), and temperature $\tau$ (vision: $0.1$, GNN: $0.07$) are tuned by grid search or validated on held-out sets.

Variants also support incomplete supervision, using pseudo-labels or partial label strategies for target/sensitive assignments [2203.16209, 2404.06090].

## 4. Empirical Evaluation and Benchmarks

Empirical validation covers image classification, facial attribute learning, and tabular/graph data for node classification. Key datasets and evaluation criteria include:

- **Vision** [2203.16209, 2211.05568]: CelebA, UTK Face, CIFAR-10/100, ImageNet-100, Biased-MNIST, bFFHQ.
- **Fairness metrics**: Equalized odds (EO), statistical parity ($\Delta_{SP}$), and equal opportunity ($\Delta_{EO}$), in addition to standard accuracy/AUC.

Salient results:

- On CelebA, standard SupCon achieves $\sim$80% accuracy but EO $\approx$30%; FSCL achieves EO $\approx$11% at only 1.4% drop in accuracy, and FSCL+ cuts EO to $\approx$6% [2203.16209].
- On Biased-MNIST (color bias), FSCL achieves $\sim$90.5% vs. 11–60% for cross-entropy and LfF baselines [2211.05568].
- On tabular GNN benchmarks (German Credit, Bail, Credit Defaulter), FSCL in SCCAF yields both highest AUC/F1 and lowest statistical parity/equal opportunity disparities versus CAF and other fair GNN methods [2404.06090].

In all settings, FSCL closes accuracy/fairness tradeoffs beyond prior state-of-the-art methods such as GRL, LNL, FD-VAE, FairGNN, and EDITS.

## 5. Mechanisms for Fairness and Debiasing

FSCL mechanisms are united in their goal to prevent sensitive-attribute information from reducing the loss or contributing to downstream classification. The principal mechanisms are:

- **Negative sample restriction**: Constrains negatives so that they do not incentivize encoding sensitive-attribute signals.
- **Margin enforcement**: Requires a minimal positive-negative separation robust to spurious cues [2211.05568].
- **Distribution matching (FairKL)**: Normalizes and aligns distance distributions between bias-aligned and bias-conflicting samples, rendering them indistinguishable in representation space [2211.05568].
- **Group-wise normalization**: Ensures each $(\text{class},\text{sensitive})$ group is represented equally, controlling for demographic imbalance [2203.16209].
- **Content–environment disentanglement**: In GNNs, latent space is split to localize sensitive-attribute signals separately from task-relevant representations, enforced via environmental losses [2404.06090].

## 6. Practical Implementation and Limitations

Published FSCL implementations use standard neural encoders (e.g., ResNet-18 for vision; GCN or GraphSAGE for GNNs), and off-the-shelf optimizers (SGD, Adam). Loss and batch computation can be modularized to support new groupings and group-wise normalization. FSCL is robust to partial or noisy supervision, with explicit strategies for pseudo-labeling [2203.16209, 2404.06090].

Limitations include the need for sensitive-attribute annotations or accurate pseudo-labels, sensitivity to margin and weighting choices, and computational costs for large-batch or group-normalized operations. The method assumes explicit or inferable group structure.

## 7. Relation to Broader Research and Future Directions

FSCL generalizes classical contrastive fairness approaches by directly operationalizing the removal of sensitive-attribute information from learned representations while maintaining class discrimination. It positions itself relative to adversarial fairness learning (e.g., GRL), statistical-matching/penalty methods (e.g., FD-VAE), and recent counterfactual or invariance-driven graph learning approaches [2404.06090].

Future research directions include calibration for multi-class/multi-attribute sensitive variables, extension to self-supervised (unlabeled) or few-shot settings, and exploration of more theoretically grounded group fairness metrics. A plausible implication is that FSCL-style approaches could serve as a foundation for fairness certification in large-scale embedding models, as well as adapting to evolving notions of group/individual fairness over diverse modalities.

Source: https://www.emergentmind.com/topics/fair-supervised-contrastive-loss-fscl