---
title: 'NSCL: Negatives-only Supervised Contrastive Learning'
url: https://www.emergentmind.com/topics/negatives-only-supervised-contrastive-learning-nscl
type: topic
---

# NSCL: Negatives-only Supervised Contrastive Learning

Negatives-only Supervised Contrastive Learning (NSCL) is a supervised contrastive objective in which labels are used only to remove same-class examples from the contrastive denominator, while the numerator remains the ordinary same-instance augmentation pair. It therefore sits between self-supervised contrastive learning, which treats all other instances as negatives, and ordinary supervised contrastive learning, which turns same-class examples into explicit positives [2506.04411]. In the 2025 theory literature, NSCL became the central supervised surrogate used to explain why standard contrastive learning acquires semantic structure: the loss of self-supervised contrastive learning was shown to approximate NSCL as the number of semantic classes grows, and later work studied when this approximation extends from objective values to representation alignment during training [2506.04411][2510.08852].

## 1. Conceptual definition and scope

The defining feature of NSCL is that supervision is used only to determine which examples must **not** appear as negatives. In the formulation analyzed in 2025, same-class samples are neither explicit positives nor explicit negatives; the only positive term is the standard pair of augmented views of the *same sample* [2506.04411]. This is why the term “negatives-only” is precise but potentially misleading: it does **not** mean that the objective is a purely repulsive loss with no numerator, and it does **not** mean that all positive structure disappears. Rather, NSCL preserves the single-positive instance-discrimination structure of self-supervised contrastive learning and corrects the denominator by excluding same-class contrasts [2510.08852].

This distinguishes NSCL from ordinary supervised contrastive learning. In standard supervised contrastive learning, all same-class samples are usually treated as positives and are actively pulled together. In NSCL, same-class samples are excluded from the negative pool but are not added as class-level positives [2506.04411]. The distinction is central because the 2025 papers argue that this weaker use of labels is the supervised objective most closely aligned with self-supervised contrastive learning, precisely because it preserves the instance-level geometry of contrastive learning while removing false negatives [2510.08852].

A common misconception is to identify any negative-focused supervised contrastive method with NSCL. That characterization is too broad. Methods that reweight hard negatives, construct synthetic negatives, or partition negatives into domain-specific subsets remain distinct from NSCL if they still rely on explicit class-level or batch-level positive attraction beyond the same-instance view pair, or if they mix contrastive learning with cross-entropy in a joint objective [2308.14893][2305.10675].

## 2. Formal objective

The theoretical formulation starts from a labeled dataset
\[
S = \{(x_i,y_i)\}_{i=1}^N \subset \mathcal X \times [C],
\]
with a representation map
\[
f:\mathcal X \to \mathbb R^d.
\]
For each sample \(x_i\), \(K\) augmentations are formed,
\[
x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,
\]
with embeddings
\[
z_i^l = f(\alpha_l(x_i)).
\]
Similarity is cosine similarity,
\[
\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.
\]
The global self-supervised decoupled contrastive loss studied in the theory is
\[
\mathcal{L}^{\mathrm{DCL}}(f) ~=~ -\frac{1}{K^2N}\sum_{l_1,l_2=1}^{K}\sum_{i=1}^N \log \Bigg( \frac{\exp(\mathrm{sim}(z^{l_1}_i, z^{l_2}_i))} {\sum_{l_3=1}^K\sum_{j\in [N]\setminus \{i\}} \exp (\mathrm{sim}(z^{l_1}_i, z^{l_3}_j))} \Bigg),
\]
where positives are two augmented views of the same sample, and all other instances appear in the denominator [2506.04411].

NSCL is defined by keeping the same numerator and replacing the denominator with different-class samples only:
\[
\mathcal{L}^{\mathrm{NSCL}}(f) ~=~ -\frac{1}{K^2N}\sum_{l_1,l_2=1}^{K}\sum_{i=1}^N \log \Biggl( \frac{\exp\bigl(\mathrm{sim}(z^{l_1}_i, z^{l_2}_i)\bigr)} {\sum_{l_3=1}^K \,\sum_{\substack{j=1 \\ y_j \neq y_i}}^N \exp \bigl(\mathrm{sim}(z^{l_1}_i, z^{l_3}_j)\bigr)} \Biggr).
\]
The phrase “negatives-only” therefore means that supervision only restricts the negative set to
\[
\{j : y_j \neq y_i\},
\]
without turning same-class samples into explicit positives [2506.04411].

A minibatch version makes the same idea explicit. For a batch
\[
\mathcal B=\{(x_{j_t},x'_{j_t},y_{j_t})\}_{t=1}^B,
\]
the NSCL negative subset for anchor \(i\) is
\[
\mathcal B_i^-:=\{(x_{j_t},x'_{j_t},y_{j_t})\in\mathcal B:\; y_{j_t}\neq y_i\},
\]
and the per-anchor loss is
\[
\ell_{i,\mathcal B_i^-}(w) := -\log \left( \frac{\exp\bigl(\mathrm{sim}(f_w(x_i),f_w(x'_i))/\tau\bigr)} {\sum_{(x_j,x'_j,y_j)\in \mathcal B_i^-} \Bigl[\exp\bigl(\mathrm{sim}(f_w(x_i),f_w(x_j))/\tau\bigr) +\exp\bigl(\mathrm{sim}(f_w(x_i),f_w(x'_j))/\tau\bigr)\Bigr]} \right) [2510.08852].
\]

The mathematical significance of this construction is that NSCL preserves the instance-level positive pair of self-supervised contrastive learning while using labels only to remove false negatives. This is the exact structural reason later alignment theory treats NSCL as the closest supervised bridge to self-supervised CL [2510.08852].

## 3. Approximation of self-supervised contrastive learning

The first major theoretical result is that self-supervised contrastive learning approximately optimizes NSCL when the number of classes is large. For a labeled dataset with \(C\) classes, each containing at most \(n_{\max}\) samples, the 2025 theorem states
\[
\mathcal{L}^{\mathrm{NSCL}}(f) ~\leq~ \mathcal{L}^{\mathrm{DCL}}(f) ~\leq~ \mathcal{L}^{\mathrm{NSCL}}(f) + \log\left(1+\tfrac{n_{\max} \mathrm{e}^2}{N-n_{\max}}\right) ~\leq~ \mathcal{L}^{\mathrm{NSCL}}(f) + \tfrac{n_{\max} \mathrm{e}^2}{N-n_{\max}}.
\]
For a balanced classification problem,
\[
\tfrac{n_{\max}}{N-n_{\max}}=\tfrac{1}{C-1},
\]
so the gap decays as
\[
\mathcal O\!\left(\frac{1}{C}\right)
\]
[2506.04411].

Three properties of this bound are emphasized in the paper. It is **pointwise in representation space**, because it holds for any function \(f\); it is **architecture-independent**, because it does not depend on a specific model class; and it is **label-agnostic in a strong sense**, because the upper bound depends only on \(N\) and \(n_{\max}\), not on the semantic arrangement of the labels [2506.04411]. Later work summarizes the same conclusion as a uniform \(\mathcal O(1/C)\) CL–NSCL loss gap and uses it as the starting point for representation-level alignment analysis [2510.08852].

The mechanism behind the approximation is denominator contamination. Self-supervised CL mistakenly includes same-class examples in the denominator, whereas NSCL excludes them. When the class count is large, same-class collisions become rare, so self-supervised CL behaves increasingly like NSCL [2506.04411]. This provides a theoretical explanation for why label-free contrastive learning can exhibit class-structured behavior even though class labels are not used during training.

A practical consequence, stated directly in the theory, is that minimizing the self-supervised DCL loss necessarily drives down the NSCL loss as well [2506.04411]. The papers do, however, distinguish objective-level proximity from optimizer equivalence: closeness of losses does not by itself prove closeness of minimizers in parameter space [2506.04411].

## 4. Global minimizer geometry and few-shot structure

Once NSCL is isolated as the relevant supervised surrogate, the next question is the geometry of its global minimizers. In the unconstrained-features model, with balanced classes and ambient dimension \(d \ge C-1\), a global minimizer of NSCL satisfies three exact properties: **augmentation collapse**, **within-class collapse**, and a **simplex equiangular tight frame** structure for class centers [2506.04411].

Augmentation collapse means that all augmentations of the same sample coincide:
\[
z_i^{l_1} = z_i^{l_2}.
\]
Within-class collapse means that any two samples from the same class map to the same embedding:
\[
z_i = z_j \qquad \text{whenever } y_i = y_j.
\]
If \(\{\mu_1,\dots,\mu_C\}\) are the class centers, they satisfy
\[
\sum_{c=1}^{C} \mu_c = 0,
\qquad
\|\mu_c\|_2 = \|\mu_{c'}\|_2,
\qquad
\langle \mu_c, \mu_{c'} \rangle = -\frac{\|\mu_c\|_2^2}{C-1}
\quad \forall c\neq c'.
\]
In the unit-norm specialization this becomes
\[
\|\mu_c\|_2 = 1, \qquad \sum_{c=1}^{C}\mu_c = 0, \qquad \langle \mu_c,\mu_{c'}\rangle = -\frac{1}{C-1}.
\]
Thus NSCL minimizers exhibit a neural-collapse-like geometry even though the loss never explicitly instructs the model to pull all same-class examples together [2506.04411].

This geometry is paired with a few-shot analysis built around class-distance-normalized variance and its directional refinement. The paper defines a directional variance term
\[
\tilde{V}_{f}(D_i,D_j)=\frac{\sigma_{ij}^{2}}{\|\mu_i-\mu_j\|_2^{2}},
\]
where \(\sigma_{ij}^{2}\) measures variation only along the line connecting class centers. The resulting few-shot bound is
\[
\mathrm{err}^{\mathrm{LP}}_{m,D}(f) ~\le~ \mathrm{err}^{\mathrm{NCC}}_{m,D}(f) ~\leq~ (C'-1)\left[14\textnormal{Avg}_{i\neq j}[\tilde{V}_f(D_i,D_j)] + \tfrac{28.5}{m}\textnormal{Avg}_{i\neq j}[V_{f}(D_i,D_j)] \right].
\]
The interpretation given in the paper is that **directional variation dominates** the bound, while the effect of full within-class dispersion is suppressed by \(1/m\) and therefore diminishes as the number of labeled samples increases [2506.04411].

Empirically, direct NSCL optimization yields tighter clustering than DCL. For 100-shot test accuracy, the reported values are: CIFAR10, DCL \(85.3\) with NCCC and \(86.3\) with linear probing versus NSCL \(95.7\) and \(95.6\); CIFAR100, DCL \(57.3\) and \(61.7\) versus NSCL \(70.8\) and \(73.7\); mini-ImageNet, DCL \(69.0\) and \(72.9\) versus NSCL \(79.8\) and \(81.3\) [2506.04411]. The theoretical reading is that self-supervised CL approximates the NSCL objective closely enough to inherit useful class structure, but direct NSCL supervision produces stronger semantic collapse and separation.

## 5. Representation-level alignment with self-supervised CL

A later 2025 paper asks whether CL and NSCL remain aligned not only at the loss level but also at the representation level during training under shared randomness, meaning the same initialization, the same mini-batches, and the same augmentations [2510.08852]. The comparison is made in similarity space, using matrices
\[
\Sigma_t^{CL},\Sigma_t^{NSCL}\in[-1,1]^{N\times N},
\]
rather than solely in parameter space.

The key intermediate quantity is the softmax reweighting gap
\[
\Delta_{C,\delta}(B;\tau) := \frac{2\,\mathrm e^{2/\tau} \left(\tfrac{1}{C}+\epsilon_{B,\delta}\right)} {1-\tfrac{1}{C}-\epsilon_{B,\delta}},
\qquad
\epsilon_{B,\delta} := \sqrt{\frac{1}{2B}\log\Bigl(\frac{TB}{\delta}\Bigr)}.
\]
On the corresponding high-probability batch-composition event, the paper proves that the CL and NSCL softmaxes satisfy
\[
\|p-q\|_1 ~\le~ \Delta_{C,\delta}(B;\tau),
\qquad
\|p-q\|_2 ~\le~ \Delta_{C,\delta}(B;\tau),
\]
and that the similarity-space gradient gap is controlled by a fresh discrepancy term plus a stability term [2510.08852].

The main qualitative conclusions are explicit. Representation alignment **improves with more classes**, **improves with higher temperature**, and depends on batch size through both concentration and learning-rate scaling [2510.08852]. By contrast, parameter-space coupling can be unstable: the paper proves that the weight difference can grow exponentially with training time even while representational similarity remains high. In the coupled SGD analysis,
\[
e_T ~\le~ \frac{G}{\beta\,\tau}\,\Delta_{C,\delta}(B;\tau)\, \Bigl(\exp\Bigl(\beta\sum_{t=0}^{T-1}\eta_t\Bigr)-1\Bigr),
\]
so parameter divergence is not ruled out even when similarity matrices remain close [2510.08852].

The empirical validation is organized around CKA and RSA. On Tiny-ImageNet after \(1\)k epochs, the paper reports
\[
\mathrm{CKA}(\mathrm{CL},\mathrm{NSCL}) \approx 0.87,
\]
whereas
\[
\mathrm{CKA}(\mathrm{CL},\mathrm{SCL}) \approx 0.043.
\]
Across datasets, NSCL tracks CL more closely than supervised contrastive learning or cross-entropy, alignment strengthens with the number of classes, and higher temperatures such as \(\tau=1.0\) yield stronger CKA and RSA alignment than lower temperatures such as \(0.1\) [2510.08852]. The paper’s stated conclusion is that NSCL is a principled bridge between self-supervised and supervised learning.

These results clarify the role of NSCL in the theory of contrastive learning. It is not introduced merely as an alternative supervised loss; it is used as the supervised object that best preserves the single-positive, instance-discriminative structure of CL while correcting denominator false negatives [2510.08852].

## 6. Negative-set design, adjacent methods, and boundary cases

NSCL is a precise objective, but it sits within a larger literature on negative-sample design. A 2022 review organizes negative sampling methods into four broad categories—**static negative sampling, dynamic negative sampling, adversarial negative sampling, and efficient negative sampling**—and treats the negative sampling distribution \(p(y^-)\) as the central design variable in contrastive representation learning [2206.00212]. That framework transfers directly to NSCL: once labels identify the admissible negative pool, the main remaining problem is how to choose or weight negatives within that pool.

A separate empirical study of contrastive instance discrimination provides a sharp hardness result: the hardest \(5\%\) of negatives were both necessary and sufficient for nearly full downstream accuracy, the easiest \(95\%\) were unnecessary and insufficient, and the very hardest \(0.1\%\) could be detrimental [2010.06682]. This suggests that NSCL should not be understood as mere exclusion of same-class negatives; its performance is likely to depend on the geometry, hardness, and contamination of the remaining different-class set.

| Paper | Negative-side mechanism | Relation to NSCL |
|---|---|---|
| "Rehabilitation Exercise Quality Assessment through Supervised Contrastive Learning with Hard and Soft Negatives" [2403.02772] | hard negatives are same-exercise incorrect samples; soft negatives are other exercise types | not NSCL; positives remain explicit and anchors are only correct samples |
| "When hard negative sampling meets supervised contrastive learning" [2308.14893] | similarity-based hard-negative weight \(\beta\) in the denominator, mixed with CE | negative-focused supervised contrastive learning, not negatives-only |
| "Synthetic Hard Negative Samples for Contrastive Learning" [2304.02971] | top-\(s\) hard-negative mining, feature-space interpolation, and debiasing | self-supervised, but largely a redesign of the negative aggregate |
| "Semantically-Conditioned Negative Samples for Efficient Contrastive Learning" [2102.06603] | top-\(k\) semantically similar classes, top-\(k\) similar samples, and latent pseudo-negatives | positive-and-negative InfoNCE with semantically conditioned negative mining |
| "Clustering-based hard negative sampling for supervised contrastive speaker verification" [2507.17540] | clustering of speaker centroids and hard/easy ratio control in batch construction | supervised contrastive, not NSCL, but shows that negative batch composition can dominate performance |

The main conceptual boundary is therefore clear. Negative-focused methods are not automatically NSCL. SCHaNe preserves the standard supervised contrastive positive numerator and combines its loss with cross-entropy [2308.14893]. TCL preserves positive attraction and introduces denominator terms controlled by \(k_1\) and \(k_2\), with the hard-negative response increasing monotonically with \(k_2\) [2305.10675]. The rehabilitation exercise framework with hard and soft negatives still relies on standard positive attraction among correct same-exercise samples and is therefore best characterized as supervised contrastive learning with specialized negative design rather than as NSCL [2403.02772].

At the same time, adjacent work supplies implementation lessons that NSCL can inherit. Feature-space synthetic hard negatives, similarity-based weighting, and debiasing of false-negative contamination were developed in self-supervised form by SSCL [2304.02971]. Semantically conditioned top-\(k\) negative sampling and latent pseudo-negatives were developed for supervised and distillation settings by SCNS [2102.06603]. In robust contrastive learning with negatives of diminished semantics, same-source negatives were explicitly constructed to preserve superficial cues while weakening semantics, showing that the semantics of the negative distribution can change robustness and shape bias [2110.14189]. In speaker verification, CHNS changes only the sampling of negatives, not the loss, yet reports up to \(18\%\) relative EER and minDCF improvement on VoxCeleb, reinforcing the claim that the composition of the negative set can dominate learning dynamics [2507.17540].

Taken together, these results delimit what NSCL is and what it is not. Strictly speaking, NSCL is the supervised contrastive objective that excludes same-class examples from the denominator while retaining the ordinary same-instance positive pair [2506.04411][2510.08852]. More broadly, the surrounding literature shows that once this supervised negative set is defined, the crucial remaining questions concern hardness, semantic proximity, false-negative control, tail robustness, and batch or queue construction [2206.00212][2010.06682]. In that broader ecosystem, NSCL is the canonical supervised objective; negative engineering methods are the mechanisms that determine how informative its denominator becomes.

Source: https://www.emergentmind.com/topics/negatives-only-supervised-contrastive-learning-nscl