NSCL: Negatives-only Supervised Contrastive Learning

• NSCL is defined as a supervised contrastive objective that removes same-class negatives while retaining same-instance positive pairs.
• The framework shows that self-supervised contrastive learning approximates NSCL with a loss gap diminishing as the number of classes increases, leading to superior clustering and performance.
• NSCL minimizers exhibit neural collapse with augmentation and within-class collapse, forming a simplex equiangular tight frame structure that underpins few-shot learning benefits.

Negatives-only Supervised Contrastive Learning (NSCL) is a supervised contrastive objective in which labels are used only to remove same-class examples from the contrastive denominator, while the numerator remains the ordinary same-instance augmentation pair. It therefore sits between self-supervised contrastive learning, which treats all other instances as negatives, and ordinary supervised contrastive learning, which turns same-class examples into explicit positives (Luthra et al., 4 Jun 2025). In the 2025 theory literature, NSCL became the central supervised surrogate used to explain why standard contrastive learning acquires semantic structure: the loss of self-supervised contrastive learning was shown to approximate NSCL as the number of semantic classes grows, and later work studied when this approximation extends from objective values to representation alignment during training (Luthra et al., 4 Jun 2025, Luthra et al., 9 Oct 2025).

1. Conceptual definition and scope

The defining feature of NSCL is that supervision is used only to determine which examples must not appear as negatives. In the formulation analyzed in 2025, same-class samples are neither explicit positives nor explicit negatives; the only positive term is the standard pair of augmented views of the same sample (Luthra et al., 4 Jun 2025). This is why the term “negatives-only” is precise but potentially misleading: it does not mean that the objective is a purely repulsive loss with no numerator, and it does not mean that all positive structure disappears. Rather, NSCL preserves the single-positive instance-discrimination structure of self-supervised contrastive learning and corrects the denominator by excluding same-class contrasts (Luthra et al., 9 Oct 2025).

This distinguishes NSCL from ordinary supervised contrastive learning. In standard supervised contrastive learning, all same-class samples are usually treated as positives and are actively pulled together. In NSCL, same-class samples are excluded from the negative pool but are not added as class-level positives (Luthra et al., 4 Jun 2025). The distinction is central because the 2025 papers argue that this weaker use of labels is the supervised objective most closely aligned with self-supervised contrastive learning, precisely because it preserves the instance-level geometry of contrastive learning while removing false negatives (Luthra et al., 9 Oct 2025).

A common misconception is to identify any negative-focused supervised contrastive method with NSCL. That characterization is too broad. Methods that reweight hard negatives, construct synthetic negatives, or partition negatives into domain-specific subsets remain distinct from NSCL if they still rely on explicit class-level or batch-level positive attraction beyond the same-instance view pair, or if they mix contrastive learning with cross-entropy in a joint objective (Long et al., 2023, Animesh et al., 2023).

2. Formal objective

The theoretical formulation starts from a labeled dataset

$$S = \{(x_i,y_i)\}_{i=1}^N \subset \mathcal X \times [C],$$

with a representation map

$f:\mathcal X \to \mathbb R^d.$

For each sample $x_i$, $K$ augmentations are formed,

$x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$

with embeddings

$z_i^l = f(\alpha_l(x_i)).$

Similarity is cosine similarity,

$$\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.$$

The global self-supervised decoupled contrastive loss studied in the theory is

$$\mathcal{L}^{\mathrm{DCL}}(f) ~=~ -\frac{1}{K^2N}\sum_{l_1,l_2=1}^{K}\sum_{i=1}^N \log \Bigg( \frac{\exp(\mathrm{sim}(z^{l_1}_i, z^{l_2}_i))} {\sum_{l_3=1}^K\sum_{j\in [N]\setminus \{i\}} \exp (\mathrm{sim}(z^{l_1}_i, z^{l_3}_j))} \Bigg),$$

where positives are two augmented views of the same sample, and all other instances appear in the denominator (Luthra et al., 4 Jun 2025).

NSCL is defined by keeping the same numerator and replacing the denominator with different-class samples only: $$\mathcal{L}^{\mathrm{NSCL}}(f) ~=~ -\frac{1}{K^2N}\sum_{l_1,l_2=1}^{K}\sum_{i=1}^N \log \Biggl( \frac{\exp\bigl(\mathrm{sim}(z^{l_1}_i, z^{l_2}_i)\bigr)} {\sum_{l_3=1}^K \,\sum_{\substack{j=1 \ y_j \neq y_i}}^N \exp \bigl(\mathrm{sim}(z^{l_1}_i, z^{l_3}_j)\bigr)} \Biggr).$$ The phrase “negatives-only” therefore means that supervision only restricts the negative set to

$\{j : y_j \neq y_i\},$

without turning same-class samples into explicit positives (Luthra et al., 4 Jun 2025).

A minibatch version makes the same idea explicit. For a batch

$f:\mathcal X \to \mathbb R^d.$0

the NSCL negative subset for anchor $f:\mathcal X \to \mathbb R^d.$1 is

$f:\mathcal X \to \mathbb R^d.$2

and the per-anchor loss is

$f:\mathcal X \to \mathbb R^d.$3

The mathematical significance of this construction is that NSCL preserves the instance-level positive pair of self-supervised contrastive learning while using labels only to remove false negatives. This is the exact structural reason later alignment theory treats NSCL as the closest supervised bridge to self-supervised CL (Luthra et al., 9 Oct 2025).

3. Approximation of self-supervised contrastive learning

The first major theoretical result is that self-supervised contrastive learning approximately optimizes NSCL when the number of classes is large. For a labeled dataset with $f:\mathcal X \to \mathbb R^d.$4 classes, each containing at most $f:\mathcal X \to \mathbb R^d.$5 samples, the 2025 theorem states

$f:\mathcal X \to \mathbb R^d.$6

For a balanced classification problem,

$f:\mathcal X \to \mathbb R^d.$7

so the gap decays as

$f:\mathcal X \to \mathbb R^d.$8

(Luthra et al., 4 Jun 2025).

Three properties of this bound are emphasized in the paper. It is pointwise in representation space, because it holds for any function $f:\mathcal X \to \mathbb R^d.$9; it is architecture-independent, because it does not depend on a specific model class; and it is label-agnostic in a strong sense, because the upper bound depends only on $x_i$0 and $x_i$1, not on the semantic arrangement of the labels (Luthra et al., 4 Jun 2025). Later work summarizes the same conclusion as a uniform $x_i$2 CL–NSCL loss gap and uses it as the starting point for representation-level alignment analysis (Luthra et al., 9 Oct 2025).

The mechanism behind the approximation is denominator contamination. Self-supervised CL mistakenly includes same-class examples in the denominator, whereas NSCL excludes them. When the class count is large, same-class collisions become rare, so self-supervised CL behaves increasingly like NSCL (Luthra et al., 4 Jun 2025). This provides a theoretical explanation for why label-free contrastive learning can exhibit class-structured behavior even though class labels are not used during training.

A practical consequence, stated directly in the theory, is that minimizing the self-supervised DCL loss necessarily drives down the NSCL loss as well (Luthra et al., 4 Jun 2025). The papers do, however, distinguish objective-level proximity from optimizer equivalence: closeness of losses does not by itself prove closeness of minimizers in parameter space (Luthra et al., 4 Jun 2025).

4. Global minimizer geometry and few-shot structure

Once NSCL is isolated as the relevant supervised surrogate, the next question is the geometry of its global minimizers. In the unconstrained-features model, with balanced classes and ambient dimension $x_i$3, a global minimizer of NSCL satisfies three exact properties: augmentation collapse, within-class collapse, and a simplex equiangular tight frame structure for class centers (Luthra et al., 4 Jun 2025).

Augmentation collapse means that all augmentations of the same sample coincide: $x_i$4 Within-class collapse means that any two samples from the same class map to the same embedding: $x_i$5 If $x_i$6 are the class centers, they satisfy

$x_i$7

In the unit-norm specialization this becomes

$x_i$8

Thus NSCL minimizers exhibit a neural-collapse-like geometry even though the loss never explicitly instructs the model to pull all same-class examples together (Luthra et al., 4 Jun 2025).

This geometry is paired with a few-shot analysis built around class-distance-normalized variance and its directional refinement. The paper defines a directional variance term

$x_i$9

where $K$0 measures variation only along the line connecting class centers. The resulting few-shot bound is

$K$1

The interpretation given in the paper is that directional variation dominates the bound, while the effect of full within-class dispersion is suppressed by $K$2 and therefore diminishes as the number of labeled samples increases (Luthra et al., 4 Jun 2025).

Empirically, direct NSCL optimization yields tighter clustering than DCL. For 100-shot test accuracy, the reported values are: CIFAR10, DCL $K$3 with NCCC and $K$4 with linear probing versus NSCL $K$5 and $K$6; CIFAR100, DCL $K$7 and $K$8 versus NSCL $K$9 and $x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$0; mini-ImageNet, DCL $x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$1 and $x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$2 versus NSCL $x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$3 and $x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$4 (Luthra et al., 4 Jun 2025). The theoretical reading is that self-supervised CL approximates the NSCL objective closely enough to inherit useful class structure, but direct NSCL supervision produces stronger semantic collapse and separation.

5. Representation-level alignment with self-supervised CL

A later 2025 paper asks whether CL and NSCL remain aligned not only at the loss level but also at the representation level during training under shared randomness, meaning the same initialization, the same mini-batches, and the same augmentations (Luthra et al., 9 Oct 2025). The comparison is made in similarity space, using matrices

$x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$5

rather than solely in parameter space.

The key intermediate quantity is the softmax reweighting gap

$x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$6

On the corresponding high-probability batch-composition event, the paper proves that the CL and NSCL softmaxes satisfy

$x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$7

and that the similarity-space gradient gap is controlled by a fresh discrepancy term plus a stability term (Luthra et al., 9 Oct 2025).

The main qualitative conclusions are explicit. Representation alignment improves with more classes, improves with higher temperature, and depends on batch size through both concentration and learning-rate scaling (Luthra et al., 9 Oct 2025). By contrast, parameter-space coupling can be unstable: the paper proves that the weight difference can grow exponentially with training time even while representational similarity remains high. In the coupled SGD analysis,

$x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$8

so parameter divergence is not ruled out even when similarity matrices remain close (Luthra et al., 9 Oct 2025).

The empirical validation is organized around CKA and RSA. On Tiny-ImageNet after $x_i^l = \alpha_l(x_i), \qquad l=1,\dots,K,$9k epochs, the paper reports

$z_i^l = f(\alpha_l(x_i)).$0

whereas

$z_i^l = f(\alpha_l(x_i)).$1

Across datasets, NSCL tracks CL more closely than supervised contrastive learning or cross-entropy, alignment strengthens with the number of classes, and higher temperatures such as $z_i^l = f(\alpha_l(x_i)).$2 yield stronger CKA and RSA alignment than lower temperatures such as $z_i^l = f(\alpha_l(x_i)).$3 (Luthra et al., 9 Oct 2025). The paper’s stated conclusion is that NSCL is a principled bridge between self-supervised and supervised learning.

These results clarify the role of NSCL in the theory of contrastive learning. It is not introduced merely as an alternative supervised loss; it is used as the supervised object that best preserves the single-positive, instance-discriminative structure of CL while correcting denominator false negatives (Luthra et al., 9 Oct 2025).

6. Negative-set design, adjacent methods, and boundary cases

NSCL is a precise objective, but it sits within a larger literature on negative-sample design. A 2022 review organizes negative sampling methods into four broad categories—static negative sampling, dynamic negative sampling, adversarial negative sampling, and efficient negative sampling—and treats the negative sampling distribution $z_i^l = f(\alpha_l(x_i)).$4 as the central design variable in contrastive representation learning (Xu et al., 2022). That framework transfers directly to NSCL: once labels identify the admissible negative pool, the main remaining problem is how to choose or weight negatives within that pool.

A separate empirical study of contrastive instance discrimination provides a sharp hardness result: the hardest $z_i^l = f(\alpha_l(x_i)).$5 of negatives were both necessary and sufficient for nearly full downstream accuracy, the easiest $z_i^l = f(\alpha_l(x_i)).$6 were unnecessary and insufficient, and the very hardest $z_i^l = f(\alpha_l(x_i)).$7 could be detrimental (Cai et al., 2020). This suggests that NSCL should not be understood as mere exclusion of same-class negatives; its performance is likely to depend on the geometry, hardness, and contamination of the remaining different-class set.

Paper	Negative-side mechanism	Relation to NSCL
"Rehabilitation Exercise Quality Assessment through Supervised Contrastive Learning with Hard and Soft Negatives" (Karlov et al., 2024)	hard negatives are same-exercise incorrect samples; soft negatives are other exercise types	not NSCL; positives remain explicit and anchors are only correct samples
"When hard negative sampling meets supervised contrastive learning" (Long et al., 2023)	similarity-based hard-negative weight $z_i^l = f(\alpha_l(x_i)).$8 in the denominator, mixed with CE	negative-focused supervised contrastive learning, not negatives-only
"Synthetic Hard Negative Samples for Contrastive Learning" (Dong et al., 2023)	top-$z_i^l = f(\alpha_l(x_i)).$9 hard-negative mining, feature-space interpolation, and debiasing	self-supervised, but largely a redesign of the negative aggregate
"Semantically-Conditioned Negative Samples for Efficient Contrastive Learning" (Neill et al., 2021)	top-$$\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.$$0 semantically similar classes, top-$$\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.$$1 similar samples, and latent pseudo-negatives	positive-and-negative InfoNCE with semantically conditioned negative mining
"Clustering-based hard negative sampling for supervised contrastive speaker verification" (Masztalski et al., 23 Jul 2025)	clustering of speaker centroids and hard/easy ratio control in batch construction	supervised contrastive, not NSCL, but shows that negative batch composition can dominate performance

The main conceptual boundary is therefore clear. Negative-focused methods are not automatically NSCL. SCHaNe preserves the standard supervised contrastive positive numerator and combines its loss with cross-entropy (Long et al., 2023). TCL preserves positive attraction and introduces denominator terms controlled by $$\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.$$2 and $$\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.$$3, with the hard-negative response increasing monotonically with $$\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.$$4 (Animesh et al., 2023). The rehabilitation exercise framework with hard and soft negatives still relies on standard positive attraction among correct same-exercise samples and is therefore best characterized as supervised contrastive learning with specialized negative design rather than as NSCL (Karlov et al., 2024).

At the same time, adjacent work supplies implementation lessons that NSCL can inherit. Feature-space synthetic hard negatives, similarity-based weighting, and debiasing of false-negative contamination were developed in self-supervised form by SSCL (Dong et al., 2023). Semantically conditioned top-$$\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.$$5 negative sampling and latent pseudo-negatives were developed for supervised and distillation settings by SCNS (Neill et al., 2021). In robust contrastive learning with negatives of diminished semantics, same-source negatives were explicitly constructed to preserve superficial cues while weakening semantics, showing that the semantics of the negative distribution can change robustness and shape bias (Ge et al., 2021). In speaker verification, CHNS changes only the sampling of negatives, not the loss, yet reports up to $$\mathrm{sim}(x_1,x_2)=\frac{\langle x_1,x_2\rangle}{\|x_1\|_2\cdot \|x_2\|_2}.$$6 relative EER and minDCF improvement on VoxCeleb, reinforcing the claim that the composition of the negative set can dominate learning dynamics (Masztalski et al., 23 Jul 2025).

Taken together, these results delimit what NSCL is and what it is not. Strictly speaking, NSCL is the supervised contrastive objective that excludes same-class examples from the denominator while retaining the ordinary same-instance positive pair (Luthra et al., 4 Jun 2025, Luthra et al., 9 Oct 2025). More broadly, the surrounding literature shows that once this supervised negative set is defined, the crucial remaining questions concern hardness, semantic proximity, false-negative control, tail robustness, and batch or queue construction (Xu et al., 2022, Cai et al., 2020). In that broader ecosystem, NSCL is the canonical supervised objective; negative engineering methods are the mechanisms that determine how informative its denominator becomes.