Free-Negative Pairs in Self-Supervised Learning

• Free-negative pairs are negatives drawn from the same unlabeled dataset, offering a cost-free way to improve contrastive learning objectives.
• They leverage strategies like hard mining, false-negative correction, and augmentation-derived negatives to refine representation quality.
• Their practical use enhances training efficiency and accuracy, yielding modest performance gains on benchmarks such as ImageNet.

Searching arXiv for relevant papers on free-negative pairs and closely related negative-free / free-negatives methods. Free-negative pairs are negative pairs sampled from the same unlabeled dataset used for self-supervision, without requiring any external oracle, label, or human in the loop (Desai et al., 12 Feb 2025). They are “free” in that they cost zero annotation or privileged information beyond the raw images themselves. In the taxonomy of negative pair curation, they fall under the broader problem of choosing dissimilar pairs so that representation learning improves in quality, training efficiency, and computational cost. Across the literature, the term encompasses several mechanisms: direct reuse of other instances in a batch, dense spatial locations from other images, and augmentation-derived negatives. Closely related work also clarifies a boundary condition: free-negative methods still use negatives, whereas negative-free methods eliminate them entirely and replace the repulsive role of negatives with alternative alignment or uniformity mechanisms (Song et al., 2022, Liu et al., 2024).

1. Definition and taxonomic placement

In the survey taxonomy, free negatives fall under the broad heading of Negative Pair Curation, which is subdivided into three categories: Hard Negative Selection, Removal of False Negatives, and Synthetic Negative Generation (Desai et al., 12 Feb 2025). All three categories can be implemented “for free,” meaning without labels, but each imposes different costs and yields different benefits.

The defining property is not a particular loss function but the source of the negatives: they are drawn from the unlabeled corpus itself. In the standard batch-based setting, this means that the anchor’s negatives are typically the other instances in the batch. In more specialized settings, the same principle extends to spatial features extracted from other images, as in dense contrastive learning, or to deliberately perturbed variants of benign samples, as in intrusion detection (Iskender et al., 2022, Wilkie et al., 8 Sep 2025).

A common source of ambiguity is the distinction between “free-negative” and “negative-free.” The former uses negatives obtained without annotation cost; the latter removes negative pairs from the objective altogether. “Self-Supervised Predictive Learning” localizes sound sources with no negative pairs in any term, using only two positive branches that predict each other (Song et al., 2022). “Negative-Free Self-Supervised Gaussian Embedding of Graphs” likewise seeks uniformity without negatives by matching the learned distribution to an isotropic Gaussian (Liu et al., 2024). This contrast is central for interpreting the literature correctly.

2. Loss formulations and where free negatives enter

The survey recalls the standard InfoNCE loss over a batch of size $N$ as

$$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$

Here $z_i$ is the anchor, $z_i^+$ its positive, and $\{z_j^-\}$ the negatives drawn freely from the other $N-1$ instances in the batch (Desai et al., 12 Feb 2025).

Free-negative curation typically modifies the denominator either by re-weighting negatives or by dropping suspected false negatives. The unified form given in the survey is

$$L = -\frac1N \sum_i \log \frac{\exp(\mathrm{sim}(z_i,z_i^+)/\tau)} {\exp(\mathrm{sim}(z_i,z_i^+)/\tau) + \sum_{j\neq i} w_{ij}\,\exp(\mathrm{sim}(z_i,z_j)/\tau)}.$$

Within this formulation, $w_{ij}=0$ removes a candidate negative, $w_{ij}\in(0,1)$ down-weights easy or uncertain negatives, and $w_{ij}>1$ up-weights hard negatives (Desai et al., 12 Feb 2025).

The same logic extends to the triplet-loss setting:

$$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$0

where free-negative mining simply chooses $$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$1 from the unlabeled pool according to a curation strategy (Desai et al., 12 Feb 2025).

Dense contrastive learning provides a more specialized instantiation. In DenseCL++, a dense negative pair is a pairing

$$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$2

between a location $$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$3 in the anchor and a location $$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$4 in any other image’s view $$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$5. Such pairs are “free” in the sense that no additional forward or backward passes are needed—once the batch of views and their spatial features is available, they may be paired arbitrarily (Iskender et al., 2022). DenseCL++ replaces global negatives with all dense features from other images in the dense-contrastive term:

$$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$6

This makes the free-negative construction explicit at the spatial level (Iskender et al., 2022).

CLAN alters the semantics more radically. During pre-training on benign network flows, every $$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$7 views the remaining benign samples $$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$8 with $$L_{\mathrm{InfoNCE}} = -\frac1N \sum_{i=1}^N \log \frac{\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr)} {\exp\bigl(\mathrm{sim}(z_i,z_i^+)/\tau\bigr) + \sum_{j=1}^{N-1}\exp\bigl(\mathrm{sim}(z_i,z_j^-)/\tau\bigr)}.$$9 in the batch as positives, while an augmented version $z_i$0 of each sample is taken as the negative view (Wilkie et al., 8 Sep 2025). Its batch loss is

$z_i$1

with the first term shrinking distances among benign embeddings and the second pushing augmented embeddings at least $z_i$2 away from every benign latent point (Wilkie et al., 8 Sep 2025). A plausible implication is that “free-negative pairs” need not be restricted to other raw samples; augmentation can itself instantiate the negative mechanism if the task semantics justify it.

3. Mining strategies and concrete algorithms

The survey details two free-negative mining algorithms. UnReMix, attributed to Tabassum et al., selects negatives that are simultaneously close in embedding space to the anchor, uncertain under the model, and representative of the global data distribution (Desai et al., 12 Feb 2025). For each candidate $z_i$3, it computes a score from cosine similarity, model uncertainty, and similarity to the mean of a queue of past embeddings, then selects the top-$z_i$4 indices. Its stated complexity is $z_i$5 per anchor to compute the relevant quantities, with overall $z_i$6 per batch and a modest overhead of 10–20% extra compute (Desai et al., 12 Feb 2025).

Debiased Contrastive Learning, attributed to Chuang et al., addresses false negatives by analytically estimating the probability $z_i$7 that a candidate is a false negative and then correcting the denominator (Desai et al., 12 Feb 2025). The modified term is

$z_i$8

where $z_i$9 and $z_i^+$0. The survey characterizes this method as retaining the $z_i^+$1 per-batch complexity of standard InfoNCE but adding only constant-time arithmetic (Desai et al., 12 Feb 2025).

DenseCL++ instantiates hard-negative selection in a dense setting. Inter-image negatives can be obtained by random sampling or by guided sampling. In guided sampling, one draws $z_i^+$2 candidate sets of spatial indices, computes for each set a mean clipped similarity

$z_i^+$3

and picks the set with highest $z_i^+$4 to form hard negatives (Iskender et al., 2022). The same paper also introduces intra-image negatives by sampling the least similar locations in the positive view, thereby widening the cross-view similarity distribution (Iskender et al., 2022).

These strategies reveal that free-negative mining is not synonymous with naive batch reuse. The unlabeled pool can be curated by hardness, by false-negative suppression, or by synthetic transformation, while remaining “free” in the sense defined by the survey (Desai et al., 12 Feb 2025).

4. Representative instantiations across domains

Method	Free-negative mechanism	Reported outcome
DenseCL++ (Iskender et al., 2022)	Dense–dense negatives sampled from spatial locations of other images’ views	63.4% mAP with random dense–dense negatives; 64.1% with guided sampling
CLAN (Wilkie et al., 8 Sep 2025)	Augmented samples treated as negative views; other benign samples treated as positives	mean AUROC = 0.95859 on Lycos2017 binary classification
Survey exemplars (Desai et al., 12 Feb 2025)	Hard mining, false-negative correction, synthetic interpolation	0.5–1.5% Top-1 ImageNet gains at modest additional cost

DenseCL++ is an example in which free-negative pairs arise from internal structure already present in the batch. The original DenseCL used global negatives pooled over spatial locations from other images, whereas DenseCL++ replaces each global negative by all of its dense features (Iskender et al., 2022). On COCO multi-label classification, the reported progression is SimCLR at 59.6% mAP, DenseCL at 59.9% mAP, and DenseCL++ with random dense–dense negatives at 63.4% mAP; guided sampling further increases this to 64.1% mAP (Iskender et al., 2022). In segmentation, the same paper reports SimCLR at 69.3 and 61.5 mIoU on VOC and COCO respectively, versus DenseCL++ at 70.0 and 63.3 mIoU (Iskender et al., 2022).

CLAN demonstrates a different free-negative construction. Instead of using other instances as negatives, it treats augmentations $z_i^+$5 as representing a potentially malicious perturbation of benign traffic, while all other benign samples in the batch are regarded as positives (Wilkie et al., 8 Sep 2025). After pre-training, it caches the centroid $z_i^+$6 of the benign cluster in latent space, and inference reduces to a single distance computation to that centroid, which the paper states is $z_i^+$7 rather than the $z_i^+$8 nearest-neighbour lookups required by conventional contrastive methods (Wilkie et al., 8 Sep 2025). On Lycos2017 binary classification without fine-tuning, CLAN reports mean AUROC $z_i^+$9, compared with 0.92722 for DUAD and approximately 0.90211 for the best self-supervised baseline (Wilkie et al., 8 Sep 2025).

The survey places such constructions inside a broader design space rather than treating them as isolated tricks. Hard Negative Mixing (MoCHi) is summarized as giving +1.2% Top-1 accuracy with approximately 25% per-batch compute overhead; UnReMix gives +1.0% Top-1 with approximately 15% compute overhead but converges in approximately 0.9 times the epochs; Debiased Contrastive gives +0.8% Top-1 with approximately no extra forward/backward time beyond standard InfoNCE; False Negative Attraction/Elimination gives +0.6% with an approximately 1.1 times convergence speedup; Synthetic Hard Negatives give +0.7% but require approximately twice the memory footprint if naively implemented (Desai et al., 12 Feb 2025).

5. Trade-offs, failure modes, and practical guidance

The survey’s empirical summary emphasizes that free-negative methods trade annotation-free pair curation for algorithmic overhead and hyperparameter sensitivity (Desai et al., 12 Feb 2025). Hard-negative methods improve discrimination but can increase compute, especially when interpolation or generative models are involved. False-negative correction adds little arithmetic overhead but depends on the accuracy of the false-negative prior. Synthetic methods can improve representation quality while materially increasing memory or engineering cost.

The survey also provides operational guidelines. It recommends a curriculum in which training begins with random negatives and gradually introduces harder ones, maintaining approximately 10–30% of negatives as hard in each batch (Desai et al., 12 Feb 2025). To avoid false negatives, it suggests using a cosine-similarity threshold such as 0.7 to detect and either drop or reclassify very high-similarity negatives, and tuning the prior $\{z_j^-\}$0 on a held-out validation set in debiased-loss approaches (Desai et al., 12 Feb 2025). For large datasets, it recommends a memory bank or momentum encoder, enabling sublinear nearest-neighbour search with FAISS or approximate k-NN, and caching per-sample uncertainties in UnReMix-style pipelines (Desai et al., 12 Feb 2025). Monitoring the ratio of dropped versus kept negatives is also advised; eliminating more than 30% of the batch risks undertraining (Desai et al., 12 Feb 2025).

DenseCL++ supplies a domain-specific caution. Although one might expect additional positives to help, attempts to add more than one positive per anchor, using top-$\{z_j^-\}$1 positives, consistently degraded performance (Iskender et al., 2022). The same paper also notes that an auxiliary reconstruction loss can provide only a marginal $\{z_j^-\}$2–0.3% mAP bump and must be weighted very lightly, with $\{z_j^-\}$3 in the range $\{z_j^-\}$4 to $\{z_j^-\}$5, or contrastive performance collapses (Iskender et al., 2022).

CLAN identifies a different failure mode: it assumes the entire pre-training set is free of malicious contamination (Wilkie et al., 8 Sep 2025). The paper states that robustness under contamination remains to be studied. It also notes that the simple uniform-resampling augmentation $\{z_j^-\}$6 may not span the full variety of real attacks, and extending the method to mixed-domain or continual settings is an open direction (Wilkie et al., 8 Sep 2025). This suggests that, even when negatives are “free,” their semantic validity remains task-dependent.

6. Relation to negative-free methods and conceptual boundaries

The literature also makes clear what free-negative pairs are not. SSPL is explicitly described as a negative-free method for sound source localization via explicit positive mining (Song et al., 2022). It forms two visual views $\{z_j^-\}$7 from one video frame, associates both with the co-occurring audio clip, and trains them with a SimSiam-style predictor–projection scheme:

$\{z_j^-\}$8

Crucially, there are no negative pairs in any term—only the two positive branches predict each other (Song et al., 2022). The paper motivates this by arguing that random negatives can include the same object class, producing “false negatives,” which confuse localization and break feature alignment (Song et al., 2022). On SoundNet-Flickr 144k, SSPL with PCM reports cIoU 0.759 and AUC 0.610 versus HardWay at 0.699 and 0.590, corresponding to improvements of +8.6% cIoU and +3.4% AUC (Song et al., 2022).

SSGE for graphs also eliminates negatives, but for a different reason. It begins from the alignment–uniformity perspective on graph contrastive learning and argues that uniformity can be achieved without negative pairs by matching the representation distribution to an isotropic Gaussian (Liu et al., 2024). The alignment term is

$\{z_j^-\}$9

and the negative-free uniformity term is based on the Wasserstein distance to $N-1$0:

$N-1$1

The total loss is

$N-1$2

The paper emphasizes the absence of a parameterized mutual information estimator, an additional projector, asymmetric structures, and negative samples (Liu et al., 2024). On multiple graph benchmarks it reports competitive performance with fewer parameters, shorter training times, and lower memory consumption than existing graph contrastive learning methods (Liu et al., 2024).

This boundary is conceptually important. Free-negative pair methods refine how negatives are selected from unlabeled data. Negative-free methods deny that explicit negatives are necessary for the objective at all. A common misconception is to collapse these two families under a single label because both aim to reduce the pathologies of naive random negatives. The sources instead indicate two distinct responses to the same problem: either curate negatives more carefully, or redesign the learning objective so that negatives disappear (Desai et al., 12 Feb 2025, Liu et al., 2024).

7. Significance and scope

Within contrastive self-supervision, free-negative pairs represent a pragmatic answer to the cost and brittleness of negative sampling. They preserve the contrastive structure of the objective while avoiding any requirement for external supervision, and they can be instantiated at multiple granularities: instance level, dense spatial level, and augmentation level (Desai et al., 12 Feb 2025, Iskender et al., 2022, Wilkie et al., 8 Sep 2025). The reported empirical pattern in the survey is that such methods routinely yield 0.5–1.5% Top-1 ImageNet gains at modest additional cost (Desai et al., 12 Feb 2025).

At the same time, the surrounding literature shows that free-negative design is not merely an implementation detail. In DenseCL++, replacing dense–global negatives with dense–dense negatives changes the geometry of learned spatial features and improves downstream multi-label classification and segmentation (Iskender et al., 2022). In CLAN, redefining augmentations as negatives and other benign samples as positives changes the latent topology from sample-centric clusters to one unified benign manifold (Wilkie et al., 8 Sep 2025). These cases suggest that the semantics of what counts as a “negative” are domain-dependent and task-dependent.

A plausible implication is that free-negative pairs are best understood as a curation principle rather than a single method family. Their common feature is the reuse of unlabeled data to construct repulsive relationships without annotation cost. Their main technical questions are how to avoid false negatives, how to identify hard negatives without destabilizing training, and when synthetic or augmentation-derived negatives align with the target data-generating process. The recent negative-free literature further sharpens these questions by showing that some tasks can sidestep them entirely, replacing negative selection with explicit positive mining or distributional regularization (Song et al., 2022, Liu et al., 2024).