Synthetic Hard Negatives

• Synthetic hard negatives are artificially generated training samples that closely mimic positives to challenge model discrimination.
• They are constructed through mechanisms like LLM-driven text generation, feature mixing, and graph-based synthesis to enhance tasks such as retrieval and anomaly detection.
• Their integration improves model robustness and decision boundaries, though careful tuning is needed to mitigate risks like false negatives and supervision noise.

Searching arXiv for papers on synthetic hard negatives across retrieval, contrastive learning, and related domains. Synthetic hard negatives are artificially constructed negative training instances designed to be difficult for a model to distinguish from positives. Across retrieval, contrastive learning, metric learning, anomaly detection, knowledge graph embedding, hyperedge prediction, hallucination detection, and code completion, they are used to sharpen decision boundaries, increase the informativeness of supervision, and mitigate the limitations of random or corpus-bound negative sampling. The common premise is that standard negatives are often too easy, while mined hard negatives may be noisy, false, or insufficiently diverse; synthetic construction instead seeks negatives that are close to the anchor, query, or positive in representation or content while remaining non-matching or unsupported (Li et al., 2024, Dong et al., 2023, Deng et al., 11 Mar 2025).

1. Conceptual scope and problem setting

In supervised and self-supervised learning, negatives are examples the model should separate from a target. A negative is “hard” when it lies close to the decision boundary or is highly similar to the anchor, making discrimination non-trivial. Several works explicitly motivate synthetic hard negatives by showing that random or routine negative selection produces negatives that are too easily distinguishable, which can lead to premature convergence, weak gradients, or limited semantic discrimination (Deng et al., 11 Mar 2025, Liu et al., 2024, Dong et al., 2023).

Synthetic hard negatives differ from conventional hard-negative mining in that they are generated or synthesized rather than merely selected from an existing corpus or batch. This synthesis can occur in text space, feature space, graph embedding space, knowledge graph triple space, or code completion space. In dense retrieval, synthetic hard negatives may be generated with LLMs or refined with LLM-based relabeling and filtering (Li et al., 2024, Choi et al., 13 Apr 2026, Sinha, 20 Apr 2025). In contrastive visual learning, they are often created directly in representation space by mixing or perturbing hard negatives already present in a queue or batch (Giakoumoglou et al., 2024, Dong et al., 2023). In metric learning, they can be produced algebraically through symmetry or globally correlation-aware interpolation (Gu et al., 2020, Peng et al., 2024).

A recurrent issue is the false negative problem: an example treated as negative may actually be relevant, partially relevant, or semantically aligned with the target. This concern appears in dense retrieval, where mined negatives can contain answers or partial answers (Choi et al., 13 Apr 2026), in contrastive learning, where unlabeled samples may belong to the same semantic class (Dong et al., 2023), and in knowledge graphs, where missing facts are not necessarily false (Bernardi et al., 2024). Synthetic-hard-negative methods therefore vary not only in how they increase difficulty, but also in how they control ambiguity and label noise.

2. Construction mechanisms

Synthetic hard negatives are constructed through several recurring mechanisms.

One family uses generation in natural language. SyNeg employs LLMs to synthesize dense-retrieval negatives through a multi-attribute self-reflection prompting strategy, asking for negatives that “appear to address the user query at first glance, but subtly diverge in a way that doesn't truly answer the query” (Li et al., 2024). “Don't Retrieve, Generate” uses an all-LLM pipeline in which an LLM first generates a query from a passage and then generates hard negative passages using only that query text, without corpus access (Sinha, 20 Apr 2025). TAGS-DC edits positive captions into synthetic negative sentences by masking scene-graph-derived tokens and refilling them with mismatched words via an image-aware masked LLM (Fan et al., 2021). UNA generates hard negative sentences by replacing terms according to TF-IDF-driven importance, ensuring the highest-TF-IDF term is always replaced (Shu et al., 2024).

A second family uses feature-space synthesis. SSCL linearly mixes pairs of hard negative features:

$$h_p = \alpha_p \widetilde{z}_i'' + (1-\alpha_p)\widetilde{z}_j''$$

to enlarge the set of difficult negatives for contrastive learning (Dong et al., 2023). HNCSE-HNM similarly creates synthetic hard negatives by mixup of top-k hardest negatives:

$$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$

for sentence embedding learning (Liu et al., 2024). SynCo synthesizes feature-level negatives on the fly from the hardest queue elements using six strategies: interpolation, extrapolation, mixup, noise-injection, perturbed, and adversarial negatives (Giakoumoglou et al., 2024). The same general pattern is adapted to vision transformers in “Unsupervised Training of Vision Transformers with Synthetic Negatives,” where synthetic negatives are generated from the $N=256$ hardest negatives chosen from a queue of $K=4096$ (Giakoumoglou et al., 2 Sep 2025), and in Syn2Co, where an overview function $F(q,n)$ creates normalized hard negatives added to the queue (Giakoumoglou et al., 2 Sep 2025).

A third family uses geometry- or structure-based synthesis. Symmetrical synthesis generates synthetic feature points from two same-class points using each other as axes of symmetry:

$$\mathbf{x}'_k = \beta\left[ \alpha(\mathbf{r}_k^l - \mathbf{x}_k) + \mathbf{x}_k \right]$$

with $\alpha=2.0$ and $\beta=1.0$, then mines the hardest negative among original and synthetic points (Gu et al., 2020). GCA-HNG constructs a graph over a minibatch, propagates messages globally over nodes and edges, predicts a channel-adaptive interpolation vector

$$\bm{\lambda}_{ij} = \operatorname{Sigmoid}(FC(E_{ij}^K)),$$

and synthesizes negatives by interpolation between anchor and negative embeddings (Peng et al., 2024). In hyperedge prediction, HNS builds a weighted “synthetic positive” from positive embeddings and blends it with a simple negative embedding:

$$e_i^{\text{hard}} = (1-\alpha)\hat{e}_i + \alpha \bar{e}_i$$

to place negatives closer to the positive cluster (Deng et al., 11 Mar 2025).

A fourth family uses model- or ontology-guided corruption. In knowledge graph embedding, domain- and range-aware hybrid negative sampling combines random corruption with type-constrained corruption via a mixing parameter $$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$0, thereby balancing semantic plausibility and diversity (Bernardi et al., 2024). In graph contrastive learning, Khan-GCL perturbs latent dimensions identified as independent or discriminative from KAN coefficient statistics to synthesize semantically meaningful hard negatives (Wang et al., 21 May 2025). In anomaly detection, a jointly trained normalizing flow samples synthetic negatives at the border of the inlier distribution rather than from unrelated real outlier datasets (Delić et al., 2023).

3. Retrieval-centric formulations

Dense retrieval has become a central setting for synthetic hard negatives, with distinct strategies for generation, refinement, and training integration.

SyNeg frames the problem as LLM-driven generation plus hybrid integration. Its synthetic negatives are explicitly designed to be contextually rich and diverse, while the final training set combines one synthetic negative with $$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$1 retrieved negatives per query-positive pair. The loss is standard InfoNCE:

$$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$2

with

$$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$3

The paper reports that hybrid mixing stabilizes optimization relative to direct mixing, and that fine-tuning on only synthetic negatives reduces performance (Li et al., 2024).

ARHN addresses a different retrieval failure mode: the contamination of hard negatives by false negatives and ambiguous negatives. Its two-stage answer-centric pipeline first asks an open-source LLM to extract a verbatim answer snippet or emit NO_ANSWER for each query-passage pair, then asks the LLM to rank snippets by direct answerability to the query (Choi et al., 13 Apr 2026). It promotes a hard negative to positive when

$$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$4

filters an answer-bearing negative ranked below the positive when

$$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$5

and retains as true negative any passage with $$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$6 (Choi et al., 13 Apr 2026). On BEIR, the combined relabeling-plus-filtering strategy is reported to improve over either step in isolation, and larger LLMs such as Qwen3-32B are said to achieve non-trivial Cohen’s $$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$7 agreement with human assessments (Choi et al., 13 Apr 2026).

A related but simpler retrieval pipeline is “Don't Retrieve, Generate,” which replaces BM25 or cross-encoder hard-negative mining with query-conditioned LLM generation. For E5-Base and GTE-Base on 10 BEIR datasets, the averages reported are identical across LLM-generated, BM25-mined, and cross-encoder-mined negatives: for GTE-Base, nDCG@10 $$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$8, Precision@10 $$\overline{u_o} = \alpha G_i + (1-\alpha) G_j,\qquad u_o = \frac{\overline{u_o}}{||\overline{u_o}||_2}$$9, and Recall@100 $N=256$0; for E5-Base, nDCG@10 $N=256$1, Precision@10 $N=256$2, and Recall@100 $N=256$3 (Sinha, 20 Apr 2025). This paper therefore presents synthetic hard negatives not merely as an augmentation, but as a corpus-free substitute for conventional mining.

In generative retrieval, “On Synthetic Data Strategies for Domain-Specific Generative Retrieval” uses LLM-generated hard queries and model-based hard negatives for preference learning. Negatives are the model’s own top-ranked candidates above the positive; random negatives degrade performance, while top-5 and top-10 hard negatives improve HIT@4, HIT@10, MAP@10, and MRR@10 on MultiHop-RAG (Wen et al., 25 Feb 2025). In domain adaptation for dense retrieval, R-GPL does not synthesize new negatives in free-form text, but it refreshes hard negatives during training. The paper reports improvements over GPL in 13/14 BEIR datasets and 9/12 LoTTE datasets, with BEIR average nDCG@10 improving from 46.4 to 47.7 (Yuksel et al., 24 Jan 2025). This suggests that synthetic-hard-negative practice in retrieval includes not only generation but also dynamic maintenance of hardness.

4. Contrastive and metric-learning paradigms

Contrastive learning and metric learning provide a broad technical substrate for synthetic hard negatives because the learning objective depends directly on positive-negative geometry.

In visual contrastive learning, SSCL argues that only a small portion of ordinary negatives are hard, then generates more and harder negatives by feature-level mixing and weights them according to similarity:

$N=256$4

To handle false negatives, it incorporates a debiased contrastive term derived from a mixture view of positives and true negatives (Dong et al., 2023). Quantitatively, the paper reports Top-1 gains from 82.35% to 86.20% on CIFAR10, from 52.02% to 60.01% on CIFAR100, and from 25.22% to 37.31% on TinyImageNet relative to SimCLR (Dong et al., 2023).

SynCo operates within the MoCo framework and extends InfoNCE by augmenting the denominator with synthetic negatives:

$N=256$5

Its reported gains include 67.9% Top-1 on ImageNet linear evaluation versus 67.5% for MoCo-v2 and 66.9% for MoCHI at 200 epochs, plus 41.0 AP on COCO detection and 35.7 AP on COCO instance segmentation (Giakoumoglou et al., 2024). The later transformer-focused adaptation reports +0.2 Top-1 for both DeiT-S and Swin-T over MoBY, specifically 72.8 to 73.0 and 75.0 to 75.2, respectively (Giakoumoglou et al., 2 Sep 2025).

For sentence representation learning, HNCSE defines hard negatives by a cosine-similarity threshold,

$N=256$6

then introduces positive mixing and hard negative mixup (Liu et al., 2024). The paper reports average STS improvements from 76.25 to 78.38 for HNCSE-PM with BERT-base and from 78.11 to 79.20 for HNCSE-HNM with BERT-large (Liu et al., 2024). UNA approaches the same problem from lexical retrieval statistics rather than embedding similarity, using TF-IDF to control which words to replace in a sentence, and reports average STS improvements from 0.7532 to 0.7614 on BERT-base and from 0.7649 to 0.7674 on RoBERTa-base (Shu et al., 2024).

In deep metric learning, synthetic hard negatives are often integrated directly into the loss by expanding the candidate negative set. Symmetrical synthesis considers all 16 negative pairs between original and synthetic points across two classes and selects the hardest for triplet, N-pair, lifted-structure, or angular losses (Gu et al., 2020). On Stanford Online Products, Symm+N-pair reaches Recall@1 of 73.2%, compared with 68.7% for HDML+N-pair and 68.4% for DAML+N-pair (Gu et al., 2020). GCA-HNG generalizes this idea by learning global correlations before channel-adaptive interpolation and reports, for example, R@1 improvements on CUB-200-2011 from 66.5% to 70.3% and on Cars196 from 84.8% to 88.2% with ResNet-50 plus modified N-pair loss (Peng et al., 2024).

5. Domain-specific instantiations

The notion of synthetic hard negatives extends beyond standard retrieval and contrastive representation learning into domain-specific settings where the semantics of “negative” are specialized.

In image-text retrieval, TAGS-DC creates hard negative captions by editing positive captions through scene-graph masking and image-aware refilling. It supplements triplet training with Word Discrimination and Word Correction, both defined as NLL objectives over synthetic negatives (Fan et al., 2021). The paper reports 96.5% human validation that generated negatives are indeed mismatched to the image, and benchmark improvements such as COCO RSum increasing from 468.9 for UNITER (L) to 472.9 for TAGS-DC (L), and Flickr30K RSum from 551.0 to 557.4 (Fan et al., 2021).

In dense anomaly detection, synthetic negatives are not semantic distractors but samples near the support boundary of inlier data. The normalizing flow objective

$N=256$7

encourages border sampling (Delić et al., 2023). NF-Hybrid-LdLx, trained with synthetic negatives, is reported to improve over prior synthetic-negative-only models, with AP 16.67, FPR95 19.26, AUROC 92.39, and open IoU 37.88, compared with 12.12, 25.64, 90.81, and 42.94 for NFlowJS (Delić et al., 2023). Real negatives still perform better in several metrics, which makes this case a salient counterexample to the assumption that synthetic negatives always dominate.

In knowledge graph embedding, synthetic negative generation is a necessity because explicit false triples are rarely provided. The domain- and range-aware hybrid method combines random and type-constrained corruption:

$N=256$8

with $N=256$9 and $K=4096$0 (Bernardi et al., 2024). The paper reports substantial improvement, including FB15k-237 MRR increasing to 0.32 for ComplEx-N3 versus 0.29 for random and 0.24 for type-constrained sampling, and Hetionet MRR increasing to 0.25 versus 0.09 for random (Bernardi et al., 2024).

In hyperedge prediction, HNS synthesizes negatives in hyperedge embedding space rather than selecting node sets directly. The paper reports that HNS attains the highest AUC in 24 out of 28 model-dataset pairs, including HGNN on Citeseer at 0.9597 versus 0.9333 for simple negative sampling (Deng et al., 11 Mar 2025). In graph contrastive learning, Khan-GCL perturbs critical latent dimensions identified from KAN coefficients and reports an averaged ROC-AUC of 75.5% across eight biochemistry datasets, alongside ablations showing performance drops when hard negatives are removed (Wang et al., 21 May 2025).

In hallucination detection and code completion, the “negative” is a plausible but incorrect output. “Teaching with Lies” uses hallucinated answers as negatives in DPO alignment, orders them by difficulty using MiniCheck grounding probability, and reports MedHallu F1 improving from 0.694 to 0.759 and HaluEval F1 from 0.631 to 0.753 for the 3B curriculum model relative to random ordering (Pandit et al., 23 May 2025). “Synthetic Hallucinations, Real Gains” constructs paired chosen/rejected FIM examples using frontier code models across four hallucination types and reports Delulu exact match gains of +18.8 points at 7B and +12.8 points at 3B (Erfanian et al., 2 Jun 2026). These works show that synthetic hard negatives can be framed as carefully engineered false but plausible generations, not merely semantically nearby alternatives.

6. Recurring design principles, benefits, and limitations

Despite substantial variation, several design principles recur.

First, difficulty must be controlled rather than maximized blindly. SyNeg argues that overusing synthetic negatives destabilizes optimization, motivating instance-level hybrid sampling (Li et al., 2024). HNS reports that performance is maximized at intermediate $K=4096$1, while too large a value makes negatives too close to positives (Deng et al., 11 Mar 2025). SSCL shows that synthesis without debiasing may yield little or negative improvement because false negatives proliferate (Dong et al., 2023). The code-completion study similarly finds that supervisory signal is concentrated in the “disputed middle,” rather than in trivial or impossibly hard negatives (Erfanian et al., 2 Jun 2026).

Second, diversity is as important as hardness. TAGS-DC repeats masking and refilling to generate a pool of synthetic captions (Fan et al., 2021). SyNeg randomizes domain name, difficulty level, and generation length in its prompts (Li et al., 2024). GCA-HNG uses random weighting over interpolated negatives from a class to ensure diversity (Peng et al., 2024). SynCo provides six synthesis strategies rather than a single perturbation rule (Giakoumoglou et al., 2024). This suggests that synthetic hard negatives are often treated as a distributional design problem rather than a single-example construction problem.

Third, false negatives and ambiguous supervision remain central risks. ARHN is explicitly built around relabeling false negatives and filtering ambiguous negatives in dense retrieval (Choi et al., 13 Apr 2026). SSCL introduces a debiasing term for unlabeled positive contamination (Dong et al., 2023). TAGS-DC filters synthetic sentences that are too close or exact matches to any ground-truth caption (Fan et al., 2021). Knowledge-graph negative generation balances type constraints with random corruption precisely because highly plausible corruptions can collapse diversity or duplicate the original entity (Bernardi et al., 2024). A plausible implication is that synthetic hard negatives are most effective when paired with mechanisms that detect or constrain semantic overlap with positives.

Fourth, many methods are designed to be plug-and-play. Symmetrical synthesis is described as hyper-parameter free and plug-and-play for existing metric-learning losses without network modification (Gu et al., 2020). HNS is presented as a plug-and-play technique for various hyperedge prediction models (Deng et al., 11 Mar 2025). SSCL is a drop-in loss modification that can integrate with SimCLR and MoCo v2 (Dong et al., 2023). This deployment pattern reflects the practical value of negative-set augmentation methods that do not require rebuilding the full training pipeline.

The limitations are likewise recurrent. Synthetic negatives may underperform real negatives in some regimes, as shown in anomaly detection (Delić et al., 2023). They may require careful hyperparameter tuning, as in Syn2Co and HNS (Giakoumoglou et al., 2 Sep 2025, Deng et al., 11 Mar 2025). Smaller LLMs may over-label and introduce new supervision noise in retrieval refinement (Choi et al., 13 Apr 2026). Some gains depend on architecture, dataset, or language coverage, as reported for vision transformers and code FIM (Giakoumoglou et al., 2 Sep 2025, Erfanian et al., 2 Jun 2026). These observations caution against treating synthetic hardness as uniformly beneficial.

7. Relationship to supervision noise, curriculum, and future directions

Synthetic hard negatives sit at the intersection of negative-sample difficulty, supervision quality, and training dynamics. In some settings the emphasis is on producing harder negatives than the corpus naturally yields; in others it is on correcting or filtering negatives that are only nominally negative. ARHN makes this distinction explicit by jointly relabeling and filtering (Choi et al., 13 Apr 2026). R-GPL shows that hardness is dynamic: negatives selected at the start of domain adaptation become stale as the model improves (Yuksel et al., 24 Jan 2025). “Teaching with Lies” adds an explicit curriculum from easy to hard negatives, ordered by fact-checker-assessed grounding probability (Pandit et al., 23 May 2025). These works indicate that hardness is not a static property but a training-time variable.

Another recurring theme is the move from local to global or structured generation. GCA-HNG criticizes prior pair/triplet-only approaches for ignoring global sample correlations and replaces them with graph message propagation (Peng et al., 2024). Khan-GCL uses KAN coefficient structure to target semantically critical latent dimensions (Wang et al., 21 May 2025). In retrieval, ARHN uses listwise ranking rather than only pairwise judgments (Choi et al., 13 Apr 2026). This suggests a broader trajectory toward synthetic-hard-negative methods that exploit richer context than single pairwise similarity.

A further direction is corpus-free or weakly supervised data construction. The all-LLM retrieval pipeline generates negatives using only query text (Sinha, 20 Apr 2025). Domain-specific generative retrieval uses fully synthetic queries and model-prediction-based hard negatives for preference learning (Wen et al., 25 Feb 2025). Code FIM mitigation uses frontier models and LLM judges instead of execution sandboxes or large human-labeled corpora (Erfanian et al., 2 Jun 2026). A plausible implication is that synthetic hard negatives are increasingly part of end-to-end synthetic data pipelines rather than isolated augmentation tricks.

Taken together, the literature presents synthetic hard negatives as a general methodological family rather than a single algorithmic motif. The family includes lexical replacement, generative text synthesis, feature interpolation, graph-aware fusion, ontology-constrained corruption, answer-centric relabeling, and curriculum-ordered hallucinations. Their unifying role is to provide supervision that is simultaneously more challenging, more informative, and—when carefully controlled—cleaner than naive or static negative sampling (Li et al., 2024, Choi et al., 13 Apr 2026, Giakoumoglou et al., 2024).