CNSDiff: Causal Negative Sampling via Diffusion

• The paper introduces CNSDiff, a novel method that replaces heuristic negative sampling with a diffusion-based mechanism to mitigate false hard negatives.
• It integrates a causal regularization term that deconfounds environmental biases, ensuring more robust generalization across distribution shifts.
• Empirical evaluations demonstrate significant improvements in Recall and reduced false negative rates across multiple OOD scenarios on datasets like Yelp2018, Food, and KuaiRec.

Causal Negative Sampling via Diffusion (CNSDiff) is a method for out-of-distribution recommendation that replaces heuristic negative sampling from predefined candidate pools with synthesized negatives generated in latent space by a conditional diffusion process. It is motivated by the observation that candidate pools are shaped by unobserved environmental confounders, including exposure and popularity biases, which can cause heuristic methods to select false hard negatives and thereby induce spurious correlations that degrade generalization under distribution shift. CNSDiff combines diffusion-based negative generation with a causal regularization term intended to mitigate the influence of such confounders during sampling, and is reported to improve robustness across multiple distribution-shift settings in recommendation (Zhao et al., 10 Aug 2025).

1. Problem formulation and motivation

In two-tower or graph-based recommenders trained with pairwise losses such as BPR, negative sampling selects unobserved items $v^-$ for each positive pair $(u,v)$. The underlying rationale is standard: easy negatives, whose embeddings are far from the user and thus have low similarity, contribute little learning signal, whereas hard negatives, which have high similarity under the current model scores $s(u,v^-)$, force the decision boundary to tighten (Zhao et al., 10 Aug 2025).

CNSDiff is motivated by a failure mode in this procedure. The paper identifies False Hard Negative Samples (FHNS) as user–item pairs $(u,v)$ for which the user truly likes the item, $Y_{u,v}=1$, but the interaction is unobserved in the data, $\tilde Y_{u,v}=0$; the item has very low exposure probability under the current environment $e$, $\phi(v,e)\ll 1$; and the learned embeddings are nevertheless very close, $\mathrm{sim}(z_u,z_v)\ge \tau$. Under this characterization, environmental confounders such as popularity or exposure bias distort the candidate pool and cause heuristic sampling procedures to treat such instances as hard negatives.

The significance of this distinction is central to the method. If FHNS are repeatedly pushed away during training, the recommender can internalize spurious correlations—such as associating low exposure or low popularity with user dislike—and these correlations need not remain valid when the test distribution differs from the training environment. CNSDiff therefore discards the fixed candidate-pool paradigm entirely and instead synthesizes negatives in embedding space so as to reduce dependence on confounded pools.

2. Conditional diffusion mechanism for negative synthesis

The core sampling mechanism is a conditional diffusion process defined directly on item embeddings rather than on a predefined set of candidate negatives (Zhao et al., 10 Aug 2025). Starting from a positive item embedding $z_0\in\mathbb R^d$, CNSDiff defines a discrete-time Markov chain that gradually injects Gaussian noise through the forward process

$(u,v)$0

Using reparameterization, this is equivalently written as

$(u,v)$1

with $(u,v)$2.

The reverse process learns a neural noise predictor $(u,v)$3, or equivalently $(u,v)$4, to denoise sequentially: $(u,v)$5 Here the conditioning variable $(u,v)$6 encodes both the original positive embedding $(u,v)$7 and a proxy for environment or confounder, such as item popularity or exposure statistics. The description specifies that $(u,v)$8 is concatenated or cross-attended to the time embedding $(u,v)$9 and the noisy embedding $s(u,v^-)$0 within $s(u,v^-)$1.

Training minimizes a simplified noise-prediction objective: $s(u,v^-)$2 Within the formulation given in the paper, this diffusion mechanism is not only a generative module but also a hardness-control mechanism: negatives are synthesized in latent space and then selected according to their relevance to the current user representation, rather than retrieved from a confounded external pool. A plausible implication is that the hardness of negatives becomes a property of the learned denoising trajectory rather than of heuristic ranking over observed candidates.

3. Causal regularization and deconfounding objective

CNSDiff supplements diffusion-based synthesis with an explicit causal regularization term intended to deconfound the generation process (Zhao et al., 10 Aug 2025). The target quantity is an interventional log-likelihood,

$s(u,v^-)$3

where the do-operator is used to represent intervention on the positive embedding and thereby sever spurious back-door paths from environment to generated negatives.

The paper introduces a variational lower bound using an approximate posterior $s(u,v^-)$4 over environments. Theorem 3.2 states that

$s(u,v^-)$5

This bound is incorporated into the sampling objective through

$s(u,v^-)$6

The stated intuition is that minimizing $s(u,v^-)$7 forces the generated chain to be invariant to sampled environments and thus suppresses FHNS. In this formulation, causal regularization is not an auxiliary post hoc correction; it is embedded directly in the negative-sampling mechanism.

The associated theoretical analysis further links confounding to OOD failure. Theorem 3.1 states that if the false hard negative rate $s(u,v^-)$8 varies across environments, then under a shift with $s(u,v^-)$9, the generalization gap satisfies

$(u,v)$0

where

$(u,v)$1

$(u,v)$2 denotes total-variation distance, and $(u,v)$3 is irreducible invariant risk. The paper’s interpretation is that enforcing do-style invariance in the diffusion process reduces $(u,v)$4, thereby tightening the bound.

4. Training and inference procedure

The training pipeline combines graph-based representation learning, diffusion-based negative synthesis, causal regularization, and pairwise ranking (Zhao et al., 10 Aug 2025). User embeddings $(u,v)$5 and positive item embeddings $(u,v)$6 are first computed via GCN. For each positive, the method draws $(u,v)$7 noisy chains: for each chain, it samples a timestep $(u,v)$8, computes $(u,v)$9 through forward noising, and then denoises through $Y_{u,v}=1$0 back to step $Y_{u,v}=1$1 to obtain a generated embedding $Y_{u,v}=1$2.

From these $Y_{u,v}=1$3 generated candidates, the hardest negative is selected according to user–item similarity: $Y_{u,v}=1$4 This hard generated negative is then mixed with a random negative $Y_{u,v}=1$5 to form

$Y_{u,v}=1$6

with $Y_{u,v}=1$7 scheduled over epochs.

The ranking objective is a BPR loss

$Y_{u,v}=1$8

combined with the causal diffusion objective and, optionally, a contrastive alignment term $Y_{u,v}=1$9 between $\tilde Y_{u,v}=0$0 and generated negatives: $\tilde Y_{u,v}=0$1 Parameters $\tilde Y_{u,v}=0$2 and $\tilde Y_{u,v}=0$3 are updated via $\tilde Y_{u,v}=0$4.

At inference time, the method still draws hard negatives through a short diffusion chain on each positive candidate, but no gradients with respect to $\tilde Y_{u,v}=0$5 are accumulated. The final model then uses the learned embeddings for ranking. This makes the diffusion component part of both training-time negative construction and test-time ranking preparation, while the final scoring remains embedding-based.

5. Empirical evaluation under distribution shift

The experimental design covers four representative distribution-shift scenarios and evaluates both ranking quality and false-negative behavior (Zhao et al., 10 Aug 2025). The scenarios are summarized below.

Scenario	Dataset	Description
Popularity Shift	Yelp2018	OOD test set sampled to have uniform item popularity vs. biased train
Temporal Shift	Food	Most recent 20% of each user’s interactions form OOD test; the rest split IID
Exposure Shift	KuaiRec	Fully exposed small interaction matrix as test, partially exposed large matrix as train
Within-Dataset Group Shifts	Multiple settings	Items divided into four popularity or four temporal buckets to probe FHN ratio and performance per bucket

The baselines include RNS, DNS(M,N), MixGCF, AHNS, DMNS, OOD-aware methods such as InvCF, AdvInfoNCE, CDR, and DR-GNN, and diffusion recommenders including DiffRec, HDRM, and CausalDiff. Evaluation uses Recall@10, Recall@20, NDCG@10, and NDCG@20.

The reported averaged results identify CNSDiff as the strongest method across the tested OOD settings. On Yelp2018, it achieves $\tilde Y_{u,v}=0$6, described as a $\tilde Y_{u,v}=0$7 improvement over the second best result. On KuaiRec, it reports $\tilde Y_{u,v}=0$8 with a $\tilde Y_{u,v}=0$9 improvement, and on Food, $e$0 with a $e$1 improvement. Across all evaluation metrics, the paper reports an overall average improvement of approximately $e$2.

The paper also presents a false-negative analysis. Figure 1 is described as showing that the FHN ratio of CNSDiff remains substantially below that of MixGCF and AHNS over epochs, and that this accompanies faster convergence and higher Recall@20. Within the scope of the reported evidence, the empirical case for CNSDiff is therefore not limited to aggregate ranking metrics; it is explicitly tied to reduced FHNS incidence under confounded sampling conditions.

6. Computational profile, hyperparameters, and extensions

The computational profile of CNSDiff has two principal parts (Zhao et al., 10 Aug 2025). GCN embedding has complexity

$e$3

where $e$4 is the number of layers, $e$5 the number of edges, and $e$6 the embedding dimension. Diffusion sampling has complexity

$e$7

for $e$8 positive instances, $e$9 chains, and diffusion length $\phi(v,e)\ll 1$0. The paper states that, in practice, CNSDiff is comparable in per-epoch time to MixGCF and AHNS.

The hyperparameter study identifies several operating regions. For the diffusion length, $\phi(v,e)\ll 1$1 performs best around $\phi(v,e)\ll 1$2; if it is too small, the generated negatives are insufficiently hard, whereas if it is too large, they become noisy and distant. For the number of negatives, $\phi(v,e)\ll 1$3 to $\phi(v,e)\ll 1$4 is reported as a good trade-off. For the regularization and alignment coefficients, mild values in the range $\phi(v,e)\ll 1$5 to $\phi(v,e)\ll 1$6 are stated to suffice, while large values degrade NDCG.

The paper also lists several extensions. These include adding multi-modal contexts such as text or image features to the conditioning variable $\phi(v,e)\ll 1$7, using incremental diffusion chains for online or continual training, and applying the framework to link prediction and contrastive graph learning beyond recommendation. These extensions are proposed directions rather than experimentally established results within the reported study.

7. Positioning, interpretation, and open questions

The paper characterizes CNSDiff as the first framework to combine three elements: hardness-controllable negative generation via conditional diffusion in latent space, explicit causal invariance against unobserved environment confounders by a back-door regularization, and integration of the resulting negatives into a paired BPR plus contrastive mixup training scheme (Zhao et al., 10 Aug 2025). Within that positioning, the method addresses a specific misconception common in heuristic negative sampling: that high-similarity unobserved items are always informative hard negatives. The FHNS formulation shows that high similarity can also indicate missing exposure rather than true irrelevance.

A second interpretive point concerns the role of diffusion. In CNSDiff, diffusion is not presented merely as a generative augmentation device. It functions as a mechanism for decoupling negative construction from biased candidate pools, while causal regularization attempts to constrain the generated samples so that they are less entangled with environmental confounders. This suggests a broader view in which negative sampling, representation learning, and OOD robustness are treated as a single coupled problem.

The open questions named in the study concern scaling and generality. One is narrowing the diffusion cost for ultra-large catalogs. Another is extending the causal analysis to multi-step sequential recommenders. These questions follow directly from the model’s two defining commitments: latent-space diffusion introduces additional computation, and the current causal theory is formulated around environment-dependent false hard negative rates rather than sequential decision processes.