On the Wasserstein Gradient Flow Interpretation of Drifting Models

Abstract: Recently, Deng et al. (2026) proposed Generative Modeling via Drifting (GMD), a novel framework for generative tasks. This note presents an analysis of GMD through the lens of Wasserstein Gradient Flows (WGF), i.e., the path of steepest descent for a functional in the space of probability measures, equipped with the geometry of optimal transport. Unlike previous WGF-based contributions, GMD can be thought of as directly targeting a fixed point of a specific WGF flow. We demonstrate three main results: first, that one algorithm proposed by Deng et al. (2026) corresponds to finding the limiting point of a WGF on the KL divergence, with Parzen smoothing on the densities. Second, that the algorithm actually implemented by Deng et al. (2026) corresponds to a different procedure, which bears some resemblance to the fixed point of a WGF on the Sinkhorn divergence, but lacks certain desirable properties of the latter. Third, the same same idea can be extended to the limiting point of other WGFs, including the Maximum Mean Discrepancy (MMD), the sliced Wasserstein distance, and GAN critic functions.

• The paper establishes a formal mapping between GMD and fixed points of Wasserstein gradient flows, clarifying its non-gradient-flow behavior.
• It rigorously analyzes drift methods, including score-difference and Sinkhorn Proxy, revealing convergence challenges and limitations in transferring mass between modes.
• Empirical results across synthetic datasets highlight optimal sample quality and robustness trade-offs, emphasizing best practices for non-adversarial generative modeling.

Formal Analysis of "On the Wasserstein Gradient Flow Interpretation of Drifting Models" (2605.05118)

Introduction and Theoretical Framing

The paper rigorously examines Generative Modeling via Drifting (GMD), contextualizing its mechanisms within the landscape of Wasserstein Gradient Flows (WGF). The GMD approach leverages non-parametric drift operators, displacing generated samples towards the target distribution through iterative, mean-shift-type updates without adversarial objectives or explicit likelihood maximization. The authors demonstrate that certain GMD instantiations correspond to finding approximations to the fixed points of WGFs for specific divergences, notably the KL divergence (via Parzen smoothing), Sinkhorn divergence, Maximum Mean Discrepancy (MMD), sliced Wasserstein distances, and GAN-related $f$-divergences.

GMD as Fixed Points of Wasserstein Gradient Flows

The central claim is that the GMD algorithm, in its simplest form (score-difference drift with a Gaussian kernel), approximates the fixed point of a WGF minimizing the KL divergence between kernel-smoothed densities. The mean-shift drift field is expressed as $V_{p,q}(x)$, which for the Gaussian kernel reduces to a score difference on noised distributions. Convergence analyses in the paper clarify that this field is zero if and only if the generated and target distributions match, consistent with findings from related literature.

However, the paper counters prior claims (e.g., Cao et al., 2026) by demonstrating that the drift employed does not correspond to the true Wasserstein gradient for the KL of Parzen density estimates; instead, the actual gradient field involves integrating kernel-weighted score differences. Consequently, guarantees about functional dissipation and convergence properties need adjustment.

Sinkhorn Proxy and Algorithmic Interpretation

The practically adopted GMD algorithm diverges from pure score-difference drift, instead closely aligning with a proxy for the WGF of the Sinkhorn divergence. The paper introduces the Sinkhorn Proxy drift—an efficient, single-step approximation to the entropy-regularized optimal transport plan. This proxy yields a drift field for generator updates, modulated by a spatially varying conditioning term.

Crucially, the authors rigorously prove that, for Gaussian kernels, the Sinkhorn Proxy drift is zero only at equality between generated and target distributions. Despite superficial affinity with optimal transport, the Sinkhorn Proxy velocity does not generally represent any true Wasserstein gradient, except under the highly restrictive condition that the pre-conditioner and score gradients are collinear everywhere. This result undermines the assumption that convergence of the proxy inherits robust OT-like transfer across distant modes (as detailed in synthetic mode-separation experiments in the paper).

Figure 1: MMD between true and generated samples trained by different drift types.

Numerical and Empirical Illustration

The paper offers empirical validation across five synthetic 2D datasets (e.g., Moons, Circles, Pinwheel, Swiss Roll), using MMD as a quality metric and varying drift hyperparameters systematically. Strong numerical findings include:

• All flow objectives (KL, MMD, Sinkhorn, Sliced-Wasserstein) reach optimal sample quality within specific hyperparameter regions, but differ in robustness to parameter mis-specification.
• KL flow suffers from mode collapse at small noise scales and degrades with excessive smoothness, Sinkhorn Proxy demonstrates superior tolerance to low regularization (small $\tau$) and outperforms the practical GMD Algorithm 2 in some regimes.
• The Sinkhorn drift, as well as the score-difference drift, fail to transfer mass between distant modes in mixture settings, confirming theoretical limitations exposed in synthetic analysis.

  Figure 2: True and generated samples for different types of drift and hyperparameters; diverged samples shown as empty panels.

Comparative Analysis of Drift Fields

Beyond Sinkhorn and KL, extensions to MMD flow, Sliced-Wasserstein flow, and GAN critic-function-based flows are fleshed out. Each is derived via first variations of energy functionals and their corresponding drift fields for particle evolution. The authors elucidate that, for characteristic kernels, MMD vanishes only for distributional equality, and guiding the generator with adaptive kernel choice improves convergence. The paper details practical estimation schemes for all flows and emphasizes the importance of the stop-gradient operator in loss objectives to avoid degenerate solutions.

Figure 3: Results for the 8 Gaussian dataset, highlighting sensitivity to drift type and hyperparameters.

Figure 4: Results for the Circles dataset; empty panels signal divergence.

Figure 5: Results for the Pinwheel dataset illustrating performance variability among flows.

Figure 6: Results for the Swiss roll dataset, reflecting generative fidelity and stability trade-offs.

Implications and Limitations

Theoretically, the paper's critical contribution is the formal mapping between GMD and fixed points of Wasserstein flows—not time-discretized flows—ensuring practical feasibility for a single-stage generator. The identification of non-gradient-flow behavior in practical algorithms warns against over-interpretation of mass transfer capability in generative settings, particularly with non-overlapping multimodal targets.

Practically, the results support the adoption of GMD for stable, non-adversarial generative modeling, especially in settings where mode transfer is unnecessary. The work invites further investigation into adaptive kernel selection, alternate divergence objectives, and neural critic parameterization for improved sample quality and robustness.

Future Directions

Possible future advances in AI generative modeling via GMD and related flows include:

• Exploration of hybrid drifts combining score-based and OT-based elements to promote mass transfer across separated modes,
• The development of dynamical schemes for adaptive kernel selection and regularization tuning driven by real-time MMD or Sinkhorn metrics,
• Extension to high-dimensional generative domains and structured data beyond current synthetic benchmarks,
• Incorporation of neural critic architectures within drift fields, possibly leveraging the Fenchel duality for $f$-divergences,
• Formal characterization of convergence and stability in the presence of complex support mismatches and highly multimodal data.

Conclusion

This paper delivers an authoritative mathematical and algorithmic analysis of drifting-based generative modeling through the lens of Wasserstein gradient flows. It resolves ambiguities regarding drift field interpretations, proves essential properties and limitations for both score-based and Sinkhorn-based methods, and validates these claims with empirical results. The work serves as a touchstone for researchers designing non-adversarial, one-stage generative models, prescribing both the boundaries and the opportunities for future advancement in flow-based sampling algorithms.