---
title: Entropic Sinkhorn Coreset Compression
url: https://www.emergentmind.com/topics/entropic-sinkhorn-coreset
type: topic
---

# Entropic Sinkhorn Coreset Compression

Entropic Sinkhorn coreset denotes a compressed representation of an entropically regularized optimal transport problem or a Sinkhorn-based divergence that preserves the relevant transport geometry while reducing computational cost. In the strictest sense, it is either a convexly weighted submeasure approximating an empirical distribution for the Sinkhorn divergence, or a learned \(K\)-atomic approximation obtained from a KL-regularized entropic transport objective. In a broader, coreset-like sense, the literature also includes compression of Sinkhorn kernel entries, low-rank kernel surrogates, dynamic compressed dual representations, and exact low-parameter reductions in special Gaussian models [2504.20194][2605.12951].

## 1. Foundational entropic Sinkhorn structure

The modern basis of the topic is entropically regularized optimal transport as introduced by Cuturi. In the discrete setting, one considers histograms \(r,c\in\Sigma_d\), the transport polytope
\[
U(r,c)\defeq \{P\in\mathbb{R}_+^{d\times d}\;|\;P\mathbf{1}_d=r,\;P^T\mathbf{1}_d=c\},
\]
and a cost matrix \(M\in\mathbb{R}_+^{d\times d}\). Classical optimal transport solves
\[
d_M(r,c)\defeq \min_{P\in U(r,c)} \langle P,M\rangle,
\]
with worst-case complexity at least \(O(d^3\log d)\). Entropic regularization replaces this LP by the strictly convex problem
\[
P^\lambda=\arg\min_{P\in U(r,c)} \langle P,M\rangle - \frac{1}{\lambda} h(P),
\]
whose unique optimizer has the Gibbs-scaling form
\[
P^\lambda=\operatorname{diag}(u)\,K\,\operatorname{diag}(v),\qquad K=e^{-\lambda M}.
\]
The resulting Sinkhorn updates require only repeated matrix-vector products and rescalings,
\[
u \leftarrow r./(Kv),\qquad v\leftarrow c./(K^Tu),
\]
which makes the computation trivially parallelizable, vectorizable, and suitable for GPUs [1306.0895].

This formulation is the structural reason coreset ideas become natural. The objective depends only on support weights and pairwise costs, the entropic plan is generally dense rather than an extreme sparse LP vertex, and the dominant per-iteration cost is quadratic in the support size. The same paper explicitly notes that reducing the effective support size from \(d\) to \(k\ll d\) would in principle reduce cost from quadratic in \(d\) to quadratic in \(k\), or enable structured approximations to \(K\) [1306.0895].

## 2. Direct coreset formulations

A direct coreset construction for Sinkhorn-based losses appears in CO2. Given i.i.d. data \(X_1,\dots,X_n\sim\mathbb P\), the empirical measure is
\[
\mathbb P_n=\frac{1}{n}\sum_{i=1}^n \delta_{X_i},
\]
and the goal is a convexly weighted coreset
\[
P_m=\sum_{i=1}^n w_i\delta_{X_i},\qquad w_i\ge 0,\quad \sum_i w_i=1,
\]
supported on at most \(m\) observed points. The key result is that sufficiently regular divergences admit a second-order Hadamard expansion whose quadratic term is an MMD. For the Sinkhorn divergence,
\[
S(\mu,\nu):=OT(\mu,\nu)-\frac12\bigl(OT(\mu,\mu)+OT(\nu,\nu)\bigr),
\]
the Hadamard operator is
\[
G_{S,\mathbb P}=\varepsilon (I-\mathcal A^2)^{-1}\xi,
\]
and the resulting kernel is equivalent to the Gaussian kernel. CO2 then compresses by minimizing the induced quadratic form through Nyström approximation and recombination. The main asymptotic guarantee is that, for
\[
m=\omega(\log n),
\]
one can construct \(P_m\) such that
\[
S(\mathbb P,P_m)=S(\mathbb P,\mathbb P_n)+o_p(n^{-1}),
\]
with runtime \(O(n^2m+m^3)\) [2504.20194].

A different literal use of the phrase appears in CCVFM. There, Stage I defines an “entropic Sinkhorn coreset” by approximating the empirical target
\[
\hat\rho_{1,n}=\frac1n\sum_{i=1}^n \delta_{x_i}
\]
with a \(K\)-atomic weighted measure
\[
\sum_{k=1}^K w_k\delta_{\mu_k},
\]
where the atoms are optimized by the KL-regularized objective
\[
\min_{\{\mu_k,w_k\},\,T_{ik}\ge 0} \sum_{i,k} T_{ik}\|x_i-\mu_k\|^2 +\lambda\,\mathrm{KL}\!\big(T\,\big\|\,\tfrac{1}{n}\mathbf{1}_n\otimes w\big),
\qquad \sum_k T_{ik}=\tfrac{1}{n},\;\sum_i T_{ik}=w_k.
\]
Holding \((\mu,w)\) fixed, the soft assignments satisfy
\[
T_{ik}\propto w_k \exp\!\left(-\frac{\|x_i-\mu_k\|^2}{\lambda}\right).
\]
The atomic coreset is then lifted to a Gaussian mixture
\[
\tilde\rho_1=\sum_{k=1}^K w_k\,\mathcal N(\mu_k,\Sigma_k),
\]
which serves as a closed-form surrogate source distribution in hierarchical conditional velocity flow matching. Under the explicit compression assumption
\[
W_2(\mu_{\mathrm{in}},\tilde\rho_1)\le C_Q K^{-1/d},
\]
the surrogate transport burden satisfies
\[
D(Q_{\mathrm{CCVFM}})=W_2(\rho_1,\tilde\rho_1)
\]
at \(t=0\) [2605.12951].

These two formulations share the same entropic transport ancestry, but they are not identical notions. CO2 produces a convexly weighted subset of the observed sample, whereas CCVFM optimizes anchor locations continuously and then smooths the resulting atomic measure into a Gaussian mixture [2504.20194][2605.12951].

## 3. Coreset-adjacent compression paradigms

Several important methods are coreset-like without being classical point coresets. Spar-Sink compresses the Sinkhorn kernel itself. For entropic OT and UOT with kernel
\[
K=\exp(-C/\varepsilon),
\]
it constructs a sparse unbiased sketch
\[
\widetilde{K}_{ij}=
\begin{cases}
K_{ij}/p^\ast_{ij} & \text{with prob. } p^\ast_{ij}=\min(1,sp_{ij})\\
0 & \text{otherwise,}
\end{cases}
\]
and then runs generalized Sinkhorn on \(\widetilde K\). The paper is explicit that this is not a point coreset: it sparsifies kernel entries, candidate transport edges, or pairwise interactions. Its main computational claim is to reduce the cost of each Sinkhorn iteration from \(O(n^2)\) to \(\widetilde O(n)\) when the sketch size is \(s=\widetilde O(n)\), with consistency results for the regularized OT and UOT objectives [2306.06581].

Nys-Sink compresses the Gaussian Sinkhorn kernel through a Nyström factorization. For squared Euclidean cost,
\[
K_{ij}=e^{-\eta \|x_i-x_j\|_2^2},
\]
landmarks produce
\[
\widetilde K=VA^{-1}V^\top,
\]
and Sinkhorn scaling is run on the low-rank surrogate rather than the full kernel. The end-to-end guarantee is that the method returns a feasible coupling \(\widehat P\) with
\[
|V_C(\widehat P)-W_\eta(p,q)|\le \varepsilon
\]
and
\[
\mathrm{KL}(\widehat P\,\|\,P^\eta)\le \eta\varepsilon.
\]
This is again not a support-reduced coreset; it is a low-rank operator sketch of the entropic kernel [1812.05189].

Compressed online Sinkhorn adopts yet another compression target: the growing atomic representation of the dual iterates. In the online algorithm,
\[
u_t = \sum_{i=1}^{n_t} \exp\!\left(\frac{q_{i,t}-C(\cdot,y_i)}{\epsilon}\right),\qquad
v_t = \sum_{i=1}^{n_t} \exp\!\left(\frac{p_{i,t}-C(x_i,\cdot)}{\epsilon}\right),
\]
and compression replaces the current atomic measure by a much smaller one that matches selected moments. Under the abstract approximation assumption
\[
\sup_x |f_t(x)-\hat f_t(x)| = O(m_t^{-\zeta}),\qquad
\sup_y |g_t(y)-\hat g_t(y)| = O(m_t^{-\zeta}),
\]
with
\[
m_t=t^{(a-b)/\zeta},
\]
the compressed iterates retain the same asymptotic rate as uncompressed online Sinkhorn,
\[
\hat\delta_N = O\!\left(N^{-\frac{a}{2a+1}}\right).
\]
This is a dynamic compressed summary of the optimization state rather than a reusable coreset of the input measures [2310.05019].

In Gaussian multivariate models, compression can become exact. Gaussian entropic OT and Sinkhorn iterations admit a finite-dimensional recursion in means, covariances, gains, and Riccati equations, with Sinkhorn parameter updates
\[
\theta_{2n+1}=B_{m,\sigma}(\theta_{2n}),\qquad \theta_{2(n+1)}=B_{\bar m,\tau}(\theta_{2n+1}),
\]
and covariance recursion
\[
V_{2(n+1)} = \big(I+\gamma_0 V_{2n+1}\gamma_0'\big)^{-1}, \qquad
V_{2n+1} = \big(I+\gamma_{01}V_{2n}\gamma_{01}'\big)^{-1}.
\]
This is not called a coreset, but it is an exact finite-dimensional reduction of the entropic problem to \(O(d^2)\)-scale parameters [2412.18432].

## 4. Stability criteria and approximation theory

A central theoretical question is what an entropic Sinkhorn coreset should preserve. One answer is plan stability under marginal perturbation. Under a semiconcavity assumption on
\[
y\mapsto c(x,y)+\psi^\nu_\varepsilon(y)
\]
that is uniform in \(x\in \operatorname{supp}(\rho)\), the optimal entropic plans satisfy
\[
KL\!\left(\pi^\varepsilon_{\rho,\mu}\,\middle\|\,\pi^\varepsilon_{\rho,\nu}\right)
\le
KL(\mu\mid \nu)+\frac{\Lambda}{2\varepsilon}W_2^2(\mu,\nu).
\]
The same semiconcavity framework yields exponential convergence of Sinkhorn’s algorithm under Talagrand-type conditions, including unbounded settings and quadratic costs for log-concave marginals. For coreset purposes, this gives a direct route from \(W_2\)-accurate marginal compression to stability of the corresponding entropic coupling [2412.09235].

A second line of work establishes quantitative stability of the dual solution itself. In the quadratic-cost entropic OT problem with regularization parameter \(T\), one has
\[
\|\nabla\psi^\mu-\nabla\psi^\nu\|_{L^2(\mu)} \lesssim W_2(\nu,\tilde\nu),
\]
and
\[
\|\nabla^2\psi^\mu-\nabla^2\psi^\nu\|_{L^1(\mu)} \lesssim W_2(\nu,\tilde\nu)+W_2^2(\nu,\tilde\nu),
\]
with explicit polynomial dependence on \(T\) in the constants. The same analysis yields exponential convergence rates for gradients and Hessians of Sinkhorn iterates in unbounded settings. This suggests that a \(W_2\)-accurate coreset can preserve not only entropic plans but also first- and second-order structure of the associated entropic potentials [2504.11133].

A third perspective uses relative entropy between couplings as the native approximation metric. If a Markov transition \(K\) satisfies a quadratic transport inequality \(T_2(\rho)\), then for any \(\mu\) and comparison transition \(L\),
\[
\|m_K-m_L\|_{2,\mu}^2 \le 2\rho\, H(\mu\times L\mid \mu\times K),
\]
and
\[
\|\sigma_K-\sigma_L\|_{1,\mu}
\le
4\rho\, H(\mu\times L\mid \mu\times K)
+
c_{\mu,L}\,\big(8\rho\,H(\mu\times L\mid \mu\times K)\big)^{1/2}.
\]
Applied to Schrödinger and Sinkhorn bridges, these bounds control conditional means, conditional covariances, and, in linear Gaussian reference models, gradients and Hessians of bridge potentials. This suggests that KL between approximate and exact couplings is a particularly meaningful certification target for entropic Sinkhorn compression [2504.18822].

## 5. Applications and empirical manifestations

The most direct empirical use of Sinkhorn coresets as weighted dataset summaries is in Sinkhorn-divergence compression. CO2 reports a practical application to subsampling image data, compressing the full \(70{,}000\)-point MNIST dataset to a coreset of size \(m=1000\) while substantially improving Sinkhorn reconstruction relative to random subsampling and also better preserving label proportions [2504.20194].

In generative modeling, the CCVFM construction makes the coreset an explicit algorithmic object. Reported Stage-I coreset sizes include \(K=2000\) for MNIST, \(K=10000\) for CIFAR-10, \(K=5000\) or \(K=10000\) for ImageNet-32, and \(K=10000\) for CelebA-HQ latent space. The paper reports that increasing \(K\) improves performance on MNIST, with \(\mathrm{FID}_{50k}=3.79\) at \(K=500\), \(1.93\) at \(K=1000\), and \(1.09\) at \(K=2000\) for \(21\) NFE. On ImageNet-32, under matched HRF2 architecture and protocol, CCVFM reports \(12.55\) versus \(20.29\) at \(11\) NFE, \(9.51\) versus \(12.49\) at \(21\) NFE, and \(8.76\) versus \(9.02\) at \(51\) NFE, with the main architectural difference being the coreset-induced surrogate source rather than isotropic Gaussian noise [2605.12951].

Coreset-adjacent entropic Sinkhorn compression has also been deployed in scientific and streaming settings. Spar-Sink reports synthetic-data improvements over mainstream competitors in both estimation error and speed, together with a real-world echocardiogram analysis in which it effectively estimates and visualizes cardiac cycles and performs as well as classical Sinkhorn for predicting the end-systole time point while requiring significantly less computational time. Compressed online Sinkhorn reports practical gains from periodically compressing the dual atomic representation, with wall-clock reductions such as \(367\)s to \(130\)s in \(1\)D Fourier compression and \(386\)s to \(92\)s in the \(5\)D example while keeping potential and relative objective errors competitive [2306.06581][2310.05019].

The point-cloud registration literature provides a further application-adjacent motivation. Sinkhorn-CPD reformulates rigid registration as fully unbalanced entropic OT with dual KL marginal penalties and generalized Sinkhorn iterations, with per-outer-iteration complexity
\[
\mathcal O(MND)+\mathcal O(LMN),
\]
and total complexity
\[
\mathcal O(TMN(D+L)).
\]
Although it does not construct a coreset, it makes explicit the dense pairwise bottleneck and the weighted-measure structure that a future entropic Sinkhorn coreset for registration would need to preserve [2606.16672].

## 6. Conceptual boundaries, limitations, and open directions

The literature uses the phrase “Entropic Sinkhorn Coreset” in several non-equivalent ways. A strict subset coreset is exemplified by CO2, which outputs a convexly weighted submeasure of the empirical sample. An optimized atomic approximation with moved anchors is exemplified by CCVFM. Kernel-entry sparsification, low-rank Nyström approximation, and compressed dual-iterate representations are better described as edge sparsifiers, operator sketches, or dynamic compressed summaries rather than classical coresets [2504.20194][2605.12951].

Several recurring misconceptions are therefore addressed directly in the literature. Cuturi’s original entropic Sinkhorn formulation is foundational for regularized OT and matrix scaling, but it does not provide a coreset construction [1306.0895]. Spar-Sink is explicitly an importance-sampled sparse interaction or kernel representation, not a support-point coreset [2306.06581]. Nys-Sink compresses the kernel operator, not the input measures [1812.05189]. Compressed online Sinkhorn preserves the current dual state rather than producing a reusable summary of the original distributions [2310.05019].

The direct coreset theory is also qualified. CO2 is asymptotic, relies on second-order Hadamard differentiability, and the practical Sinkhorn kernel
\[
G_n=\varepsilon n(I-(\pi_{\mathbb P_n,\mathbb P_n}^\varepsilon)^2)^{-1}\pi_{\mathbb P_n,\mathbb P_n}^\varepsilon
\]
is acknowledged to be computationally cumbersome, which is why the paper advocates cheaper surrogates based on the empirical Sinkhorn plan matrix itself [2504.20194]. In CCVFM, the key compression rate
\[
W_2(\mu_{\mathrm{in}},\tilde\rho_1)\le C_Q K^{-1/d}
\]
is stated as an assumption on the deployed Stage-I output, not as a theorem for finite-iteration entropic Sinkhorn plus Gaussian-mixture lifting [2605.12951].

The stability-based route to coreset certification is similarly powerful but not turnkey. Semiconcavity, \(T_2\), log-concavity, or Gaussian-reference assumptions provide clear perturbation bounds for entropic plans, potentials, and Sinkhorn bridges, but these are strongest in continuous or smoothed settings and do not by themselves produce subset-selection rules or coreset-size formulas. This suggests that a mature theory of entropic Sinkhorn coresets will likely combine three ingredients: a compression mechanism, a metric such as \(W_2\) or coupling-level KL to certify the compressed instance, and structural regularity conditions under which the induced entropic plans and potentials remain stable [2412.09235][2504.18822][2504.11133].

In that sense, “Entropic Sinkhorn coreset” now names a spectrum rather than a single object: a weighted subset approximating a Sinkhorn divergence, a KL-regularized atomic transport summary, a sparse interaction graph for Sinkhorn updates, a low-rank kernel surrogate, a compressed dual representation in streaming settings, or an exact finite-dimensional parametric reduction in special Gaussian families. The common thread is not the form of the summary, but the attempt to preserve the geometry induced by entropic optimal transport while replacing dense pairwise structure by a smaller computational object.

Source: https://www.emergentmind.com/topics/entropic-sinkhorn-coreset