---
title: 'Causal Sinkhorn Discrepancy: Theory & Applications'
url: https://www.emergentmind.com/topics/causal-sinkhorn-discrepancy-csd
type: topic
---

# Causal Sinkhorn Discrepancy: Theory & Applications

The Causal Sinkhorn Discrepancy (CSD) is an entropy-regularized variant of the causal optimal transport (OT) metric, tailored to enforce both causal consistency and continuous transport plans when comparing stochastic processes or joint probability distributions. Designed to address challenges in contextual distributionally robust optimization (DRO), CSD provides a tractable and interpretable means for quantifying divergence between probability measures while preserving non-anticipative structures essential for modeling causal systems [2601.11016].

## 1. Formal Definition and Mathematical Structure

Let $\mathcal{X} \subseteq \mathbb{R}^{d_x}$ and $\mathcal{Y} \subseteq \mathbb{R}^{d_y}$ be Polish (or normed) spaces. Given two probability measures $\mathbb{P}, \mathbb{Q} \in \mathcal{P}(\mathcal{X} \times \mathcal{Y})$, reference measures $\mu \in \mathcal{M}(\mathcal{X} \times \mathcal{Y})$, $\nu_{\mathcal{X}} \in \mathcal{M}(\mathcal{X})$, $\nu_{\mathcal{Y}} \in \mathcal{M}(\mathcal{Y})$, regularization parameter $\epsilon \ge 0$ and $p \in [1, \infty)$, define the cost:
$$
c_p\big((\hat x, \hat y), (x, y)\big) := \|x - \hat x\|^p + \|y - \hat y\|^p.
$$
A coupling $\gamma \in \Gamma(\mathbb{P}, \mathbb{Q})$ is causal if for $((\hat X, \hat Y), (X, Y)) \sim \gamma$, it satisfies $X \perp\!\!\!\perp \hat Y \mid \hat X$. The collection of all such couplings is $\Gamma_c(\mathbb{P}, \mathbb{Q})$.

The $p$–Causal Sinkhorn Discrepancy $R_p(\mathbb{P}, \mathbb{Q})$ is
$$
R_p(\mathbb{P}, \mathbb{Q}) := \Bigg( \inf_{\gamma \in \Gamma_c(\mathbb{P}, \mathbb{Q})}\; \mathbb{E}_{((\hat x, \hat y), (x, y))\sim\gamma} \big[\, c_p((\hat x, \hat y), (x, y)) + \epsilon H(\gamma \mid \mu \otimes \nu_{\mathcal{X}} \otimes \nu_{\mathcal{Y}}) \big] \Bigg)^{1/p},
$$
where the relative entropy is
$$
H(\gamma \mid \mu \otimes \nu_{\mathcal{X}} \otimes \nu_{\mathcal{Y}}) = 
\mathbb{E}_\gamma \Big[ \log \frac{d\gamma}{d (\mu\, d\nu_{\mathcal{X}}\, d\nu_{\mathcal{Y}})} \Big].
$$
For $\epsilon = 0$, this reduces to the causal $p$-Wasserstein distance; for $\Gamma_c \to \Gamma$ (dropping causal restrictions), the non-causal Sinkhorn distance is recovered.

## 2. Causal Consistency and Entropic Regularization

The CSD distinguishes itself from the standard Sinkhorn discrepancy—a purely entropic OT metric—by enforcing causal consistency. Standard Sinkhorn distance over $\Gamma(\mathbb{P}, \mathbb{Q})$ allows couplings to “anticipate” future outcomes (i.e., $X$ may depend on $\hat Y$), potentially violating natural temporal or informational orderings. In contrast, CSD restricts to $\Gamma_c(\mathbb{P}, \mathbb{Q})$ where, conditioning on past covariates $\hat X$, future covariates $X$ must not depend on unseen outcomes $\hat Y$. This ensures the resulting transport plans do not introduce feedback loops or causally implausible information flows. In the context of DRO, this yields ambiguity sets that exclude alternatives where future decisions would be made with foreknowledge of future uncertainties [2601.11016, Definition 2.2 and Figure 1.2].

Entropic regularization ($\epsilon > 0$) compels the solution to yield continuous, rather than purely discrete, couplings, enhancing the smoothness and computational tractability of the resulting discrepancy metric.

## 3. Duality, Strong Dual Formulation, and Optimization

Given an empirical distribution $\hat{\mathbb{P}} \in \mathcal{P}(\mathcal{X} \times \mathcal{Y})$, the robust optimization problem
$$
v_P := \max_{\mathbb{P} \in \mathcal{P}(\mathcal{X} \times \mathcal{Y}),\; R_p(\hat{\mathbb{P}}, \mathbb{P})^p \le \rho^p}
\mathbb{E}_{(x,y) \sim \mathbb{P}}[\Psi(f(x), y)]
$$
admits a strong dual formulation:
$$
v_D = \inf_{\lambda \geq 0} \Bigg\{ \lambda \rho^p + 
\mathbb{E}_{\hat x \sim \hat{\mathbb{P}}_x} \left[ 
\lambda \epsilon \log \int_{\mathcal{X}} 
\exp\left(\frac{g(\hat x, x; \lambda)}{\lambda \epsilon}\right) d\nu_{\mathcal{X}}(x)
\right] \Bigg\},
$$
where
$$
g(\hat x, x; \lambda) = \mathbb{E}_{\hat y \sim \hat{\mathbb{P}}_{y|x=\hat x}} \left[
\lambda \epsilon \log \int_{\mathcal{Y}} 
\exp \left( \frac{\Psi(f(x), y) - \lambda c_p((\hat x, \hat y), (x,y))}{\lambda \epsilon} \right) d\nu_{\mathcal{Y}}(y)
\right].
$$
Under mild conditions, strong duality holds and the infimum is strictly attained at a unique $\lambda^*>0$ (by strict convexity in $\lambda$) [2601.11016, Theorem 3.1].

## 4. Structure of the Worst-Case Distribution

If $\lambda^*>0$ solves the dual problem, the worst-case distribution $\mathbb{P}^*$ achieving $v_P$ exhibits the density
$$
\frac{d\mathbb{P}^*(x,y)}{d(\nu_{\mathcal{X}} \otimes \nu_{\mathcal{Y}})} 
= \mathbb{E}_{(\hat x, \hat y) \sim \hat{\mathbb{P}}} \bigg[ 
\alpha(\hat x)\, \beta(\hat x, \hat y; x)\;
\exp\big\{ r(\hat x, x) + s(\hat x, \hat y; x, y) \big\} 
\bigg],
$$
with
\begin{align*}
r(\hat x, x) &:= \mathbb{E}_{\hat y | \hat x} \left[ \log \int \exp(s(\hat x, \hat y; x, y)) d\nu_{\mathcal{Y}}(y) \right], \\
s(\hat x, \hat y; x, y) &:= \frac{\Psi(f(x), y) - \lambda^* c_p((\hat x, \hat y), (x, y))}{\lambda^* \epsilon}, \\
\alpha(\hat x) &:= \left[ \int_{\mathcal{X}} \exp(r(\hat x, x')) d\nu_{\mathcal{X}}(x') \right]^{-1}, \\
\beta(\hat x, \hat y; x) &:= \left[ \int_{\mathcal{Y}} \exp(s(\hat x, \hat y; x, y')) d\nu_{\mathcal{Y}}(y') \right]^{-1}.
\end{align*}
This density is a continuous mixture of Gibbs kernels, in contrast to the discrete support typically arising in non-regularized or non-causal OT settings [2601.11016, Theorem 3.2].

## 5. Theoretical Properties: Metricity, Regularity, and Optimization

- **Metric properties and continuity**: $R_p(\cdot, \cdot)$ is a true metric on $\mathcal{P}(\mathcal{X} \times \mathcal{Y})$ under causal couplings. The entropic regularization ($\epsilon > 0$) ensures strong convexity and smoothness in the dual formulation, providing stability and numerical tractability.
- **Tractability**: The dual reformulation reduces the CSD evaluation to a single-level minimization over $\lambda$ involving expectations of log-sum-exp functions. For parametric decision rules $f_\theta$, the outer min-max problem can be expressed as a multi-level stochastic compositional optimization.
- **Optimization methods**: Sample-Average Approximation (SAA) achieves $\delta$-optimality in $O(\delta^{-2})$ Monte Carlo samples, and a Stochastically Corrected Stochastic Compositional (SCSC) gradient method achieves an $\varepsilon$-stationary point in $O(\varepsilon^{-4})$ gradient iterations, which matches lower complexity bounds for nonconvex stochastic optimization [2601.11016, Propositions 4.2, 4.3; Theorems 5.1, 5.2].

## 6. Causal Sinkhorn Discrepancy in Contextual DRO

CSD enables the formulation of ambiguity sets in contextual DRO that respect both entropy-induced continuity and causal transport restrictions. Given empirical joint law $\hat{\mathbb{P}}$, the ambiguity set is
$$
\mathcal{A}_\rho := \{ \mathbb{P} \in \mathcal{P}(\mathcal{X} \times \mathcal{Y}) : R_p(\hat{\mathbb{P}}, \mathbb{P})^p \leq \rho^p \}.
$$
This set excludes (i) discrete couplings for $\epsilon>0$ (thus enforcing continuity), and (ii) causally implausible transports (enforcing $X \perp\!\!\!\perp \hat Y | \hat X$). The resulting contextual DRO problem is:
$$
\inf_{f \in \mathcal{F}} \sup_{\mathbb{P} \in \mathcal{A}_\rho} \mathbb{E}_{(x,y) \sim \mathbb{P}} [\Psi(f(x), y)].
$$
This structure ensures that worst-case analysis never incorporates information unavailable under natural causal orderings and that predictions/policies are robust to plausible distributional perturbations consistent with the observed data-generating process [2601.11016, Section 6].

## 7. Summary Table: Key Distinctions

| Property                   | Standard Sinkhorn   | Causal Sinkhorn Discrepancy (CSD)     |
|----------------------------|---------------------|--------------------------------------|
| Coupling Constraint        | All couplings ($\Gamma$) | Causal couplings ($\Gamma_c$)      |
| Feedback Allowed           | Yes                 | No (enforces $X \perp\!\!\!\perp \hat Y | \hat X$) |
| Entropic Regularization    | Optional            | Required for continuity, tractability |
| Metricity                  | Non-causal OT metric| Causal metric on $\mathcal{P}(\mathcal{X} \times \mathcal{Y})$ |
| Worst-case Distribution    | Discrete/Continuous | Continuous mixture of Gibbs kernels  |

The Causal Sinkhorn Discrepancy is a foundational tool for robust, interpretable, and causally consistent optimization in modern machine learning and statistics [2601.11016].

Source: https://www.emergentmind.com/topics/causal-sinkhorn-discrepancy-csd