Wasserstein Distance Joint Estimation

• WDJE is a framework for estimating joint distributions by minimizing the Wasserstein distance between empirical data and structured model families.
• It combines combinatorial, numerical, and algebraic methods to yield piecewise-algebraic solutions with enhanced robustness compared to likelihood-based methods.
• The approach provides explicit solutions in small models and outlines an algorithmic pipeline for larger discrete structured estimation tasks.

Wasserstein Distance Based Joint Estimation (WDJE) is a mathematical and algorithmic paradigm for estimating joint distributions or structured models by minimizing the Wasserstein distance between observed empirical data and structured model families. WDJE frameworks are particularly notable in discrete independence modeling, where the independence model is algebraically a Segre-Veronese variety and the core optimization is a polyhedral norm-distance minimization over the probability simplex. The method yields structured, often piecewise-algebraic solutions with favorable robustness and stability properties compared to likelihood-based approaches (Çelik et al., 2020).

1. Geometric and Algebraic Structure

Let $S = [m_1]\times\cdots\times[m_k]$ be the finite joint state space of $k$ discrete variables and $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$, $n=|S|$, the simplex of probability mass functions. The independence model $\mathcal{M}\subset\Delta_{n-1}$ consists of points $\mu$ admitting the factorization $$\mu_{i_1 \dots i_k} = \prod_{\ell=1}^k p^{(\ell)}_{i_\ell}$$ with $p^{(\ell)}$ a probability vector for the $\ell$th marginal. In algebraic geometry, $\mathcal{M}$ is the Segre variety (labeled case) or a Segre–Veronese variety (with symmetries).

A metric $k$0 on $k$1, specified by $k$2 with $k$3 and $k$4, induces a Wasserstein (optimal transport) metric $k$5 on $k$6: $k$7 The feasible set $k$8 is the Lipschitz polytope, and the Wasserstein norm coincides with the Minkowski functional of its dual—the (unit) Wasserstein ball.

2. Optimization Formulation: Closest Independence Model

WDJE for independence models asks, given observed empirical law $k$9 on $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$0, for the independence distribution $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$1 minimizing

$$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$2

This is a nonconvex optimization problem because $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$3 is nonlinear/algebraic in $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$4. The solution $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$5 is generically piecewise–algebraic: for each $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$6, the optimal $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$7 is such that $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$8 hits a unique face $$\Delta_{n-1} = \{\mu\in\mathbb{R}^n_{\ge 0}:\sum\mu_i = 1\}$$9 of the Wasserstein ball, reducing the problem to a polynomial system on a polyhedral cell of $n=|S|$0.

3. Explicit Solutions, Piecewise Structure, and Algorithms

For small models, WDJE yields fully explicit, piecewise-defined solutions. For example, the symmetric $n=|S|$1 ("Hardy–Weinberg") model produces five algebraic cells in $n=|S|$2 with closed-form expressions involving square roots:

• $n=|S|$3 when $n=|S|$4,
• $n=|S|$5 when $n=|S|$6,
• $n=|S|$7 otherwise.

For the $n=|S|$8-bit model ($n=|S|$9 with $\mathcal{M}\subset\Delta_{n-1}$0), there are eight combinatorial regions, with quadratic and rational explicit expressions for $\mathcal{M}\subset\Delta_{n-1}$1 and $\mathcal{M}\subset\Delta_{n-1}$2. The partitions correspond to faces of the Wasserstein unit ball $\mathcal{M}\subset\Delta_{n-1}$3, e.g., a cube or hexagon, and cell boundaries correspond to combinatorial features ("walls of indecision") (Çelik et al., 2020).

For general models, a three-stage algorithmic pipeline applies:

• Combinatorial preprocessing: Enumerate faces of the Lipschitz polytope
• Numerical optimization: For each face, solve a constrained minimization of a linear functional over $\mathcal{M}\subset\Delta_{n-1}$4 via (mixed-integer) polynomial optimization solvers
• Algebraic postprocessing: For the active cell, solve for Lagrange multipliers to recover $\mathcal{M}\subset\Delta_{n-1}$5 via symbolic algebra (e.g., elimination in Macaulay2).

Computational and algebraic complexity is governed by the $\mathcal{M}\subset\Delta_{n-1}$6-vector of $\mathcal{M}\subset\Delta_{n-1}$7 and the polar degree of $\mathcal{M}\subset\Delta_{n-1}$8 (explicit in the case of Segre–Veronese via Luca’s formula, Theorem 5.2 in (Çelik et al., 2020)).

4. Empirical Case Studies and Partition Structure

The explicit analytic structure of WDJE is illustrated in case studies. For the 2-bit model and Hamming metric, the solution partition of $\mathcal{M}\subset\Delta_{n-1}$9 can be visualized as a subdivision of the simplex into polyhedral regions of algebraic regime, with regions corresponding to “corners” (support concentrated on a vertex of $\mu$0) and “interior” (solution via a quadratic equation). The solution structure reflects the combinatorial geometry of the Wasserstein ball for the chosen ground metric, with transition surfaces corresponding to singularities or multiple minimizers.

5. Implications and Robustness of WDJE

WDJE provides a robust approach to joint estimation and independence approximation. Unlike KL-divergence minimization (MLE), the Wasserstein criterion is less sensitive to degeneracies, particularly when support changes or small probabilities vanish. The piecewise-algebraic structure yields interpretable and in some cases closed-form solutions. For larger models, though complexity grows combinatorially, many faces never activate in practice and effective implementations rely on pruning and algebraic reduction.

Open questions include:

• Combinatorial understanding of $\mu$1 for product metrics beyond complete graphs,
• Reduction of face enumeration in large $\mu$2,
• Extension to latent-variable independence models,
• Analysis of statistical properties: convergence rates, and bias-variance tradeoffs for WDJE versus KL-based methods.

6. Software and Computational Tools

Implementation of WDJE requires computation with polytopes, global polynomial optimization, and symbolic elimination. Standard tools include:

• Polymake: for Lipschitz polytope and face enumeration,
• SCIP: for mixed-integer polynomial or LP relaxations in the numerical step,
• Macaulay2 (or analogous systems): for exact algebraic solution and zero-dimensional ideal decomposition.

This enables algebraic certificates of optimality, symbolic description of solution strata, and a rigorous pipeline for moderate-size models.

7. Summary and Outlook

Wasserstein Distance Based Joint Estimation recasts the estimation of (e.g., independence) models in the probability simplex as a structured, piecewise-algebraic minimization over polyhedral norms derived from optimal transport geometry. The combination of combinatorial, algebraic, and numerical approaches enables explicit solution in small cases and a principled blueprint for larger cases. The method’s robustness to support mismatch and degeneracies, together with its geometric transparency, positions WDJE as a canonical framework for structured joint estimation problems in the discrete setting, with ongoing research extending to latent, continuous, and high-dimensional scenarios (Çelik et al., 2020).