Two-Sided Bounds for Entropic Optimal Transport via a Rate-Distortion Integral

Abstract: We show that the maximum expected inner product between a random vector and the standard normal vector over all couplings subject to a mutual information constraint or regularization is equivalent to a truncated integral involving the rate-distortion function, up to universal multiplicative constants. The proof is based on a lifting technique, which constructs a Gaussian process indexed by a random subset of the type class of the probability distribution involved in the information-theoretic inequality, and then applying a form of the majorizing measure theorem.

• The paper presents sharp two-sided bounds for the maximal expected inner product under mutual information constraints using a truncated rate-distortion integral.
• It leverages lifting techniques and majorizing measure arguments to connect optimal transport with information-theoretic metrics.
• The results extend to high-dimensional and regularized settings, offering practical insights for algorithm design in statistical learning.

Two-Sided Bounds for Entropic Optimal Transport via a Rate-Distortion Integral

Overview and Motivation

The paper "Two-Sided Bounds for Entropic Optimal Transport via a Rate-Distortion Integral" (2604.14061) investigates information-constrained and regularized optimal transport problems, focusing on scenarios where one measure is the standard Gaussian. The primary contribution is the establishment of sharp two-sided bounds on the maximal expected inner product between a random vector and a Gaussian under mutual information constraints, leveraging a truncated rate-distortion integral. The methodology hinges upon lifting techniques from information theory and deploying majorizing measure arguments, thus connecting the geometry of probability distributions with information-theoretic metrics.

Main Technical Contributions

Information-Constrained and Regularized Transport Formulation

The study considers two formulations:

• Constrained Optimal Transport: Given measures $\gamma$ (Gaussian) and $\mu$ (arbitrary with finite second moments) on $\mathbb{R}^n$, and a constraint on mutual information $I(Y; Z) \le R$, the maximal expected inner product $\mathbb{E}[\langle Y, Z \rangle]$ over all couplings is denoted $\textsf{w}(\gamma, \mu, R)$.
• Regularized Transport: Introducing a Lagrange multiplier $\beta$ for mutual information, the regularized objective $\textsf{f}(\gamma, \mu, \beta)$ optimizes $\mathbb{E}[\langle Y, Z \rangle] - \beta I(Y; Z)$.

Both settings encapsulate entropic regularization, which is prominent in computational optimal transport and forms the basis for iterative algorithms such as Sinkhorn’s method.

Integral Representation via Rate-Distortion

The principal theorem asserts that $\textsf{w}(\gamma, \mu, R)$ is equivalent (up to universal multiplicative constants) to the integral:

$\mu$0

where $\mu$1 is the rate-distortion function of $\mu$2 for squared Euclidean distortion. This mirrors Dudley’s integral for Gaussian processes, but notably provides both upper and lower bounds rather than only an upper bound as in classical chaining arguments.

For the regularized version, a similar representation arises via a function $\mu$3 constructed from $\mu$4, yielding bounds of the form:

$\mu$5

with exact tensorization, confirming additivity under product measures.

Methodological Refinements

The proof deploys a "lifting" construction: a Gaussian process is indexed by subsets randomly sampled from the type class of $\mu$6, thereby enabling control of mutual information. Unlike previous approaches relying on permutation invariance, the process is nearly stationary due to concentration properties of empirical distributions. Majorizing measure theorems are recovered as corollaries, and the truncated integral precisely restricts attention to settings where concentration holds, thus avoiding overfitting phenomena.

This construction demonstrates sharpness of the bounds in the stationary case and yields an incremental form of the rate-distortion integral with exact tensorization, which could potentially permit more streamlined analytic proofs for majorizing measure results.

Numerical Results and Claims

For $\mu$7, it is shown that both constrained and regularized objective functions converge to absolute constants as $\mu$8 and scale as $\mu$9 for $\mathbb{R}^n$0. The two-sided bounds are thus tight up to universal constants, both for unimodal and multimodal Gaussian measures, and extend via tensorization to high-dimensional settings.

Implications and Future Directions

Theoretical Implications

• Optimal Transport Theory: The results refine classical bounds by providing sharp, dimension-independent inequalities for entropic optimal transport, facilitating precise analysis in high-dimensional statistics and machine learning.
• Information Theory and Statistical Learning: The equivalence between expected inner product objectives and rate-distortion integrals bridges geometric and information-theoretic interpretations, deepening the connection between transport inequalities and data representation trade-offs.
• Majorizing Measure Theorems: The construction simplifies proofs of the celebrated majorizing measure theorem, suggesting the possibility of even more elementary analytic approaches and extending applicability to non-stationary or regularized settings.

Practical Implications

• Algorithmic Design: Entropic regularization is critical for efficient algorithms in optimal transport. The incremental and tensorized integral bounds could inform sharper convergence guarantees and complexity analyses for iterative solvers, especially in high-dimensional applications.
• Regression and Sub-Gaussian Comparison: The bounds support sharper rates in statistical regression and enable comparison theorems for processes exhibiting sub-Gaussian tails.
• Generative Modeling: The focus on couplings with Gaussian marginals is directly relevant for generative models in machine learning, where optimal transport between Gaussian priors and learned distributions is a standard paradigm.

Open Problems and Future Research

• An open question is whether there exists a purely analytic proof of the majorizing measure theorem utilizing the exact tensorization and incremental form provided by the integral bounds, obviating the need for combinatorial lifting arguments.
• Extending these two-sided bounds beyond Gaussian marginals to other classes of measures could broaden applicability, particularly for generative models with non-Gaussian priors.
• Further investigation into the concentration properties underlying the nearly stationary process indexing could refine bounds for non-asymptotic, finite-sample settings.

Conclusion

This paper advances the theory of entropic optimal transport by establishing sharp two-sided bounds for the expected inner product between coupled random vectors under mutual information constraints and regularization. The bounds are described via truncated rate-distortion integrals, providing dimension-independent inequalities and facilitating a unified information-theoretic and geometric analysis. These results yield tighter rates in statistical learning, reinforce the connection to covering arguments and majorizing measure theory, and invite further exploration into algorithmic and analytic simplifications for optimal transport problems.