Information Bottleneck in Optimal Transport (iBOT)

• The paper introduces a novel framework that aligns high-entropy context with low-entropy content using recursive bootstrapping and OT regularization.
• It reformulates the constrained information bottleneck problem as an entropy-regularized optimal transport task, solved via methods like the Generalized Alternating Sinkhorn algorithm.
• The approach offers theoretical convergence guarantees and provides a fresh perspective on hierarchical inference with applications in graph learning and concept bottleneck models.

Information Bottleneck in Optimal Transport (iBOT) denotes a family of ideas at the intersection of information-theoretic compression and optimal transport, but the phrase has a specific and narrower meaning in recent work. In "Information Must Flow: Recursive Bootstrapping for Information Bottleneck in Optimal Transport" (Li, 8 Jul 2025), iBOT is introduced as the problem of aligning high-entropy context $\Psi$ with low-entropy content $\Phi$ when transport or inference must remain information-preserving enough for both prediction and reconstruction. In adjacent literature, the same thematic space also includes exact OT-based reformulations of the classical Information Bottleneck (IB) problem, most notably the posterior-probability formulation of IB as an entropy-regularized OT model in "Information Bottleneck Revisited: Posterior Probability Perspective with Optimal Transport" (Chen et al., 2023). Related but non-equivalent uses appear in graph representation learning via OT-derived curvature (Fu et al., 2024), in concept bottleneck models with OT alignment but without an IB objective (Xie et al., 12 May 2025), and in older KL-based IB formulations for Markov aggregation that are not OT in the Wasserstein sense (Geiger et al., 2013).

1. Conceptual scope and terminology

In the paper that explicitly names the topic, iBOT is defined as a bottleneck created by entropy asymmetry between context and content. The variables are $\Psi$, representing high-entropy, ambiguous, noisy, entangled contextual information, and $\Phi$, representing low-entropy, structured latent content, with the stated asymmetry $H(\Phi) \ll H(\Psi)$ (Li, 8 Jul 2025). The paper argues that any latent encoding $\Psi \to Z$ that attempts to preserve relevance to $\Phi$ cannot simultaneously preserve enough information for both content specificity and context recovery, and treats that incompatibility as the core bottleneck.

The same work places this interpretation inside the Context-Content Uncertainty Principle (CCUP). CCUP formalizes the directional asymmetry through the relations

$$H(\Phi \mid \Psi) > 0, \qquad H(\Psi \mid \Phi) \gg H(\Phi \mid \Psi) > 0,$$

and uses the entropy decomposition

$H(\Phi,\Psi) = H(\Phi) + H(\Psi \mid \Phi)$

to motivate the principle that structure precedes specificity (Li, 8 Jul 2025). In that framework, inference is not treated as a single bottom-up compression map, but as a cycle of bottom-up contextual disambiguation and top-down content reconstruction.

A distinct but technically sharper usage of the IB–OT connection appears in work on the classical IB problem. "Information Bottleneck Revisited: Posterior Probability Perspective with Optimal Transport" (Chen et al., 2023) does not adopt the CCUP vocabulary, but it directly reformulates constrained discrete IB as an entropy-regularized OT problem with additional constraints. In that setting, the OT link is not metaphorical: the posterior variable plays the role of a transport plan, and the objective solves the relevance-compression function $R(I)$ directly.

Work	Core object	Relation to iBOT
(Li, 8 Jul 2025)	Context-content inference cycle	Explicitly introduces the term iBOT
(Chen et al., 2023)	Constrained IB reformulated as OT	Direct OT-based IB solver
(Fu et al., 2024)	VIB with OT-derived Ricci curvature	Indirect IB–OT connection
(Xie et al., 12 May 2025)	OT in concept bottleneck models	Bottlenecked architecture, not IB
(Geiger et al., 2013)	IB for Markov aggregation via KL rate	IB, but not OT in the Wasserstein sense

2. Formal problem statements

The classical IB problem is stated in constrained form as

$\Phi$0

with the Markov chain

$\Phi$1

and is often handled through the Lagrangian

$\Phi$2

The key point in (Chen et al., 2023) is that the fixed-$\Phi$3 Blahut–Arimoto route is not equivalent to solving the constrained problem in general, particularly when the IB curve is not strictly concave.

That paper’s central move is a posterior probability perspective. It rewrites the objective through

$\Phi$4

introduces

$\Phi$5

and obtains the IB-OT model

$\Phi$6

subject to marginal, consistency, and relevance constraints (Chen et al., 2023). The quantity $\Phi$7 is interpreted as a transport plan or coupling between $\Phi$8 and $\Phi$9, so the constrained IB problem becomes an OT-structured entropy-regularized optimization problem rather than merely an analogy.

In contrast, the explicit iBOT paper formulates the bottleneck through conditional entropy and OT regularization over context distributions induced by content states. Its most direct update rule is

$\Psi$0

and it also writes an entropically regularized OT coupling

$\Psi$1

together with the latent factorization

$\Psi$2

and the variational objective

$\Psi$3

(Li, 8 Jul 2025). The formal style is therefore closer to recursive variational inference with OT regularization than to the classical IB Lagrangian.

3. Algorithmic mechanisms

For the exact IB–OT reformulation, the computational contribution is the Generalized Alternating Sinkhorn (GAS) algorithm. GAS alternates over three primal blocks—$\Psi$4, $\Psi$5, and $\Psi$6—and generalizes Sinkhorn scaling through the updates

$\Psi$7

followed by

$\Psi$8

with $\Psi$9 and $\Phi$0 updated in closed form and the relevance multiplier $\Phi$1 obtained by solving a scalar monotone equation via Newton’s method (Chen et al., 2023). The method is designed to solve the constrained problem at a target relevance threshold $\Phi$2, rather than sweeping a fixed Lagrange multiplier.

The explicit iBOT framework proposes a different mechanism: recursive bootstrapping. Its operational cycle starts from a current content state $\Phi$3, samples or computes

$\Phi$4

generates predicted context

$\Phi$5

re-encodes context through

$\Phi$6

and reconstructs or refines content by

$\Phi$7

or by minimizing $\Phi$8 (Li, 8 Jul 2025). The paper refers to this as a “half-cycle right” or recursive predictive coding process and extends it to temporal bootstrapping and spatial bootstrapping.

These two algorithmic families occupy different points in the design space. GAS is an explicit solver for a constrained discrete IB problem with OT structure. Recursive bootstrapping is a broader inferential scheme that uses entropy reduction, top-down priors, and OT regularization to stabilize low-entropy content states.

4. Convergence, geometry, and hierarchical structure

The most explicit convergence claims associated with iBOT appear in (Li, 8 Jul 2025). The paper states a Delta Convergence Theorem: under monotonic uncertainty reduction, contractivity, boundedness, and context stabilization, recursive updates converge to a delta-like attractor

$\Phi$9

and the fixed point minimizes expected conditional entropy,

$H(\Phi) \ll H(\Psi)$0

A second theorem specialized to iBOT assumes continuity and convexity of $H(\Phi) \ll H(\Psi)$1 and of the OT term, then concludes that $H(\Phi) \ll H(\Psi)$2 converges weakly to $H(\Phi) \ll H(\Psi)$3 under the update rule above (Li, 8 Jul 2025). These results are conditional on strong assumptions, and the paper presents them as the formal basis for stable perceptual schemas, motor plans, and hierarchical inference.

The hierarchical extension in the same work introduces temporal latents $H(\Phi) \ll H(\Psi)$4, spatial latents $H(\Phi) \ll H(\Psi)$5, and the spatiotemporal objective

$H(\Phi) \ll H(\Psi)$6

The paper interprets this as delta-seeded inference, in which low-entropy content seeds diffusion along task-relevant pathways (Li, 8 Jul 2025). A plausible implication is that iBOT, in this formulation, is less a single optimization problem than a recursive principle for organizing multiscale inference.

A different geometric bridge between IB and transport appears in graph learning. "Discrete Curvature Graph Information Bottleneck" (Fu et al., 2024) uses the Variational Information Bottleneck together with Ollivier-Ricci curvature, where curvature is defined through the Wasserstein distance between local neighborhood measures,

$H(\Phi) \ll H(\Psi)$7

The paper’s main objective is

$H(\Phi) \ll H(\Psi)$8

This does not insert a full OT optimization directly into the IB objective; instead, it uses OT-derived curvature as a proxy for transport efficiency and message-passing quality.

5. Related bottleneck architectures and common misconceptions

A recurring source of confusion is that not every bottlenecked model with OT is an Information Bottleneck model. "Discovering Fine-Grained Visual-Concept Relations by Disentangled Optimal Transport Concept Bottleneck Models" (Xie et al., 12 May 2025) is a clear example. DOT-CBM replaces coarse image-level concept prediction with an entropically regularized OT alignment between local image patches and textual concept embeddings, with transport plan

$H(\Phi) \ll H(\Psi)$9

concept prediction

$\Psi \to Z$0

and concept inversion masks obtained directly from columns of $\Psi \to Z$1. The model uses a standard concept bottleneck decomposition

$\Psi \to Z$2

but the paper explicitly states that it does not formulate the Information Bottleneck principle, does not optimize a mutual-information objective such as $\Psi \to Z$3, and has no connection stated to the self-supervised method iBOT (Xie et al., 12 May 2025). Its relevance is therefore architectural and interpretability-oriented, not IB-theoretic.

An older but still relevant non-example is "Optimal Kullback-Leibler Aggregation via Information Bottleneck" (Geiger et al., 2013). That paper uses IB to compress a finite Markov chain by minimizing a KL-based upper bound on aggregation error and then applies the agglomerative information bottleneck algorithm. Its key surrogate objective is

$\Psi \to Z$4

derived from KL divergence rate and relevant information loss. The work is explicitly not an OT paper in the Wasserstein or Monge–Kantorovich sense; its geometry is KL-based, not transport-cost-based (Geiger et al., 2013).

These distinctions matter because the label iBOT can otherwise blur three separate research lines: classical IB solved through OT structure, OT-informed bottleneck architectures that are not IB, and conceptual frameworks that use IB and OT together without adopting the standard constrained IB formalism.

6. Empirical status, limitations, and present understanding

The strongest empirical evidence for a direct IB–OT method comes from (Chen et al., 2023). GAS matches theoretical IB curves for jointly Bernoulli and jointly Gaussian models, recovers entire IB curves in a non-strictly-concave example where Blahut–Arimoto returns only two points, and on discretized Gaussian problems achieves speed-up factors roughly $\Psi \to Z$5 to $\Psi \to Z$6 over BA depending on the relevance threshold. On the Iris dataset, GAS produces a complete IB curve over thresholds $\Psi \to Z$7, whereas BA misses many threshold values (Chen et al., 2023). These results support the claim that OT-structured constrained optimization can solve IB regimes that fixed-$\Psi \to Z$8 methods do not recover reliably.

For indirect IB–OT connections, the empirical picture is more mixed. CurvGIB reports best or near-best node classification performance, improved robustness under random edge addition and deletion, and curvature visualizations suggesting that learned transport structure differs from generic rewiring heuristics (Fu et al., 2024). However, the paper also acknowledges a differentiable approximation to Ricci curvature, and the evidence does not fully isolate the specific contribution of the OT-derived curvature bridge relative to other ingredients.

The explicit iBOT framework in (Li, 8 Jul 2025) is different again: it contains no numerical experiments, no datasets, no benchmarks, and no implementation details. Its formal content consists of recursive update rules, conditional-entropy and OT regularization terms, and fixed-point or delta-convergence theorems under strong assumptions. Broader claims about cognition, memory, language, and collective intelligence are therefore theoretical or interpretive rather than experimentally validated (Li, 8 Jul 2025).

The current literature therefore supports a precise but differentiated conclusion. In the narrow algorithmic sense, Information Bottleneck in Optimal Transport is most concretely instantiated by exact OT-based reformulations of the constrained IB problem, where posterior probabilities act as transport plans and generalized Sinkhorn methods solve the relevance-compression function directly (Chen et al., 2023). In the broader conceptual sense, iBOT also names a recursive inference framework in which low-entropy content and high-entropy context are aligned through conditional-entropy minimization, variational latent inference, and OT regularization (Li, 8 Jul 2025). Adjacent works show that OT can also enter bottlenecked learning through graph curvature (Fu et al., 2024) or concept alignment (Xie et al., 12 May 2025), but those mechanisms should not be conflated with a classical IB objective.