Maxitive Donsker–Varadhan Theorem

• The Maxitive Donsker–Varadhan theorem is a framework in possibility theory that replaces additive constructs with suprema and maxitive analogues to handle epistemic uncertainty.
• It establishes a dual variational representation that mirrors the classical formula by substituting integrated expectations with supremal operations and KL divergence with max-relative entropy.
• The theorem underpins coordinate-ascent updates in possibilistic variational inference, facilitating robust optimization in maxitive exponential family models.

The Maxitive Donsker–Varadhan theorem provides the cornerstone variational representation for inference within the framework of possibility theory, mirroring the classical additive Donsker–Varadhan formula while replacing probability-centric constructs with maxitive analogues. This formulation underpins a rigorous approach to possibilistic variational inference (PVI), where epistemic uncertainty is modeled via possibility functions, and essential operations such as integration, expectation, and divergence are inherently maxitive rather than additive. The theorem enables coordinate-ascent update schemes analogous in spirit to classical variational inference, but based on modes and max-relative entropy.

1. Classical Donsker–Varadhan Formula

The classical Donsker–Varadhan variational principle provides a dual representation for the cumulant generating function in terms of probability measures and relative entropy. For a measurable space $(\Theta, \mathcal{F})$ with probability measure $\nu$ and measurable function $h: \Theta \rightarrow \mathbb{R}$ such that $\int e^{h} \, d\nu < \infty$, the formula is:

$$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$

The supremum is attained at the Gibbs measure $d\rho^* / d\nu \propto e^{h(\theta)}$. This result underpins much of variational inference (VI), where expectations and $\mathrm{KL}$ divergences govern the optimization objective and its gradients.

2. Key Maxitive Structures in Possibility Theory

Possibility theory replaces additive probabilistic constructs with structures tailored to imprecise or incomplete information:

• Possibility Functions: A function $\pi: \Theta \rightarrow [0,1]$ with $\sup_\theta \pi(\theta) = 1$. The set $\mathcal{F}(\Theta)$ consists of all such $\nu$0, ordered pointwise ($\nu$1 iff $\nu$2).
• Maxitive Integral: For a nonnegative "reward" $\nu$3, the maxitive (supremal) integral with respect to $\nu$4 is $\nu$5, or $\nu$6 if $\nu$7 already incorporates $\nu$8.
• Maxitive Expectation: The counterpart to expectation is the mode; for a real-valued $\nu$9, $h: \Theta \rightarrow \mathbb{R}$0.
• Max-Relative Entropy: For $h: \Theta \rightarrow \mathbb{R}$1, define $h: \Theta \rightarrow \mathbb{R}$2.

These constructs accommodate maximally informative choices under epistemic uncertainty, bypassing the need for probabilistic additivity.

3. Maxitive Donsker–Varadhan Theorem

Let $h: \Theta \rightarrow \mathbb{R}$3 be a prior possibility function and $h: \Theta \rightarrow \mathbb{R}$4 a nonnegative loss. Define the maxitive model evidence ("consistency") and its logarithm:

$h: \Theta \rightarrow \mathbb{R}$5

The Maxitive Donsker–Varadhan theorem asserts a saddle-point-like characterization:

$h: \Theta \rightarrow \mathbb{R}$6

• The sup-inf form (3.1) is maximized by any $h: \Theta \rightarrow \mathbb{R}$7, while the inf-sup form (3.2) is minimized by any $h: \Theta \rightarrow \mathbb{R}$8, where the "Gibbs" posterior possibility function $h: \Theta \rightarrow \mathbb{R}$9 is

$\int e^{h} \, d\nu < \infty$0

The additive integral $\int e^{h} \, d\nu < \infty$1 of the classical theorem is replaced with a supremum, and the Kullback–Leibler divergence by $\int e^{h} \, d\nu < \infty$2. This represents a maxitive duality structure inherent in possibility theory (Singh et al., 26 Nov 2025).

4. Sketch of Proof and Theoretical Parallels

The proof begins by expressing the log-consistency as

$\int e^{h} \, d\nu < \infty$3

For any $\int e^{h} \, d\nu < \infty$4 with $\int e^{h} \, d\nu < \infty$5,

$\int e^{h} \, d\nu < \infty$6

taking the supremum over $\int e^{h} \, d\nu < \infty$7 yields (3.1), with equality at $\int e^{h} \, d\nu < \infty$8. Dually, as $\int e^{h} \, d\nu < \infty$9,

$$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$0

and taking the supremum over $$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$1 recovers (3.2), again tight at $$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$2. The structure thus mirrors the classical Donsker–Varadhan proof—integrals are replaced with suprema, and KL divergence with max-relative entropy.

5. Maxitive Exponential Families and Possibilistic Variational Inference

Maxitive exponential families provide tractable variational classes for PVI:

• For $$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$3,
• $$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$4 ensures $$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$5.

The lower consistency bound (CBO)

$$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$6

is the PVI analogue of the ELBO. Maximizing this over $$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$7 yields the best approximation within the chosen class, i.e.,

$$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$8

6. Coordinate-Ascent Updates and Connections to Classical Variational Inference

Coordinate ascent in the PVI framework for exponential families is justified via the Maxitive Donsker–Varadhan theorem. For any maximizer

$$\Lambda(h) := \log \int e^{h(\theta)} \nu(d\theta) = \sup_{\rho \in \mathcal{P}(\Theta)} [ \mathbb{E}_\rho[h] - \mathrm{KL}(\rho\|\nu) ]$$9

a legitimate ascent step is

$d\rho^* / d\nu \propto e^{h(\theta)}$0

where $d\rho^* / d\nu \propto e^{h(\theta)}$1 is the mode. For key families:

• Gaussian (known covariance $d\rho^* / d\nu \propto e^{h(\theta)}$2): With $d\rho^* / d\nu \propto e^{h(\theta)}$3, $d\rho^* / d\nu \propto e^{h(\theta)}$4, the update formula recovers standard gradient descent in the mean parameter $d\rho^* / d\nu \propto e^{h(\theta)}$5.
• Binomial ($d\rho^* / d\nu \propto e^{h(\theta)}$6 trials): $d\rho^* / d\nu \propto e^{h(\theta)}$7, standard parameter $d\rho^* / d\nu \propto e^{h(\theta)}$8, yields the familiar gradient-descent recursion on $d\rho^* / d\nu \propto e^{h(\theta)}$9.

These updates strongly parallel classical variational coordinate ascent, but all expectations and entropic quantities are maxitive.

7. Implications and Research Directions

The Maxitive Donsker–Varadhan theorem enables principled variational inference in contexts dominated by epistemic uncertainty, imprecision, or incomplete information, where additivity is not justified. The PVI methodology with maxitive divergences admits direct analogues of probabilistic update rules, facilitating robust and interpretable optimization in exponential-family models. The construction and analysis of new variational families, as well as the extension to more complex loss landscapes and hierarchical models, represent active research directions within possibilistic inference frameworks (Singh et al., 26 Nov 2025).