---
title: Gap Maximality Principle in Stochastic Games
url: https://www.emergentmind.com/topics/gap-maximality-principle
type: topic
---

# Gap Maximality Principle in Stochastic Games

to=arxiv_search.search 山大发cript  天天中彩票能json
{"query":"\"maximality inheritance principle\" stochastic games bounded value iteration 2508.06088", "max_results": 5}
to=arxiv_search.search 񹚂ാസcript  尚度քայինjson
{"query":"\"Gap Maximality Principle\" OR \"gap maximizer\" arXiv", "max_results": 10}
to=arxiv_search.search anasiyana  微信天天彩票ңизjson
{"query":"\"Widest Path Games and Maximality Inheritance in Bounded Value Iteration for Stochastic Games\"", "max_results": 5}
“Gap Maximality Principle” is best understood as an *Editor’s term* for the argument pattern called the **maximality inheritance principle** in the analysis of bounded value iteration for reachability in finite stochastic games. In that setting, the central object is a state \(s^\star\) that maximizes the pointwise gap between an upper fixed point and the reachability value, typically \((\nu-V)(s)\). The principle asserts, informally, that under suitable action choices this maximality is inherited by successors, and then by all states reachable under an induced memoryless strategy profile. The resulting max-gap region yields a contradiction with target reachability in the widest-path semantics, establishing uniqueness of the relevant fixed point and convergence of the upper bound without explicit end-component detection [2508.06088].

## 1. Terminological scope and basic idea

The term **“Gap Maximality Principle”** is not formally introduced in the literature under that name. In the relevant stochastic-game source, the actual term is **maximality inheritance principle**, and the repeatedly used technical notion is the **gap maximizer**. The intended object is a state whose gap
\[
(g-h)(s)
\]
is globally maximal among all states, with the paper’s main instances taking \(g=\nu\) and \(h=V\).

The proof pattern has four characteristic features. First, the goal is to show that two fixed points on \([0,1]^S\) coincide, typically the least fixed point of one operator and the greatest fixed point of another. Second, one argues by contradiction and, using finiteness of the state space, selects a gap maximizer \(s^\star\). Third, one proves that maximality is inherited along suitable transitions. Fourth, one derives a contradiction from the existence of the induced max-gap reachable region. In this sense, the “gap” in the informal label refers not to a metric gap in state space, but to the numerical discrepancy between an upper approximation or greatest fixed point and the true reachability value.

The principle arose in a setting where ordinary bounded value iteration can stagnate in the presence of end components. Its significance is therefore algorithmic as well as proof-theoretic: it provides a route to convergence arguments that does not rely on explicit end-component decomposition.

## 2. Stochastic games, bounded value iteration, and the gap

The underlying model is a finite stochastic game
\[
\mathcal{G} = (S, S_\vartriangle, S_\triangledown, s_I, A, \mathrm{Av}, \delta, T),
\]
where \(S = S_\vartriangle \uplus S_\triangledown\) is partitioned into Maximizer and Minimizer states, \(s_I\) is the initial state, \(A\) is a finite action set, \(\mathrm{Av}(s)\neq\emptyset\) is the available-action set, \(\delta : S\times A\to \mathrm{Dist}(S)\) is the transition function, and \(T\subseteq S\) is the target set. For an action \(a\) at \(s\), the successor support is
\[
\mathrm{Post}(s,a):=\{s'\in S:\delta(s,a,s')>0\}.
\]

Strategies are memoryless and pure. For a strategy pair \((\sigma,\tau)\), the induced Markov chain has reachability probability
\[
\mathcal P_s^{(\sigma,\tau)}(\lozenge T),
\]
and the reachability value is
\[
V(s):=\max_\sigma\min_\tau \mathcal P_s^{(\sigma,\tau)}(\lozenge T)
=\min_\tau\max_\sigma \mathcal P_s^{(\sigma,\tau)}(\lozenge T).
\]
The Bellman operator is
\[
(\mathcal B f)(s):=
\begin{cases}
1 & \text{if } s\in T,\\[1mm]
\max_{a\in\mathrm{Av}(s)} \phi_f(s,a) & \text{if } s\in S_\vartriangle\setminus T,\\[1mm]
\min_{a\in\mathrm{Av}(s)} \phi_f(s,a) & \text{if } s\in S_\triangledown\setminus T,
\end{cases}
\]
with
\[
\phi_f(s,a):=\sum_{s'\in S}\delta(s,a,s')\cdot f(s'),
\]
and the value function is the least fixed point
\[
V=\mu\mathcal B.
\]

Bounded value iteration maintains a lower sequence
\[
\ell_{i+1}=\mathcal B\ell_i,\qquad \ell_0=\bot,
\]
and an upper sequence
\[
u_{i+1}=\mathcal U u_i,\qquad u_0=\top.
\]
The pointwise gap is then \(u_i(s)-\ell_i(s)\), or in fixed-point uniqueness arguments the gap is \((\nu-V)(s)\), where \(\nu\) is a greatest fixed point of an upper-bound operator. The stopping criterion is
\[
u(s_I)-\ell(s_I)\le 2\varepsilon,
\]
which guarantees that
\[
\frac12\bigl(\ell(s_I)+u(s_I)\bigr)
\]
is \(\varepsilon\)-close to \(V(s_I)\).

The obstruction comes from end components. Ordinary Bellman iteration may satisfy
\[
\mu\mathcal B \prec \nu\mathcal B.
\]
Then the lower sequence converges to \(V=\mu\mathcal B\), but the upper sequence converges only to \(\nu\mathcal B\), which may remain strictly above \(V\). Informally, non-target end components can sustain spurious upper values, so the gap does not vanish. This failure mode is exactly the context in which the max-gap argument becomes relevant.

## 3. Formal content of gap maximality

A **gap maximizer** for \(g,h\in[0,1]^S\) is a state \(s^\star\in S\) such that
\[
(g-h)(s^\star)\ge (g-h)(s)\qquad\text{for all }s\in S.
\]
In the stochastic-game proof and in the motivating Markov-chain proof, the choice is
\[
(\nu-V)(s^\star)\ge (\nu-V)(s)\qquad\text{for all }s\in S.
\]

The technical engine is the paper’s “max. vs. average” lemma. If \(S\) is finite, \(f:S\to[0,1]\), and \(\delta\) is a distribution on \(S\), then for an \(f\)-maximizer \(s^\star\),
\[
f(s^\star)\ge \sum_{s\in S}\delta(s)\cdot f(s).
\]
Moreover, if equality holds,
\[
f(s^\star)=\sum_{s\in S}\delta(s)\cdot f(s),
\]
then every state in the support of \(\delta\) has the same value:
\[
\delta(s)>0 \implies f(s^\star)=f(s).
\]
This is the exact inheritance mechanism. Once the value at a maximizer equals the average over a successor distribution, every positive-probability successor is itself a maximizer.

The paper isolates this pattern from a classical Markov-chain uniqueness proof. There the modified Bellman operator is
\[
(\mathcal B' f)(s):=
\begin{cases}
1 & \text{if } s\in T,\\
0 & \text{if } s\in S_{=0},\\
\sum_{s'\in S}\delta(s,s')\cdot f(s') & \text{otherwise},
\end{cases}
\]
and the result is
\[
V=\mu\mathcal B'=\nu\mathcal B'.
\]
The key equality in that proof is
\[
(\nu'-V)(s^\star)=\sum_{s'\in S}\delta(s^\star,s')\cdot(\nu'-V)(s').
\]
Because the left side is the maximum of the gap and the right side is its average over successors, all successors inherit maximality. The stochastic-game argument generalizes this prototype from a single probabilistic transition structure to a two-player Bellman/game setting.

## 4. Maximality inheritance in the correctness proof of 2WP-BVI

The principle becomes central in the analysis of **2WP-BVI**, whose updates are
\[
\ell\gets \mathcal B\ell,\qquad u\gets \mathcal W u,
\]
with termination when
\[
u(s_I)-\ell(s_I)\le 2\varepsilon.
\]
Its main fixed-point theorem is
\[
V=\mu\mathcal W=\nu\mathcal W,
\]
so \(\mathcal W\) has a unique fixed point [2508.06088].

The proof has two phases. First, if \(f=\mathcal W f\), then \(f\) is also a fixed point of the ordinary Bellman operator \(\mathcal B\), because
\[
\phi_f(s,a)\le \max_{s'\in \mathrm{Post}(s,a)} f(s'),
\]
so the widest-path Bellman term
\[
\min\bigl(\phi_f(s,a),\max_{s'\in\mathrm{Post}(s,a)} f(s')\bigr)
\]
collapses to \(\phi_f(s,a)\). Since \(V\) is itself a fixed point of \(\mathcal W\), this yields
\[
V=\mu\mathcal W.
\]

Second, to prove
\[
V=\nu\mathcal W,
\]
one assumes \(V\prec \nu\) and chooses a gap maximizer \(s^\star\) with
\[
(\nu-V)(s^\star)>0.
\]
Because targets satisfy \(\nu(s)=V(s)=1\), such a state cannot lie in \(T\).

The argument then splits by controller. If \(s^\star\in S_\vartriangle\), an action \(a_\star\) is chosen optimal for the widest-path operator, yielding
\[
(\nu - V)(s^\star)
\le \phi_{\nu}(s^\star,a_\star)-\phi_V(s^\star,a_\star)
= \phi_{\nu-V}(s^\star,a_\star)
\le \max_{s' \in \mathrm{Post}(s^\star,a_\star)} (\nu - V)(s')
\le (\nu - V)(s^\star).
\]
If \(s^\star\in S_\triangledown\), an action \(a_\star\) is chosen optimal for the reachability Bellman operator, and one obtains similarly
\[
(\nu - V)(s^\star)
\le \phi_{\nu}(s^\star,a_\star)-\phi_V(s^\star,a_\star)
\le (\nu - V)(s^\star).
\]
In either case, all inequalities are equalities, and the max-vs.-average lemma gives
\[
(\nu-V)(s^\star)=(\nu-V)(s')
\qquad\text{for all }s'\in \mathrm{Post}(s^\star,a_\star).
\]

Because the same reasoning applies at every gap-maximizing state, the selected actions define memoryless strategies \((\sigma,\tau)\) such that **every state reachable from \(s^\star\) under \((\sigma,\tau)\) is also a gap maximizer**. Since targets are not gap maximizers, \(T\) is unreachable from \(s^\star\) under these strategies.

The contradiction comes from the greatest-fixed-point semantics of \(\nu=\nu\mathcal W\). Under \((\sigma,\tau)\), Maximizer plays optimally for the widest-path objective, so the widest path width from \(s^\star\) is at least \(\nu(s^\star)\). Because \((\nu-V)(s^\star)>0\), one has \(\nu(s^\star)>0\), hence there must exist a path from \(s^\star\) to \(T\) of width at least \(\nu(s^\star)\). That makes \(T\) reachable from \(s^\star\) under \((\sigma,\tau)\), contradicting the inherited max-gap region. The contradiction eliminates the possibility of a positive persistent gap.

This proof avoids explicit end-component detection. Instead of computing end components and then repairing the upper bound, it shows that any putative positive-gap region would have to be closed under inherited maximality and simultaneously support a positive-width target-reaching path, which is impossible.

## 5. Widest-path games and algorithmic significance

The upper-bound operator of 2WP-BVI is built from **widest path games**. Given an SG and \(u\in[0,1]^S\), the state-action width is
\[
\phi_u(s,a):=\sum_{s'\in S}\delta(s,a,s')u(s').
\]
For an infinite path \(\rho=s_0a_0s_1a_1\ldots\), the path width is
\[
w_u(\rho):=
\begin{cases}
1 & \text{if } s_0\in T,\\
0 & \text{if } s_i\notin T \text{ for all } i\in\mathbb N,\\
\min\bigl(\phi_u(s_0,a_0),\,w_u(s_1a_1s_2a_2\ldots)\bigr) & \text{otherwise}.
\end{cases}
\]
For strategies \((\sigma,\tau)\),
\[
\mathcal W^{(\sigma,\tau)}_{u,s}:=\max\{w_u(\rho):\rho\in \mathrm{Path}^{(\sigma,\tau)}_s\},
\]
and the widest-path value is
\[
W_u(s):=\min_\tau\max_\sigma \mathcal W^{(\sigma,\tau)}_{u,s}.
\]
This induces the global operator
\[
\mathcal W:[0,1]^S\to[0,1]^S,\qquad \mathcal W u:=W_u.
\]

The associated widest-path Bellman operator is
\[
(\mathcal B_u^{\mathrm{wp}} f)(s):=
\begin{cases}
1 & \text{if } s\in T,\\[1mm]
\displaystyle \max_{a \in \mathrm{Av}(s)} \min\left(\phi_u(s,a), \max_{s' \in \mathrm{Post}(s,a)} f(s')\right) & \text{if } s \in S_\vartriangle \setminus T,\\[3mm]
\displaystyle \min_{a \in \mathrm{Av}(s)} \min\left(\phi_u(s,a), \max_{s' \in \mathrm{Post}(s,a)} f(s')\right) & \text{if } s \in S_\triangledown \setminus T.
\end{cases}
\]
The paper proves
\[
\mu\mathcal B_u^{\mathrm{wp}} = W_u.
\]

This construction over-approximates reachability because, after an action choice, Maximizer effectively receives the best successor compatible with the bottleneck width. At the same time, \(\phi_u(s,a)\) preserves the stochastic expectation that still matters for Bellman comparison. This mixture is what makes the contradiction proof with inherited gap maximality possible.

The paper contrasts **2WP-BVI** with the earlier **1WP-BVI**. In 1WP-BVI, player reduction is performed first: at Minimizer states, only actions minimizing \(\phi_\ell(s,a)\) with respect to the current lower bound are retained, reducing the game to an MDP. A 1-player widest-path problem is then solved on that reduced structure. By contrast, 2WP-BVI computes
\[
u\gets \mathcal W u
\]
directly on the original two-player structure. The resulting proof is correspondingly cleaner: the paper emphasizes that 2WP-BVI replaces the earlier infinitary pigeonhole-style reasoning with a direct maximality-inheritance contradiction [2508.06088].

The algorithmic effect is visible in the running example. Ordinary BVI keeps the upper sequence at
\[
\top=(1,1,1,1,1,1),
\]
while the true value is
\[
V=\left(\tfrac45,\tfrac35,1,0,0,0\right).
\]
Under 2WP-BVI, the upper sequence descends:
\[
\top=(1,1,1,1,1,1)\succcurlyeq (1,1,1,0,0,0)\succcurlyeq
\left(1,\tfrac34,1,0,0,0\right)\succcurlyeq
\left(\tfrac{11}{12},\tfrac34,1,0,0,0\right)\succcurlyeq\cdots
\]
and converges to
\[
V=\left(\tfrac45,\tfrac35,1,0,0,0\right).
\]
This suggests that widest-path propagation removes unsupported optimistic values in end-component-like regions without explicitly computing those regions.

## 6. Relation to other maximality principles

The phrase “maximality principle” has multiple unrelated meanings across mathematics and logic, and “Gap Maximality Principle” is therefore terminologically ambiguous. In modal and intuitionistic logic, maximality principles include Fine’s and Esakia’s principles; the relevant distinction there is a strength gap between weaker clopen or algebraic formulations and stronger closed-set formulations, with the former tied to the Boolean Prime Ideal Theorem and the latter to the Axiom of Choice [2412.13706]. In forcing theory, Hamkins-style principles take the modal form \(\Diamond\Box\varphi\to\varphi\) relative to a forcing class \(\Gamma\), yielding schemes such as \(\mathrm{MP}(\kappa,\Gamma)\) rather than numerical gap propagation [1608.05691]. In optimal stopping, the “maximality principle” refers to selecting a free boundary as the maximal solution of a nonlinear ODE below an admissibility curve, again a distinct mechanism [2505.18394].

Against that background, the stochastic-game notion discussed here is specific. It does not assert existence of maximal points, maximal ideals, or maximal forcing persistence. It concerns a **state maximizing a pointwise value gap**, and the crucial claim is that this maximality is inherited along suitable transitions. The inherited-max-gap region is then used to refute the existence of a spurious greatest fixed point above the reachability value. Accordingly, the most precise encyclopedia description is that **“Gap Maximality Principle” is an informal label for the maximality inheritance principle in widest-path-based bounded value iteration, where the maximal quantity is the upper-versus-exact gap \((\nu-V)(s)\)**.

Source: https://www.emergentmind.com/topics/gap-maximality-principle