Extension of Krasnoselskii–Mann and Halpern iteration to stochastic two-player games

Establish whether Krasnoselskii–Mann and Halpern iteration produce optimal policies for arbitrary stochastic two-player games, and determine whether these iterative methods have polynomial smoothed complexity in the stochastic setting or admit a counterexample demonstrating that such complexity fails.

Background

The paper’s correctness analysis for Krasnoselskii–Mann and Halpern iteration relies on determinism. The authors observe that optimal-policy production is not immediately established for arbitrary stochastic games, even though it is known in the one-player setting cited by the paper.

Consequently, the unresolved problem has two stages: first extend the optimal-policy result to two-player stochastic games; then analyze smoothed complexity or determine whether a counterexample prevents such a result. The paragraph also notes that Howard’s all-switches rule already has a negative smoothed-complexity result for stochastic MDPs, motivating the search for different policy-iteration rules.

References

For value iteration methods, we note that our analysis of the correctness of Krasnoselskii--Mann and Halpern iteration uses the fact that the underlying games are deterministic. It is not difficult to show that these methods will eventually produce optimal policies for constant-value stochastic games, but it is not immediately clear if they produce optimal policies for arbitrary stochastic games.

— Smoothed analysis of policy iteration and value iteration for deterministic discounted and mean-payoff games played on any graph  (2610.22149 - Loff et al., 15 Oct 2026) in Section Conclusion and future work, paragraph 2

In our work, we have used general results showing that the error bound behaves like $O(1/\sqrt{k})$, but we do not know if this rate is optimal in the setting of deterministic games.

— Smoothed analysis of policy iteration and value iteration for deterministic discounted and mean-payoff games played on any graph  (2610.22149 - Loff et al., 15 Oct 2026) in Section Conclusion and future work, paragraph 3