Automated Approach for Solving Infinite-state Polynomial Reachability Games

Abstract: Reachability games are two-player games played on a graph, where the objective of $\texttt{REACH}$ player is to reach the target set whereas the objective of $\texttt{SAFE}$ player is to stay away from the target set. Reachability games have important applications in artificial intelligence and reactive synthesis, and many of these applications give rise to infinite-state reachability games. In this paper, we study turn-based reachability games on infinite-state graphs defined over valuations of a finite set of real variables. We consider the problem of determining the existence of and computing a winning strategy for $\texttt{REACH}$ player. Our contributions are twofold. First, we propose ranking certificates for reachability games, a sound and complete proof rule for proving that $\texttt{REACH}$ player has a winning strategy from the specified initial state. Second, we consider polynomial reachability games, where transitions and objectives are described by polynomial constraints over real variables, and propose a fully automated algorithm for computing a winning strategy for $\texttt{REACH}$ player together with a formal correctness witness in the form of a ranking certificate. The algorithm is sound, semi-complete, and runs in sub-exponential time. Our experiments demonstrate the ability of our method to solve challenging examples from the literature that were out of the reach of existing methods. Specifically, for the classical Cinderella-Stepmother game, we are able to compute an optimal winning strategy for an arbitrary precision parameter for the first time.

• The paper introduces ranking certificates that provide a clear progress measure to verify winning strategies in infinite-state polynomial reachability games.
• It develops a fully automated, template-based synthesis method utilizing quantifier elimination and SMT solvers for extracting memoryless winning strategies.
• Empirical benchmarks on classical and non-linear polynomial games demonstrate the approach's practical efficacy and its state-of-the-art contribution to controller synthesis.

Automated Solution of Infinite-state Polynomial Reachability Games

Introduction

The paper "Automated Approach for Solving Infinite-state Polynomial Reachability Games" (2605.10169) provides a rigorous computational framework for analyzing and synthesizing strategies in infinite-state reachability games described by polynomial constraints over real variables. Reachability games are fundamental models in formal verification, controller synthesis, and AI planning, capturing adversarial dynamics with one player seeking to reach a target set (REACH) and the other aiming to avoid it (SAFE). While finite-state cases are well understood algorithmically, the infinite-state scenario is undecidable in general, demanding new methods especially when transitions and objectives involve polynomial relations.

Game Model and Problem Formulation

The setting consists of two-player turn-based games defined on graphs with states parameterized by continuous variables. Each transition is governed by a (possibly nonlinear) polynomial guard and update, and the sets of target and allowed states are semialgebraic. The challenge is to determine, for a given initial state, whether REACH has a strategy guaranteeing reaching the target regardless of SAFE's choices, and if so, to algorithmically synthesize such a strategy.

A key modeling aspect is the focus on memoryless determinacy, which holds for reachability objectives even in infinite settings: if a player can win, it suffices to act based only on the current state, not the play history.

Ranking Certificates: A Sound and (Semi-)Complete Proof Principle

The core theoretical contribution is the introduction of ranking certificates: functions from game states to the reals, generalizing program-ranking functions to adversarial settings. A ranking certificate $f$ encodes, via structural inequalities, a progress measure certifying that the REACH player can drive any play starting from the initial state to the objective set, regardless of SAFE’s choices, while strictly decreasing $f$ and maintaining non-negativity.

The definition refines classical ranking construction by distinguishing player ownership of states:

• For states controlled by SAFE, $f$ must decrease for all successor states.
• For REACH-owned nodes, it suffices that a single successor achieves progress.

These conditions are sufficient for the existence of a REACH-winning memoryless strategy. Moreover, the construction encodes not only solvability but also admits extraction of explicit winning strategies from the certificate itself.

The completeness analysis identifies a crucial boundary: these certificates are complete only for games where SAFE has finitely many choices in every state. This is a strict strengthening over compactness; even compact but infinite choices can defeat the existence of a ranking certificate (see Example~\ref{ex:compact-example}), reflecting inherent lower-bound limitations in the certificate paradigm.

Figure 2: Illustration of the game from Example~\ref{ex:compact-example}; even with a compact action space for SAFE, no ranking certificate can exist if the time to win is unbounded.

Automated Synthesis via Template-based Polynomial Optimization

Building on the theoretical insights, the paper presents a fully automated algorithm for the synthesis of both ranking certificates and memoryless REACH-winning strategies in polynomial reachability games. The computational framework operates as follows:

1. Template Instantiation: From the user, the degree $D$ of candidate polynomial functions is fixed. Templates for both the ranking certificate and the REACH strategy—parametric in unknown coefficients—are instantiated for every game location.
2. Constraint Encoding: Soundness requirements on ranking certificates and strategy extraction are formalized as universally quantified polynomial inequalities (arising from guards, updates, and objective predicates).
3. Quantifier Elimination and Reduction: These constraints are compiled, following the approach in [asadi], into existential quantifier-free systems suitable for state-of-the-art SMT solvers, leveraging results such as Putinar's Positivstellensatz and Farkas’ lemma.
4. Solving and Witness Extraction: The resulting systems are dispatched to backends (e.g., Z3, MathSAT5), and upon success, concrete polynomial ranking certificates and explicit strategies are returned. If no solution is found, either the winning strategy does not exist in the class considered or a higher-degree template is required.

Theoretical Guarantees

The algorithm is sound (produced certificates and strategies are valid) and semi-complete (guaranteed completeness for games where winning polynomial certificates and strategies exist and the template degree is sufficiently high, under compact state spaces). Computational complexity is sub-exponential in the game size for any fixed degree $D$ [grigor1988solving].

Empirical Evaluation and Benchmarks

The authors implement a prototype leveraging PolyQEnt [chatterjee2024polyqent] for quantifier elimination and demonstrate substantial empirical advances:

• Cinderella-Stepmother Game: For the classic infinite-state challenge, the tool can—for the first time—synthesize optimal strategies for bucket capacities $U = 2-\epsilon$ with symbolic $\epsilon > 0$, whereas prior methods failed for $U \geq 1.7$ or provided no completeness guarantees.
• Polynomial (Non-Linear) Variants: On generalizations involving Euclidean constraints (e.g., $\|\mathbf{b} - \mathbf{b'}\|_2 \leq 1$), out-of-reach for prior linear arithmetic-based tools, the new approach consistently solves fixed-parameter instances and outperforms baselines.
• Faulty Robot Benchmark: In benchmarks where both players have infinite continuous action spaces, the prototype synthesizes witnesses and winning strategies in under a minute when alternatives time out.

These results showcase both the expressiveness (handling non-linear transitions and symbolic parameters) and practical efficacy of the approach, setting new state-of-the-art in automated infinite-state reachability game solving.

Limitations and Directions for Future Work

Although the framework extends provably and practically beyond existing solvers, it is restricted to polynomial games and reachability objectives—handling more general relations (e.g., transcendental constraints) or broader classes of objectives (e.g., $\omega$-regular, quantitative payoffs) remains open. Additionally, reliance on high-degree templates or numerical instability in quantifier elimination may introduce challenges for highly complex or finely structured games.

Implications and Future Advancement in AI

This work delivers a substantive advance in the synthesis and certification of strategies for infinite-state adversarial systems. From a theoretical perspective, the ranking certificate paradigm refines the structure of witnesses in infinite-state synthesis, making explicit the progress constraints that enable strategy construction. Practically, the automated synthesis approach enables robust controller synthesis for systems modeled with polynomial dynamics in continuous spaces—a crucial step for AI planning in robotics, cyber-physical systems, and reactive synthesis.

Potential future developments include the extension of the framework to settings with richer objectives (e.g., combining safety, liveness, quantitative rewards), scalability optimizations harnessing problem or domain symmetries, and integration with verified controller extraction for embedded and cyber-physical deployments.

Conclusion

The ranking certificates and the template-based synthesis pipeline introduced in "Automated Approach for Solving Infinite-state Polynomial Reachability Games" (2605.10169) provide strong theoretical and empirical foundations for AI and formal methods applications in adversarial systems with infinite states and polynomial transitions. By bridging sound proof principles with automated, scalable synthesis, this work both elucidates the nature of infinite-state strategic reasoning and furnishes practical tools for correct-by-construction controller design.