Existence and uniqueness of the infinite-horizon Bellman equation for general nonlinear systems

Establish the existence and uniqueness of solutions to the infinite-horizon Bellman equation with drift for nonlinear discrete-time systems, namely: find a function J∞(·,d) and scalar d such that for all x ∈ R^{n_x}, J∞(x,d) + d = min_{u ∈ U} max_{w ∈ W} [ L(x,u) + I_X(x) + J∞(f(x,u) + w, d) ], where x_{k+1} = f(x_k,u_k) + w_k describes the dynamics with arbitrary nonlinear f, L is the stage cost, and X ⊆ R^{n_x}, U ⊆ R^{n_u}, W ⊆ R^{n_x} are the given convex state, input, and disturbance sets (I_X denotes the indicator function of X). Determine whether such a solution (J∞, d) exists and is unique in this general setting for arbitrary nonlinear f and L.

Background

The paper considers the finite-horizon min–max dynamic programming recursion J_{k+1}(x) = min_{u∈U} max_{w∈W} [ L(x,u) + I_X(x) + J_k(f(x,u)+w) ], J_0(x) = I_X(x), and discusses the formal infinite-horizon limit J∞(x,d) = lim_{k→∞} (J_k(x) − k·d), which, if it exists, should satisfy the infinite-horizon Bellman equation with drift.

While this limit leads to the equation J∞(x,d) + d = min_{u∈U} max_{w∈W} [ L(x,u) + I_X(x) + J∞(f(x,u)+w, d) ], the paper explicitly notes that, in the general setting of arbitrary nonlinear dynamics f and stage cost L, the existence and uniqueness of such an equation are not established. Resolving this would clarify the theoretical foundation for using the infinite-horizon value function as a control Lyapunov function in the proposed framework.