A Generalization of Bellman's Equation with Application to Path Planning, Obstacle Avoidance and Invariant Set Estimation

Abstract: The standard Dynamic Programming (DP) formulation can be used to solve Multi-Stage Optimization Problems (MSOP's) with additively separable objective functions. In this paper we consider a larger class of MSOP's with monotonically backward separable objective functions; additively separable functions being a special case of monotonically backward separable functions. We propose a necessary and sufficient condition, utilizing a generalization of Bellman's equation, for a solution of a MSOP, with a monotonically backward separable cost function, to be optimal. Moreover, we show that this proposed condition can be used to efficiently compute optimal solutions for two important MSOP's; the optimal path for Dubin's car with obstacle avoidance, and the maximal invariant set for discrete time systems.