General Decentralized Stochastic Optimal Control via Change of Measure: Applications to the Witsenhausen Counterexample

Abstract: In this paper we present global and person-by-person (PbP) optimality conditions for general decentralized stochastic dynamic optimal control problems, using a discrete-time version of Girsanov's change of measure. The PbP optimality conditions are applied to the Witsenhausen counterexample to show that the two strategies satisfy two coupled nonlinear integral equations. Further, we prove a fixed point theorem in a function space, establishing existence and uniqueness of solutions to the integral equations. We also provide numerical solutions of the two integral equations using the Gauss Hermite Quadrature scheme, and include a detail comparison to other numerical methods of the literature. The numerical solutions confirm Witsehausen's observation that, for certain choices of parameters, linear or affine strategies are optimal, while for other choices of parameters nonlinear strategies outperformed affine strategies.