The role of information in high dimensional stochastic optimal control (2105.05974v2)

Abstract: The stochastic optimal control of many agents is an important problem in various fields. We investigate the problem of partial observations, where the state of each agent is not fully observed and the control must be decided based on noisy observations. This results in a high-dimensional Markov decision process that is impractical to handle directly. However, in the limit as the number of agents approaches infinity, a finite-dimensional mean-field optimal control problem emerges, which coincides with the problem of full information. Our main contribution is to investigate a central limit theorem for the Gaussian fluctuations of the mean-field optimal control. Our findings show that partial observations play an essential role in the fluctuations, in contrast to the mean-field limit. We establish a method that uses an approximate Kalman filter, which is straightforward to compute even when the number of states is large. This provides some theoretical evidence of the efficacy of Kalman filter methods that are commonly used across a range of practical applications. We demonstrate our results with two examples: an epidemic model with observations of positive tests and a simple two-state model that exhibits a phase transition at which point the fluctuations diverge.