Convergence of the intermediate global-in-time HJB Newton method

Characterize the convergence properties of an intermediate method that retains the outer Picard decoupling between the Fokker–Planck and Hamilton–Jacobi–Bellman equations, replaces the timestep-by-timestep Newton sweep for the Hamilton–Jacobi–Bellman equation with a single global-in-time Newton solve for the value function, and compares its convergence with those of the fully coupled Newton method and the Picard method.

Background

The paper compares a damped Picard method, which alternates backward Hamilton–Jacobi–Bellman and forward Fokker–Planck solves, with a fully coupled Newton method applied to the entire space-time discretized mean field game system. The Picard method has cheaper localized solves but may stall, while the fully coupled Newton method is more robust but requires substantially larger space-time linear systems.

The authors identify an unexamined intermediate solver: retain the outer Picard separation between the Fokker–Planck and Hamilton–Jacobi–Bellman equations, but solve the Hamilton–Jacobi–Bellman component using one global-in-time Newton system rather than separate timestep-by-timestep Newton solves. They state that this method may offer a compromise between robustness and computational cost, but leave its convergence properties relative to the two studied methods unresolved.