Backpropagation in unstable diffusions

Abstract: We derive the adjoint path-kernel method for the parameter-gradient of SDEs, where the observable is averaged at a particular time or over the stationary measure. Its cost is almost independent of the number of parameters; it extends the conventional backpropagation method to cases with gradient explosion. It works for non-hyperbolic systems with multiplicative noise controlled by parameters. We derive a Monte-Carlo-type algorithm and demonstrate it on the 40-dimensional Lorenz 96 system.