Develop stochastic natural-gradient and variational-inference methods

Develop and implement stochastic natural-gradient methods for variational inference over probabilistic programs using execution-level estimates of score and velocity energies, and establish their computational details and validity.

Background

The paper derives a variational formulation of the natural gradient in which the objective is an expectation of score and velocity energies. This suggests estimating the objective directly from program executions rather than explicitly constructing or inverting the score-velocity metric. The implementation and rigorous development of these methods are left unresolved.