Optimization of Bregman Variational Learning Dynamics

Abstract: We develop a general optimization-theoretic framework for Bregman-Variational Learning Dynamics (BVLD), a new class of operator-based updates that unify Bayesian inference, mirror descent, and proximal learning under time-varying environments. Each update is formulated as a variational optimization problem combining a smooth convex loss f_t with a Bregman divergence D_psi. We prove that the induced operator is averaged, contractive, and exponentially stable in the Bregman geometry. Further, we establish Fejer monotonicity, drift-aware convergence, and continuous-time equivalence via an evolution variational inequality (EVI). Together, these results provide a rigorous analytical foundation for well-posed and stability-guaranteed operator dynamics in nonstationary optimization.