Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm (1608.04471v3)

Abstract: We propose a general purpose variational inference algorithm that forms a natural counterpart of gradient descent for optimization. Our method iteratively transports a set of particles to match the target distribution, by applying a form of functional gradient descent that minimizes the KL divergence. Empirical studies are performed on various real world models and datasets, on which our method is competitive with existing state-of-the-art methods. The derivation of our method is based on a new theoretical result that connects the derivative of KL divergence under smooth transforms with Stein's identity and a recently proposed kernelized Stein discrepancy, which is of independent interest.

• The paper presents SVGD as a novel deterministic approach to variational inference by evolving particles to approximate the target posterior.
• It leverages Stein’s identity and kernelized Stein discrepancy to perform functional gradient descent in an RKHS for efficient updates.
• Empirical results show that SVGD achieves competitive accuracy with scalable performance on large datasets compared to other Bayesian methods.

Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

The paper "Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm" by Qiang Liu and Dilin Wang presents a novel approach to variational inference. This technique, termed Stein Variational Gradient Descent (SVGD), serves as a deterministic alternative to traditional optimization methods like gradient descent but for the domain of Bayesian inference. The work leverages a new theoretical construct that bridges the Kullback–Leibler (KL) divergence with kernelized Stein discrepancy, facilitating more efficient variational inference.

Core Contribution

The authors introduce SVGD as a general-purpose Bayesian inference algorithm employing particles to represent distributions, performing functional gradient descent on these particles to reduce the KL divergence between the approximating and the target posterior distributions. Key theoretical foundations include the utilization of Stein’s identity and kernelized Stein discrepancy, which enables the derivation of a closed-form solution for the optimal perturbation direction in a Reproducing Kernel Hilbert Space (RKHS). This guarantees the steepest descent within the unit ball of RKHS space, thus enhancing the method's applicability and efficiency.

Theoretical Foundations

The paper delineates the critical theoretical insights fundamental to SVGD:

1. Stein's Identity: This identity plays a pivotal role by associating the expectation of the Stein operator with zero for sufficiently regular functions.
2. Kernelized Stein Discrepancy (KSD): KSD extends Stein's identity to define a discrepancy measure between distributions. The optimization within RKHS significantly reduces computational complexity, bypassing the intractable functional optimizations associated with traditional Stein discrepancies.

Algorithm and Implementation

SVGD operates iteratively:

1. It starts with an initial set of particles.
2. Applies functional gradient descent iteratively to minimize the KL divergence between the particle-based approximation and the posterior distribution.

The gradient updates consist of two terms:

• A first term that directs particles towards high-probability regions of the posterior.
• A second, repulsive term preventing particle collapse, ensuring diverse representation.

This process is encapsulated meticulously in Algorithm 1 provided in the paper, with practical considerations for computational efficiency through parallelization and mini-batch gradient approximations for large datasets.

Empirical Evaluation

The authors validate SVGD across multiple scenarios, showcasing its competitive performance:

1. Toy Example: On a 1D Gaussian mixture, SVGD displays its capability to approximate distributions even with non-overlapping initial and target distributions.
2. Bayesian Logistic Regression: Comparing against NUTS and NPV, SVGD provides comparable accuracy while offering more convenient and scalable implementation.
3. Large Datasets: Testing on extensive datasets like the Covertype dataset, SVGD demonstrates superior performance over methods like SGLD and PMD, particularly in terms of efficiency and handling of large-scale data.

Implications and Future Directions

SVGD's potential lies in its generality and the balance between accuracy and computational tractability. By leveraging deterministic gradient-based updates and the mathematical rigor of kernelized Stein discrepancy, SVGD offers a robust framework for Bayesian inference across diverse models and datasets. The paper highlights practical and theoretical advancements, opening avenues for further exploration in the field of scalable deep learning models and other intricate machine learning applications.

Future developments may explore the theoretical underpinnings of SVGD’s convergence properties, the extension to more complex model spaces, and the refinement of efficient implementation strategies for high-dimensional data analytics. Furthermore, the insight into functional gradients within RKHS provides a fertile ground for novel algorithmic developments in variational inference and related fields.

In conclusion, Qiang Liu and Dilin Wang's work on Stein Variational Gradient Descent marks a significant methodological contribution to the practice of Bayesian inference, combining theoretical novelty with practical applicability.