Ratio Divergence Learning Using Target Energy in Restricted Boltzmann Machines: Beyond Kullback--Leibler Divergence Learning (2409.07679v1)

Abstract: We propose ratio divergence (RD) learning for discrete energy-based models, a method that utilizes both training data and a tractable target energy function. We apply RD learning to restricted Boltzmann machines (RBMs), which are a minimal model that satisfies the universal approximation theorem for discrete distributions. RD learning combines the strength of both forward and reverse Kullback-Leibler divergence (KLD) learning, effectively addressing the "notorious" issues of underfitting with the forward KLD and mode-collapse with the reverse KLD. Since the summation of forward and reverse KLD seems to be sufficient to combine the strength of both approaches, we include this learning method as a direct baseline in numerical experiments to evaluate its effectiveness. Numerical experiments demonstrate that RD learning significantly outperforms other learning methods in terms of energy function fitting, mode-covering, and learning stability across various discrete energy-based models. Moreover, the performance gaps between RD learning and the other learning methods become more pronounced as the dimensions of target models increase.

• The paper introduces a novel Ratio Divergence (RD) learning method that symmetrically integrates forward and reverse KLD to overcome their individual limitations.
• The paper demonstrates that RD learning boosts acceptance probabilities in Metropolis–Hastings algorithms, enhancing model stability and sampling quality.
• The paper’s experiments on models like the 2D Ising and SK models reveal superior regression accuracy and effective mode covering in high-dimensional settings.

Ratio Divergence Learning Using Target Energy in Restricted Boltzmann Machines: Beyond Kullback--Leibler Divergence Learning

In this paper, the authors introduce a novel learning method, Ratio Divergence (RD) learning, for discrete energy-based models, specifically applied to Restricted Boltzmann Machines (RBMs). RD learning addresses the limitations of traditional forward and reverse Kullback--Leibler divergence (KLD) learning methods by integrating their strengths while overcoming their respective drawbacks.

Introduction and Motivation

The paper is motivated by the need to efficiently approximate high-dimensional discrete distributions where the energy function is tractable, but the normalization factor is not. This scenario arises in various fields, such as condensed matter physics, Bayesian inference, and optimization. Traditional generative modeling techniques based on KLD have limitations: forward KLD learning often underfits the target distribution, while reverse KLD learning is prone to mode collapse.

Ratio Divergence Learning

RD learning is proposed as a method that symmetrically combines the strengths of both forward and reverse KLD learning. The core idea is to utilize a symmetrized divergence that naturally incorporates the product of the forward and reverse KLDs, eliminating the intractable normalization factor. This approach is particularly advantageous as it integrates information from both the training data and a tractable target energy function.

Mathematically, the ratio divergence is defined as:

$$L(\hat{P}, P; {\theta}) = \sum_{{x}', {x} \in {X}} \hat{P}({x}') P({x}; {\theta}) \left( \log \frac{\hat{P}({x}') P({x};{\theta})}{P({x}';{\theta}) \hat{P}({x})} \right)^2.$$

The RD method leverages this symmetrized divergence to train RBMs, ensuring that the model learns to approximate the target distribution more effectively.

Theoretical Properties and Implications

The authors provide a theoretical analysis demonstrating that RD is a valid divergence measure. They also prove that minimizing RD leads to an increase in the acceptance probability in Metropolis-Hastings (MH) algorithms, which is a significant benefit for downstream tasks involving MCMC simulations.

Furthermore, the theoretical framework includes:

• Proofs of RD's properties as a symmetric divergence.
• An inequality that guarantees an increased acceptance probability in MH algorithms.

These properties imply that RD learning not only balances the drawbacks of forward and reverse KLD but also enhances the stability and efficiency of sample generation.

Experimental Results

The paper includes comprehensive numerical experiments on various discrete energy-based models, such as the 2D Ising model, the Sherrington--Kirkpatrick (SK) model, the Maximum Independent Set (MIS) problem, and the Maximum Cut Problem (MCP). The results highlight the superior performance of RD learning compared to traditional KLD-based methods.

Key Findings:

• RD learning significantly improves the regression performance of energy functions.
• It achieves higher stability and effectiveness in mode covering.
• RD learning scales better as the dimensionality of the target models increases.

Performance Metrics:

• The Wasserstein metric is used to compare the empirical distributions of the generated samples with the training dataset.
• The mean squared error of energy differences (Eq.~(6)) evaluates regression accuracy.

Numerical results indicate that RD learning provides a balanced approach to approximating complex distributions, maintaining both high regression accuracy and effective mode covering.

Future Directions

The findings suggest several potential future developments:

• Extension of RD learning to other types of generative models beyond RBMs.
• Exploration of RD learning in more complex and higher-dimensional distributions.
• Integration of RD learning with advanced optimization and sampling techniques to further enhance performance in practical applications.

Conclusion

RD learning offers a compelling solution to the inherent challenges of approximating high-dimensional discrete distributions. By effectively combining the strengths of forward and reverse KLD learning, and ensuring robust performance across various metrics, RD learning stands as a promising tool in the field of discrete energy-based modeling. The theoretical guarantees and practical results underpin its potential for advancing large-scale simulations and offering deeper insights into high-dimensional energy landscapes.