Adaptive Thouless--Anderson--Palmer equation for higher-order Markov random fields

Abstract: The adaptive Thouless--Anderson--Palmer (TAP) mean-field approximation is one of the advanced mean-field approaches, and it is known as a powerful accurate method for Markov random fields (MRFs) with quadratic interactions (pairwise MRFs). In this study, an extension of the adaptive TAP approximation for MRFs with many-body interactions (higher-order MRFs) is developed. We show that the adaptive TAP equation for pairwise MRFs is derived by naive mean-field approximation with diagonal consistency. Based on the equivalence of the approximate equation obtained from the naive mean-field approximation with diagonal consistency and the adaptive TAP equation in pairwise MRFs, we formulate approximate equations for higher-order Boltzmann machines, which is one of simplest higher-order MRFs, via the naive mean-field approximation with diagonal consistency.