Statistical field theory for neural networks

Abstract: These notes attempt a self-contained introduction into statistical field theory applied to neural networks of rate units and binary spins. The presentation consists of three parts: First, the introduction of fundamental notions of probabilities, moments, cumulants, and their relation by the linked cluster theorem, of which Wick's theorem is the most important special case; followed by the diagrammatic formulation of perturbation theory, reviewed in the statistical setting. Second, dynamics described by stochastic differential equations in the Ito-formulation, treated in the Martin-Siggia-Rose-De Dominicis-Janssen path integral formalism. With concepts from disordered systems, we then study networks with random connectivity and derive their self-consistent dynamic mean-field theory, explaining the statistics of fluctuations and the emergence of different phases with regular and chaotic dynamics. Third, we introduce the effective action, vertex functions, and the loopwise expansion. These tools are illustrated by systematic derivations of self-consistency equations, going beyond the mean-field approximation. These methods are applied to the pairwise maximum entropy (Ising spin) model, including the recently-found diagrammatic derivation of the Thouless-Anderson-Palmer mean field theory.