Dynamical Mechanism of Sampling-based Stochastic Inference under Probabilistic Population Codes

Abstract: Animals are known to make efficient probabilistic inferences based on uncertain and noisy information from the outside world. Although it is known that generic neural networks can perform near-optimal point estimation by probabilistic population codes which has been proposed as a neural basis for encoding of probability distribution, the mechanisms of sampling-based inference has not been clarified. In this study, we trained two types of artificial neural networks: feedforward neural networks (FFNNs) and recurrent neural networks (RNNs) to perform sampling based probabilistic inference. Then, we analyzed and compared the mechanisms of sampling in the RNN with those in the FFNN. As a result, it was found that sampling in RNN is performed by a mechanism that efficiently utilizes the properties of dynamical systems, unlike FFNN. It was also found that sampling in RNNs acts as an inductive bias, enabling more accurate estimation than in MAP estimation. These results will provide important implications for the discussion of the relationship between dynamical systems and information processing in neural networks.