Equivalence of restricted Boltzmann machines and tensor network states (1701.04831v2)

Abstract: The restricted Boltzmann machine (RBM) is one of the fundamental building blocks of deep learning. RBM finds wide applications in dimensional reduction, feature extraction, and recommender systems via modeling the probability distributions of a variety of input data including natural images, speech signals, and customer ratings, etc. We build a bridge between RBM and tensor network states (TNS) widely used in quantum many-body physics research. We devise efficient algorithms to translate an RBM into the commonly used TNS. Conversely, we give sufficient and necessary conditions to determine whether a TNS can be transformed into an RBM of given architectures. Revealing these general and constructive connections can cross-fertilize both deep learning and quantum many-body physics. Notably, by exploiting the entanglement entropy bound of TNS, we can rigorously quantify the expressive power of RBM on complex data sets. Insights into TNS and its entanglement capacity can guide the design of more powerful deep learning architectures. On the other hand, RBM can represent quantum many-body states with fewer parameters compared to TNS, which may allow more efficient classical simulations.

• The paper establishes an equivalence between RBMs and tensor network states, enabling cross-disciplinary translation of methodologies.
• It details efficient algorithms for mapping RBMs to Matrix Product States, enhancing model optimization by reducing redundancy.
• The study leverages quantum entanglement to quantify RBMs' expressive power in capturing complex correlations in data and quantum states.

Analyzing the Equivalence of Restricted Boltzmann Machines and Tensor Network States

The paper "Equivalence of Restricted Boltzmann Machines and Tensor Network States" presents an exploration into the transformation of Restricted Boltzmann Machines (RBMs) into Tensor Network States (TNS) and vice versa. Its implications are multifaceted, affecting both deep learning practices and quantum many-body physics. This equivalence opens potential pathways for cross-disciplinary advancements, utilizing mathematical techniques and theoretical concepts across fields.

At the core, the RBM and TNS are explored as function approximators that provide different parametrizations for multivariable functions. The paper outlines algorithms to efficiently translate RBMs into the widely-studied Matrix Product States (MPS) and other TNS formats. Through these mappings, it is suggested that insights into the entanglement entropy — a key concept in quantum mechanics — can be leveraged to quantify the expressive power of RBMs. The entanglement entropy in the MPS representation serves as an upper bound on the networks' ability to capture complex correlations in datasets or quantum states. Specifically, highly entangled states that do not satisfy the entanglement area law can potentially be more compactly captured by dense RBMs than by corresponding MPS or PEPS representations.

Conversely, the research elaborates on the necessary and sufficient conditions for translating a TNS into an RBM with a given architecture. This transformation relies on ensuring that a set of linear equations provided by the TNS has a unique solution, and it involves a tensor rank decomposition for the efficient representation of these systems.

The practical implications are significant. The equivalence and transformations established enable the theoretical and practical optimization of RBMs using tensor network methods. By mapping an RBM to an MPS, for instance, one can discover and remove redundant degrees of freedom. This optimizes RBM computational models, potentially leading to more efficient machine learning algorithms and simulations.

Furthermore, the paper speculates on broader impacts within artificial intelligence research. Through the lens of quantum entanglement, RBMs' ability to effectively model various probability distributions, including those found in natural datasets, can be quantitatively assessed. This insight allows for more informed design and application of neural networks in learning tasks, potentially leading to more efficient machine learning architectures.

Specific examples, such as the application of the mapping methods to the ground states of the toric code model and the Ising model, bolster the case for practical usefulness of these theoretical constructs. These examples reveal the diverse utility and application scope of intertwining concepts from quantum physics with machine learning models.

Prospectively, the equivalence framework laid out by this paper encourages exploration into more complex structures and deeper Boltzmann machines, drawing parallels with multi-scale or hierarchical tensor structures in quantum theories such as MERA. As methodologies evolve, they carry the potential to drive advances in computational efficiency, model expressibility, and understanding in quantum states and deep learning models alike.

The endeavor of this paper underscores a symbolic bridging of disparate domains: by aligning the foundational structures of RBMs with TNS, new strides in optimizing and understanding these systems seem within reach, promising benefits in computational efficiencies and more nuanced quantum-physical insights applied in AI systems.