Fixed points of competitive threshold-linear networks (1804.00794v2)

Abstract: Threshold-linear networks (TLNs) are models of neural networks that consist of simple, perceptron-like neurons and exhibit nonlinear dynamics that are determined by the network's connectivity. The fixed points of a TLN, including both stable and unstable equilibria, play a critical role in shaping its emergent dynamics. In this work, we provide two novel characterizations for the set of fixed points of a competitive TLN: the first is in terms of a simple sign condition, while the second relies on the concept of domination. We apply these results to a special family of TLNs, called combinatorial threshold-linear networks (CTLNs), whose connectivity matrices are defined from directed graphs. This leads us to prove a series of graph rules that enable one to determine fixed points of a CTLN by analyzing the underlying graph. Additionally, we study larger networks composed of smaller "building block" subnetworks, and prove several theorems relating the fixed points of the full network to those of its components. Our results provide the foundation for a kind of "graphical calculus" to infer features of the dynamics from a network's connectivity.