Threshold-Linear Networks (TLNs)

• Threshold-linear networks (TLNs) are firing-rate models with rectified linear activation that capture key neural dynamics including multistability, oscillations, and chaotic regimes.
• TLNs use piecewise-linear dynamics and graph-based rules to predict fixed-point supports, enabling combinatorial analysis of network connectivity and attractor structures.
• Applications of TLNs span neuroscience modeling, sequence generation, and pattern retrieval, offering insights into memory, attention, and computational capacity in neural circuits.

Threshold-Linear Networks (TLNs) are a fundamental class of firing-rate models in mathematical neuroscience, characterizing recurrent neural systems with piecewise-linear dynamics. Their defining structure—linear interactions coupled with a rectifying nonlinearity—yields a rich spectrum of behaviors, including multistability, oscillations, and chaos. TLNs formalize how network connectivity and external drive orchestrate activity patterns and memory, and in specialized forms, they enable combinatorial predictions of dynamic regimes directly from synaptic graphs. The following sections survey the mathematical structure, dynamical characterization, motif theory, combinatorial fixed-point analysis, and principal applications of TLNs, with technical depth suitable for arXiv-level research readers.

1. Formal Structure and Model Classes

A TLN consists of $n$ neuronal populations with non-negative activity $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$, evolving via

$$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$

where $W \in \mathbb{R}^{n \times n}$ is the weight matrix, $b \in \mathbb{R}^n$ the bias vector, and $[\cdot]_+$ denotes the threshold-linear (rectified linear) activation applied coordinate-wise. All models discussed are autonomous, and often $W_{ii} = 0$ for biological realism.

Key subclasses include:

• Competitive TLNs: $W_{ij} \le 0$ for $i \ne j$; all off-diagonal interactions are inhibitory.
• Combinatorial TLNs (CTLNs): $W_{ij}$ determined by a directed graph $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$0 and two parameters $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$1, encoding weak ($x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$2) and strong ($x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$3) inhibition along/dissimilar to the directed edges of $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$4 (Morrison et al., 2016, Curto et al., 2018).
• Generalized CTLNs (gCTLNs): Allow heterogeneous inhibition parameters $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$5 across nodes (Curto, 6 Oct 2025).
• Excitatory-Inhibitory TLNs (E-I TLNs): Populations partitioned into $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$6 excitatory units and one global inhibitory unit; parameterized by excitatory weights $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$7, self-excitation $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$8, and global inhibition (Curto, 6 Oct 2025, Lienkaemper et al., 6 Jun 2025).

2. Fixed-Point Characterization and Graphical Rules

TLNs are piecewise-affine in each orthant, so fixed points are described combinatorially:

• Support-based solutions: For $x = (x_1,\ldots,x_n)^T \in \mathbb{R}_+^n$9, the restriction to $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$0 yields at most one fixed point with support $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$1, computed as $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$2, $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$3 if $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$4 and off-neuron inequalities are satisfied (Curto et al., 2015, Curto et al., 2018).

CTLN graph rules (parameter-independent in the legal regime):

• Stable fixed points ↔ Maximal cliques: In a symmetric $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$5, each maximal clique yields a stable fixed point; in general, target-free cliques—those with no common target outside—yield precisely the stable fixed points (1909.02947, Curto et al., 2015, Morrison et al., 2016).
• Cycles, parity, and instability: Oriented, sinkless graphs force absence of (stable) equilibria and support oscillatory or chaotic regimes (Morrison et al., 2016, Lienkaemper et al., 6 Jun 2025).
• Domination and reduction: Nodes that are graphically dominated can be removed without altering the set of fixed-point supports. The domination-free core subgraph uniquely dictates all fixed-point supports for the network (Curto, 6 Oct 2025).

Fixed-Point Reduction Table

Operation	Affected Nodes	Effect on $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$6
Remove dominated $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$7	$$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$8 dominated by $$\dot{x}_i = -x_i + \left[ \sum_{j=1}^n W_{ij} x_j + b_i \right]_+,$$9	$W \in \mathbb{R}^{n \times n}$0 (Curto, 6 Oct 2025)
Remove sources	Source nodes	No effect on $W \in \mathbb{R}^{n \times n}$1
Symmetric $W \in \mathbb{R}^{n \times n}$2	Maximal cliques	Stable fixed points

3. Dynamical Regimes and Attractor Structure

The dynamical behavior of a TLN, whether static or oscillatory, is tightly governed by its connectivity structure:

• Multistability: TLNs with symmetric $W \in \mathbb{R}^{n \times n}$3 and appropriate bias admit discrete sets of stable equilibria, interpreted as memory patterns (Curto et al., 2015, 1909.02947).
• Oscillatory and chaotic attractors: CTLNs with oriented, sinkless graphs exhibit no stable equilibria; all trajectories enter a positively invariant region and organize into limit cycles or chaos. Minimal motifs (e.g., 3-cycles) yield stable periodic orbits (Morrison et al., 2016, Lienkaemper et al., 6 Jun 2025).
• Attractors from core motifs: Core fixed-point supports (minimal supports with unique fixed point in the induced subgraph) predict the structure of dynamic attractors—including the order and identity of sequential activity. Every dynamic attractor observed numerically in $W \in \mathbb{R}^{n \times n}$4-node CTLNs is associated with a core motif (Parmelee et al., 2021).
• E-I TLN correspondence: For fast inhibition, E-I TLNs reduce to CTLN-like dynamics, with matched fixed-point supports and similar attractor repertoire. Deviations at slower inhibitory timescales induce bifurcations, including Hopf-type transitions to global E-I oscillations (Lienkaemper et al., 6 Jun 2025, Curto, 6 Oct 2025).

4. Motif Theory, Robustness, and Graphical Calculus

TLN attractor structure is constrained by subgraph (motif) architecture:

• Robust motifs: Motifs whose fixed-point structure is invariant under all weight choices consistent with the motif's edge pattern. Two infinite families (DAG1, DAG2) capture all robust forbidden motifs beyond singleton cases (Curto et al., 2019).
• Flexible motifs: Motifs permitting multiple fixed-point regimes depending on weight values.
• Composite graphs: Disjoint, clique, and cyclic unions of motifs admit direct calculation of global fixed-point supports in terms of the supports of their components (Curto et al., 2018).
• Nerve constructions: Covering a large graph by simpler directional motifs yields "nerve" graphs which control the existence and partitioning of fixed-point supports in the original network (Santander et al., 2021).

5. Advanced Topics: Timescales, Non-minimality, and Computational Implications

• Timescale analysis: Families of TLNs parameterized by synaptic/integration time constants interpolate between projected dynamical systems (PDS, fast limit) and hard-selector systems (HSS, slow limit). Under the diagonal Lyapunov stability (LDS) condition, global stability is preserved across the spectrum of timescales and both endpoint systems share the same equilibrium (Retnaraj et al., 17 Apr 2026).
• Non-minimal stable fixed points: Contrary to early conjectures, competitive TLNs can support stable fixed points whose supports strictly contain those of other stable fixed points. Explicit counterexamples exist for $W \in \mathbb{R}^{n \times n}$5, with methods (neuron-splitting, Kronecker products) to construct arbitrarily long chains of nested stable supports (Geneson, 26 Oct 2025).
• Computational roles: TLNs encode binary and graded patterns, support rapid associative memory via local Hebbian updates, and realize high storage capacity—often surpassing classical Hopfield network limits. Pattern retrieval is naturally sparse and noise-tolerant (Schönsberg et al., 2020). Decoding and error correction in place-cell codes have been demonstrated via clique-encoding symmetric TLNs (Curto et al., 2015).

6. Applications and Modeling Relevance

• Neuroscience modeling: TLNs, both in their competitive and excitatory-inhibitory forms, model phenomena including goal-driven selective attention (via stabilizability and inhibition-driven gating), declarative memory (multistability and permitted sets), and epilepsy (oscillation and bifurcation structure) (McCreesh et al., 2024).
• Sequence generation and spatial computation: Cycle and chain CTLNs express stable sequential attractors, modeling central pattern generators and hippocampal replay (Parmelee et al., 2021).
• Combinatorial geometry and neural codes: The geometric and combinatorial properties of TLNs and their encoding rules establish deep connections with oriented matroids, convex codes, and sparse representation theory (Curto et al., 2020).

7. Open Problems and Future Directions

• Rigorous extension of motif- and nerve-based constructions to general asymmetric/heterogeneous TLNs.
• Role of plasticity and time-dependent $W \in \mathbb{R}^{n \times n}$6 in pattern evolution, attractor creation/destruction, and memory update mechanisms (McCreesh et al., 2024).
• Systematic classification of high-dimensional chaotic or quasiperiodic attractors from core motif structure, and quantifying the robustness of core-motif predictions under perturbations and noise (Parmelee et al., 2021).
• Algorithmic and representational complexity of TLNs as function-approximators, including shortcut linear-threshold architectures and their universality for piecewise-constant/piecewise-linear functions (Khalife et al., 2021).
• Quantitative analysis of the region-of-attraction for memory retrieval and online learning protocols in TLN-based associative memory systems (Qin et al., 30 Mar 2026).
• Deeper analytical treatment of bifurcation scenarios, multi-timescale behavior, and the impact of network topology on global stability beyond the diagonal Lyapunov regime (Retnaraj et al., 17 Apr 2026).

Threshold-linear networks constitute a mathematically rigorous, physiologically plausible, and algorithmically versatile framework for capturing nonlinear population-rate dynamics, combinatorial memory, and sequence generation in recurrent neural circuits. Their graph-theoretic rules and motif-based reductions offer unique insight into the correspondence between synaptic architecture and emergent computation (Curto, 6 Oct 2025, Lienkaemper et al., 6 Jun 2025, 1909.02947, Parmelee et al., 2021).