Three-Factor Learning Rules

• Three-factor learning rules are synaptic plasticity mechanisms that modify weights based on pre-synaptic activity, post-synaptic activity, and a modulatory signal such as reward, error, or novelty.
• They extend classical Hebbian learning and spike-timing-dependent plasticity by introducing eligibility traces that bridge rapid neuronal responses with slower behavioral feedback.
• They are implemented in neuromorphic hardware and reinforcement learning models, offering efficient online adaptation and enhanced temporal credit assignment.

Three-factor learning rules are a class of synaptic plasticity mechanisms in neural systems—both biological and artificial—characterized by the requirement that synaptic modifications depend on three distinct signals: pre-synaptic activity, post-synaptic activity, and a third modulatory factor, often encoding a global or contextual variable such as reward, error, surprise, or behavioral relevance. This structure generalizes classical two-factor Hebbian rules and spike-timing-dependent plasticity (STDP), enabling improved temporal credit assignment, contextual gating, and biological realism in both theoretical and applied neural models. Three-factor rules now underlie much of contemporary work in biologically plausible reinforcement learning, adaptive spiking neural networks, neuromorphic hardware, and computational neuroscience.

1. Core Principles and Formulation

The canonical form of a three-factor learning rule expresses the change in a synaptic weight $w_{ij}$ as:

$$\Delta w_{ij}(t) = F(\text{pre}_i(t),\, \text{post}_j(t),\, M(t))$$

where:

• $\text{pre}_i(t)$: pre-synaptic activity of neuron $i$ (e.g., spike count, firing rate)
• $\text{post}_j(t)$: post-synaptic activity of neuron $j$ (e.g., membrane potential, spike output)
• $M(t)$: modulatory signal acting as a third factor (e.g., neuromodulator concentration, reward or error signal)

A generic instantiation uses a local eligibility trace $e_{ij}(t)$ to temporally accumulate pre/post correlations:

$$\frac{de_{ij}}{dt} = \eta\, \text{pre}_i(t)\, f(\text{post}_j(t)) - \frac{e_{ij}}{\tau_e}$$

with synaptic update at times when the third factor is present:

$\Delta w_{ij} = e_{ij}(t)\, M(t)$

The third factor $M(t)$ may represent phasic dopamine, reward-prediction error, error signals in supervised tasks, or novelty/surprise (Mazurek et al., 6 Apr 2025, Gerstner et al., 2018).

2. Biological Substrates and Experimental Evidence

Three-factor rules provide a unifying formalism for observed phenomena in systems neuroscience:

• Pre-synaptic factor: Glutamate release, vesicle fusion events, or neurotransmitter binding.
• Post-synaptic factor: Voltage-gated calcium influx, dendritic depolarization, or back-propagating action potentials.
• Third (modulatory) factor: Phasic bursts of neuromodulators (dopamine, norepinephrine, serotonin, acetylcholine) or global error/novelty signals.

Experimental work demonstrates that induction of LTP or LTD at a synapse often requires coincidence of pre-and post-synaptic activation (establishing an eligibility trace) and a temporally-delayed third factor (e.g., dopamine pulse). Measured eligibility time-windows span behavioral time scales, typically $\tau_e = 1$–$10$ s in striatum and cortex, up to $\sim60$ s in the hippocampal consolidation regime. These findings substantiate that three-factor mechanisms bridge the gap between rapid neuronal activity and slower behavioral feedback (Gerstner et al., 2018).

3. Computational Realizations and Algorithmic Structure

Three-factor rules support online learning, temporal credit assignment, and adaptation:

• Eligibility traces: Local per-synapse memories that integrate pre/post coincidence; implement temporal bridging between synaptic activity and subsequent reward/punishment.
• Modulatory factors: Scalar or vector signals (reward in RL, error in supervised learning, surprise for novelty detection, etc.) that globally gate synaptic plasticity across populations.
• Dual- or multi-timescale traces: Many implementations (e.g., dual traces in (Nallani et al., 17 Sep 2025)) combine fast and slow eligibility traces for improved stability–plasticity trade-off:

$$e_{ij}^{\text{comb}}(t) = \alpha_{\text{mix}}\, e_{ij}^{\text{fast}}(t) + (1 - \alpha_{\text{mix}})\, e_{ij}^{\text{slow}}(t)$$

allowing rapid adaptation while preserving consolidated memory.

Algorithmic instantiations span:

• Simple reward-modulated STDP: $\Delta w_{ij} = \eta e_{ij} R(t)$ at moments of reward.
• Reinforcement learning with delayed scalar feedback (Smith, 2024).
• Feedback-modulated, TD-error-gated rules for discrete action-spaces (Chung et al., 2020).
• Surrogate-gradient SNN training eliminating backpropagation-through-time, with all weight updates local and online (Nallani et al., 17 Sep 2025).
• Meta-learned polynomial plasticity kernels for complex credit assignment (Maoutsa, 10 Dec 2025).

4. Theoretical Foundations and Functional Roles

Three-factor rules emerge naturally from both computational and statistical learning objectives:

• Maximization of mutual-information subject to energy constraints produces three-factor updates combining local activity and a global variable representing information surprise or metabolic cost (Grytskyy et al., 2021).
• Information-bottleneck and kernelized learning objectives in deep networks yield updates with Hebbian (pre/post) factors and an error-modulatory factor based on pairwise output similarity, with local divisive normalization for biological plausibility (Pogodin et al., 2020).
• In recurrent networks, eligibility traces and modulatory factors enable structured credit assignment without non-local backpropagation (Maoutsa, 10 Dec 2025).
• In reinforcement learning, these rules instantiate the mathematics of policy-gradient and TD-learning, but implemented with local synaptic operations and global neuromodulators, supporting biologically plausible learning from sparse, delayed rewards (Gerstner et al., 2018, Mazurek et al., 6 Apr 2025).

5. Practical Implementations and Hardware Realization

Three-factor learning rules are highly amenable to neuromorphic and event-driven hardware due to their local, asynchronous, and modular nature:

• Event-driven update algorithms allow voltage- or eligibility-based three-factor rules (e.g., Clopath, Urbanczik-Senn) to operate efficiently at scale by exploiting sparse spike-event histories instead of continuous time-driven sweeps (Stapmanns et al., 2020).
• Crossbar/memristor VLSI arrays can locally realize three-factor updates, sharing inference and learning datapaths to suppress mismatch and achieve update energy in the picojoule range. Error-triggered mechanisms can reduce synaptic writes by 20–100$\times$, with negligible accuracy loss in SNNs trained on real-world benchmarks (Payvand et al., 2019).
• Local, online implementations of three-factor rules have been successfully deployed for closed-loop neural decoding in BCI systems, yielding up to 35% memory savings over backpropagation-through-time, faster convergence, and robust adaptation to signal drift and re-mapping (Nallani et al., 17 Sep 2025).

6. Applications and Empirical Performance

Three-factor rules underpin advanced capabilities in both machine learning and robotics:

• Adaptive motor control in spiking quadruped robots using meta-optimized three-factor plasticity, matching rapid motor adaptation algorithms and demonstrating resilience to environmental and body perturbations (Schmidgall et al., 2023).
• Online reinforcement learning in biological and artificial agents, solving cart-pole, LunarLander, delayed bandit, and context-dependent integration tasks with near-optimal sample efficiency and stability (Smith, 2024, Maoutsa, 10 Dec 2025, Chung et al., 2020).
• Real-time BCI decoding and continuous, on-the-fly neural adaptation, unique to three-factor-rule-trained models compared to fixed-weight or BPTT-based solutions (Nallani et al., 17 Sep 2025).

Key empirical benchmarks:

Task/Benchmark	Accuracy/Return (3-Factor)	Competing Method	Relative Memory/Convergence
MC Maze BCI Decoding	$R\geq0.81$	BPTT-SNN, LSTM	28–35% lower memory, faster
Quadruped Motor Adaptation	Return $\approx5.7$–$6.9$	RMA, STDP, fixed	Matches/Exceeds, online-only
Cart-Pole RL	6,100–6,200 steps/trial avg	Static policy (6,380)	Near-optimal, fast learning
SNN Gesture/N-MNIST [1910]	2–4% error, $>20\times$ fewer writes	BP, STDP	Efficient neuromorphic

7. Current Directions and Open Questions

Despite rapid advances, three-factor learning rules face several outstanding challenges:

• Global error propagation: Purely local rules may inadequately propagate errors in deep or recurrent architectures; combining three-factor mechanisms with cell-type-specific broadcast or global feedback alignment is under investigation (Mazurek et al., 6 Apr 2025).
• Parameter tuning and stability: Optimal settings for eligibility windows, learning rates, and mixing parameters are context-dependent and subject to ongoing research, including meta-optimization (Maoutsa, 10 Dec 2025).
• Biophysical diversity: Real circuits express a rich repertoire of neuromodulators, synaptic receptor types, and plasticity mechanisms, undersampled in current artificial models.
• Hardware scalability: Memory and compute overhead of maintaining eligibility traces per synapse can limit large-scale deployment; event-driven and compressed-history algorithms mitigate, but do not eliminate, resource demands (Stapmanns et al., 2020).
• Integration with higher cognitive functions: Extensions to hierarchical, multi-factor, or attention-gated models are required to address behavioral complexity and credit assignment in naturalistic settings.

Future research is expected to focus on cross-disciplinary integration of three-factor rules with meta-learning, neuromorphic device co-design, standardization of benchmarks for event-driven learning, and biological experiments directly quantifying eligibility traces and neuromodulatory signals across diverse brain areas (Mazurek et al., 6 Apr 2025, Gerstner et al., 2018).

---

Three-factor learning rules provide a principled and empirically substantiated framework connecting synaptic plasticity with behavioral adaptation, machine learning, and neuromorphic engineering. Ongoing developments continue to expand their algorithmic scope, neural fidelity, and real-world impact.