Structural Plasticity Module: Dynamic Neural Networks

• SPM is an architectural and algorithmic component that modulates network connectivity, neuron count, and synaptic structure based on local activity signals.
• SPMs employ dynamic rewiring, growth, pruning, and node migration driven by local error, prediction, and homeostatic criteria to optimize learning.
• Empirical results show that SPMs improve continual learning, memory capacity, and efficiency across biological, artificial, and neuromorphic systems.

A Structural Plasticity Module (SPM) is an architectural and algorithmic component that enables dynamic adaptation of network connectivity, neuron count, or synaptic structure in neural systems—biological, artificial, or neuromorphic. Unlike classic synaptic plasticity (weight adaptation under fixed topology), SPMs enact rewiring, growth, pruning, or migration of nodes and edges, typically driven by local activity statistics, prediction error, or network-wide homeostatic criteria. SPMs are found in various domains, including lifelong learning in artificial neural networks, recurrent reservoir computing, analog/digital neuromorphic substrates, and sparse spiking models. They have been shown to improve continual learning, memory capacity, efficiency, and robustness by mimicking principles of biological structural plasticity.

1. Mathematical Foundations and Variants

Structural Plasticity Modules operate via mathematical rules that modify network topology, neuron allocation, or connection sparsity based on signals such as activity or prediction error. The dominant paradigms include:

• Hebbian-Gated Parameterization: SPMs augment each synapse with an "importance" parameter $\gamma_k$, updated by Hebbian co-activation and Oja-style normalization, as in attention-based SPMs (Kolouri et al., 2019).
• Local Error-Driven Morphogenesis: SPMs use rolling windows of weight gradients or activation statistics to trigger node/edge additions or deletions. E.g., in adaptive policy networks, new relay neurons are introduced when weight updates on edges exhibit high variance and low mean—reflecting local representational instability (Jia et al., 14 Dec 2025).
• Fitness-Based Rewiring: Spiking networks employ short-term fitness traces, inspired by STDP, to select synapses for removal and substitution; swaps are performed to maintain constant fan-in and sparsity, updating connectivity matrices in-place (Roy et al., 2016, Billaudelle et al., 2019).
• Migration on Spatial Grids: Some models deploy SPMs to move processing units ("cells") within a spatial grid based on local prediction error, thus coupling receptive field optimization to homeostatic drives (Hill, 4 Nov 2025).
• Sparsity Gating and Neurogenesis: SPMs may operate by probabilistic masking (using LFSRs and control thresholds) and controlled neuron addition/removal in digital neuromorphic architectures, triggered by global performance metrics (Zyarah et al., 1 Mar 2025).

2. Algorithmic Implementations and Pseudocode

Distinct classes of SPMs deploy schedule-based or event-driven algorithms suited for their computational substrate.

• Online Hebbian SPM (Attention-Gated) (Kolouri et al., 2019):
  • Compute layer-wise attention signals by contrastive Excitation Backpropagation (c-EB).
  • Update per-synapse importance $\gamma_{ji}^l$ with:

  $$\gamma_{ji}^l \leftarrow \gamma_{ji}^l + \epsilon\left[P_c(f_j^{l-1}) P_c(f_i^l) - P_c(f_i^l)^2 \gamma_{ji}^l\right]$$ - Modify loss by $\lambda \sum_k \gamma_k (\theta_k - \theta_k^*)^2$; apply SGD as usual.

• Edge-Instability Driven Network Growth (Jia et al., 14 Dec 2025):

  • Maintain activity and $\Delta w$ buffers per node/edge; compute rolling mean/variance.
  • If $|\mu^{\Delta}_k| < \frac{1}{2}\sigma^{\Delta}_k$ and $(\sigma^{\Delta}_k)^2 > \lambda_{\rm edge}$, insert a new relay neuron between edge $k$'s endpoints.
  • Periodically prune edges with low weight magnitude and update history; remove orphaned nodes.
• Reservoir Rewiring with Fitness Traces (Roy et al., 2016):
  • On presynaptic spike: decrement fitness by postsynaptic trace; on postsynaptic spike: increment by presynaptic trace.
  • After input pattern, swap worst fitness pre-connection for best random candidate, preserving binary connection count.
• Grid Cell Migration (Hill, 4 Nov 2025):
  • Compute per-cell "desire" $D_i = |\bar V_i - E_i|$.
  • For $D_i \geq \theta_D$, select move direction (exploration or mean bias), attempt collision-free spatial migration.
  • After macro-episode, reset short-term activation accumulators.
• Synaptic Mask and Neurogenesis Control (Zyarah et al., 1 Mar 2025):
  • Apply runtime-generated binary mask $M(t)$ to control active reservoir-readout connections.
  • Trigger neuron addition or increase sparsity $s(t)$ when validation error exceeds predefined bounds, via CCU firmware.

3. Integration with Network Architectures and Substrates

SPMs have demonstrated compatibility with varied substrates and paradigms:

Architecture/Model	SPM Role	Reference
Multilayer Perceptron (MLP), CNN	Hebbian-gated importance, continual learning	(Kolouri et al., 2019)
Liquid State Machine, Reservoir ESN	Recurrent synapse rewiring, neurogenesis	(Roy et al., 2016, Zyarah et al., 1 Mar 2025)
Grid-based Predictive Networks (SAPIN)	Spatial migration driven by prediction error	(Hill, 4 Nov 2025)
Neuromorphic Hardware (BrainScaleS-2)	Local address rewiring, STDP-weighted, fixed fan-in	(Billaudelle et al., 2019)
GPU-accelerated Sparse SNNs (GeNN)	Parallel structural updates, DEEP R, e-prop/STDP	(Knight et al., 22 Oct 2025)

SPMs may operate in purely software environments, FPGA/ASIC silicon, or mixed analog-digital neuromorphic systems. In digital chips, SPMs often interface with or are orchestrated by a central control unit (CCU) that manages random number sources, counters, and performance metric triggers (Zyarah et al., 1 Mar 2025). Mixed-signal neuromorphic cores may utilize embedded microprocessors for in-place label rewiring while leveraging local hardware correlation circuits for eligibility computation (Billaudelle et al., 2019).

4. Interaction with Synaptic Plasticity and Local/Global Learning

SPMs can interleave or superimpose with conventional plasticity mechanisms:

• In supervised ANNs, synaptic weights are trained via backpropagation or local learning (e-prop/STDP); importance parameters (in attention-based SPMs) or network structure (in growth/pruning SPMs) are adapted in parallel, each using distinct local or global signals (Kolouri et al., 2019, Jia et al., 14 Dec 2025, Knight et al., 22 Oct 2025).
• In reservoir and spiking systems, SPMs maintain constant connection constraints, controlling only binary connectivity, while weights are updated under homeostatic or STDP-style rules (Roy et al., 2016, Billaudelle et al., 2019).
• Movement-based SPMs in morphogenetic networks adjust node placement to optimize long-term functional statistics (e.g., minimizing local prediction error), with synaptic learning and migration both being locally homeostatic and tightly coupled (Hill, 4 Nov 2025).
• SPMs may gate which synaptic weights undergo adaptation by applying real-time masks, ensuring that only connections retained by the SPM participate in gradient updates or hardware resource allocation (Zyarah et al., 1 Mar 2025).

5. Empirical Outcomes and Performance

SPMs have been shown to produce measurable benefits across a range of benchmarks:

• Continual/Lifelong Learning: Attention-based structural plasticity sustains 92–95% accuracy across five sequential Permuted MNIST tasks without catastrophic forgetting (vanilla MLP: 20–30%), in line with or better than EWC/SI baselines (Kolouri et al., 2019).
• Reservoir Quality: SPM-driven LSMs yield 1.36× increased inter-class separation and 2.05× broader linear separation rank versus random reservoirs, as well as extending fading memory by ≈90 ms (Roy et al., 2016).
• Neuromorphic Hardware: BrainScaleS-2’s SPM achieves 92% classification on iris through efficient in-place address rewiring, at 500× real-time speedup. Custom ESN ASICs with SPM reach 95.95% (HAR) at <50 mW (Zyarah et al., 1 Mar 2025, Billaudelle et al., 2019).
• Adaptive Topology: In SMGrNN, SPM-driven policy networks satisfy reward, stability, and network-size requirements, exhibiting lower variance, auto-adaptive hidden counts scaling with task complexity, and explicit ablation-proven necessity for both growth and pruning (Jia et al., 14 Dec 2025).
• Efficiency and Scalability: GPU-accelerated SPM frameworks demonstrate >10× faster simulation/runtime for sparse networks with negligible accuracy loss vs dense models, with scalable topographic-mapping at up to $10^5$ neurons (Knight et al., 22 Oct 2025).

6. Biological and Theoretical Motivations

SPMs are inspired by mechanisms ubiquitous in biological nervous systems: axon/dendrite sprouting, synaptic pruning, and neuronal migration drive self-organization, memory allocation, and homeostatic balance in both development and adult plasticity. Notably:

• SPMs implement purely local, resource-efficient rules analogous to cortical structural adaptation: e.g., swapping of synaptic addresses (Billaudelle et al., 2019), pruning of low-activity connections, or activity-driven neurogenesis (Zyarah et al., 1 Mar 2025, Jia et al., 14 Dec 2025).
• Variational and homeostatic frameworks, such as grid migration under local prediction error, directly aim for minimal "free energy" per cell (Hill, 4 Nov 2025).
• Modular separation of structural and synaptic plasticity, allowing future integration of Hebbian or spike-timing–dependent plasticity alongside SPM, reflects biological diversity in neural learning rules (Jia et al., 14 Dec 2025).

7. Limitations, Stability, and Open Directions

SPMs, though powerful, expose system-level tradeoffs:

• Stability: Continued structural adaptation can destabilize learned configurations—global "locking" of weights and connectivity often stabilizes performance (Hill, 4 Nov 2025).
• Capacity Control: Without effective pruning, adaptive SPMs may lead to uncontrolled parameter growth and degraded efficiency (Jia et al., 14 Dec 2025).
• Substrate Constraints: Hardware-oriented SPMs must accommodate fixed fan-in/out, memory, and silicon area; address label trickery and in-place SRAM updates are key to scalable implementation (Billaudelle et al., 2019, Zyarah et al., 1 Mar 2025).
• Sparse/Parallel Computation: Practical SPMs leverage sparsity not only for biological plausibility but also for training and inference acceleration, as with ragged-matrix and bitfield techniques on GPUs (Knight et al., 22 Oct 2025).
• Integration with Other Plasticity Rules: Future work spans combining SPMs with synaptic and non-synaptic local learning, leveraging biological plausibility and computational efficiency in hybrid architectures (Jia et al., 14 Dec 2025, Hill, 4 Nov 2025).

In summary, the Structural Plasticity Module encapsulates a family of architectural, mathematical, and algorithmic tools that endow neural systems—real or artificial—with the capacity for dynamic and local adaptation of structure, supporting efficient learning, robustness, and resource optimization across computational and physical substrates.