Expansion Layer Plasticity in Biology & AI

Updated 14 November 2025

Expansion layer plasticity is the dynamic capability of high-dimensional layers to recode input patterns through structural and synaptic changes, enhancing performance.
It encompasses mechanisms from biology, like associative and homeostatic plasticity, to algorithmic strategies such as neuron growth and pruning in artificial systems.
Empirical studies show that controlled expansion improves pattern separation, stability, and task-specific specialization across neuroscience, AI, and material science.

Expansion layer plasticity refers to the property, observed in both biological and artificial systems, whereby the high-dimensional expansion layers—characterized by their dense, combinatorial recoding of input patterns—retain the capacity to change their functional connectivity or recruitment of units throughout learning. In contrast to traditional frameworks that treat expansion layers as immutable feature coders, recent empirical and theoretical advances demonstrate that plasticity within these layers is essential for robust pattern separation, adaptability, continual learning, and task-specific specialization across domains such as neuroscience, deep reinforcement learning, continual supervised learning, and even layered materials in condensed matter physics (Rudelt et al., 13 Nov 2025, Liu et al., 10 Oct 2024, Zhu et al., 2023, Kanaan et al., 2021, Li et al., 2019, Cantos-Prieto et al., 2020). Expansion layer plasticity encompasses both synaptic modifications and topological growth/pruning, and governs the dynamic balance between flexibility (plasticity) and retention (stability) of stored representations.

1. Biological Substrates of Expansion Layer Plasticity

Expansion layer plasticity is a defining property of cerebellum-like circuits—most notably the granule cell layer in the vertebrate cerebellum and the Kenyon cell (KC) layer in the insect mushroom body (Rudelt et al., 13 Nov 2025). In these architectures:

Principal cells (GrC/KC) receive input via “clawed” dendrites contacting single afferent boutons (mossy fibers for cerebellum, projection neurons for MB), producing extreme pattern separation.
Inhibitory interneurons (Golgi/APL or PCT cells) provide both feedforward and feedback inhibition, shaping both the gain and sparseness of expansion layer responses.
Plasticity mechanisms include:
- Associative (reinforcement-gated) plasticity: Dopaminergic/acetycholinergic modulation leads to eligibility-trace-based potentiation (LTP) and depression (LTD) at expansion layer synapses contingent on behavioral outcomes (reward/punishment).
- Non-associative plasticity: Activity-dependent habituation and homeostatic regulation through bidirectional, Ca^2+-dependent rules (e.g., BCM-like) and spike-timing-dependent plasticity (STDP).

Both forms result in stimulus-specific re-weighting and structural reorganization (growth/pruning of dendritic claws or microglomeruli). Computational analyses show such plasticity enhances pattern decorrelation, code dimensionality, and the capacity for adaptive generalization (Rudelt et al., 13 Nov 2025).

2. Architectural and Algorithmic Realizations in Artificial Neural Networks

Expansion layer plasticity in artificial systems manifests as both dynamic architectural modification and synaptic plasticity within recoding layers:

Neuron and filter growth/pruning (structural plasticity):
- Plastic Support Structure (PSS): Monitoring semantic drift $\|W_i^t - W_i^{t-1}\|^2$ in hidden units, splitting neurons and growing a branch column only if drift exceeds a threshold. Pathways from split neurons are isolated to prevent disturbance of prior representations (Kanaan et al., 2021).
- Adaptive-Growth CNNs: Iterative addition of convolutional kernels when input patches are insufficiently covered by existing filters, as determined by an “inactive ratio” $H(X)$ based on patch-wise activations. Growth proceeds until $H(X)$ falls below a fixed threshold (Zhu et al., 2023).
- Dynamic Generative Memory (DGM): Expansion of layers occurs when the binary usage mask $m_\ell$ becomes too sparse, i.e., the active unit ratio $r_\ell^{(t)}$ drops below a threshold. Capacity increases are coupled to the learning of binary activation or weight masks (Ostapenko et al., 2019).
- Neuroplastic Expansion in RL (NE): Periodic addition of new neurons/connections per layer using large gradient magnitude as growth criterion, with compensating pruning of “dormant” (inactive) neurons according to the ratio of their averaged activations. Growth/pruning rates annealed over training for convergence (Liu et al., 10 Oct 2024).
Plasticity scheduling and gating:
- Neural Plasticity Networks (NPNs): All units and/or parameters are equipped with stochastic binary gates, whose activation probabilities are learned under an $L_0$ -norm regularized objective. Small $k$ in the gate sigmoid schedule allows dormant units to be awakened (expansion), while large $k$ enforces hard pruning (stability) (Li et al., 2019).
- Adaptive Rational Activation Functions (ARAF): For new subspaces in subnetwork expansion, learnable rational activations replace ReLU, endowing new layers with greater flexibility to represent novel functions rapidly (Jaziri et al., 19 Aug 2024).

3. Trigger and Regulation Mechanisms for Expansion

Expansion is typically triggered by data- or task-driven criteria that signal insufficient representational capacity or deteriorating discriminative performance:

Performance-based triggers: Expansion occurs if the performance metric $W_i$ for all current subspaces falls below a certain threshold on a new task, prompting the allocation of a new anchor (as in Continual DQN Expansion) (Jaziri et al., 19 Aug 2024).
Semantic drift: Significant change in parameter vectors of hidden neurons beyond layer-specific thresholds signals the need for splitting (as in PSS) (Kanaan et al., 2021).
Mask-based utilization ratios: Fraction of active units in learned masks falls below $\tau$ , indicating underuse and justifying capacity growth (as in DGM) (Ostapenko et al., 2019).
Activation coverage: Unsupervised coverage measures such as $H(X)$ —the ratio of input regions not recognized by any filter—drive kernel addition in CNNs (Zhu et al., 2023).
Gradient-driven selection: In RL, connections with highest average gradient magnitude are chosen for expansion to maximize learning impact (Liu et al., 10 Oct 2024).

Regulation of expansion typically involves annealing growth budgets, explicit upper bounds on subspace or unit counts, or soft penalties (e.g., $L_0$ norm or mask regularizers) to prevent unbounded parameter proliferation.

4. Plasticity–Stability Tradeoff and Forgetting Mitigation

Expansion layer plasticity must be balanced by mechanisms to protect acquired knowledge (stability):

Structural/functional compartmentalization: New branches (as in PSS or continual DQN expansion) are structurally isolated from old weights, so plasticity in added units does not overwrite prior learned representations (Kanaan et al., 2021, Jaziri et al., 19 Aug 2024).
Selective retraining and weight freezing: Once parameters fall below a “freeze” threshold, their gradients are masked to zero; only new or active units update on new tasks (Kanaan et al., 2021, Ostapenko et al., 2019).
Regularization terms: Elastic Weight Consolidation (EWC) and $L_0$ penalties assign higher cost to changing parameters deemed important for previous tasks (Jaziri et al., 19 Aug 2024, Li et al., 2019).
Experience review: In RL, decaying plasticity (measured by dormant-neuron ratios) triggers prioritized replay of old experiences, mitigating drift and enabling consolidation (Liu et al., 10 Oct 2024).

Ablation studies confirm that removal of consolidation modules (e.g., experience review, EWC, mask regularization) substantially reduces long-term retention without directional benefit for plasticity.

5. Empirical Findings Across Domains

Expansion layer plasticity confers demonstrated performance benefits in multiple settings:

Continual supervised learning:
- PSS and DEN achieve comparable average accuracies on continual MNIST and MNIST+noise streams (e.g., 95.05% vs. 95.2% on MNIST) using fewer parameters and with task-agnostic inference (Kanaan et al., 2021).
- In DGM, dynamic expansion boosts Split CIFAR-100 accuracy from ~61.2% (static) to 66.8% (+30% parameter growth) (Ostapenko et al., 2019).
- NPNs attain high test accuracy with either substantial pruning or expansion from minimal initial architectures (e.g., LeNet-5 pruned from $\sim 4.2\times10^5$ to $5.3\times10^3$ params, accuracy 98.9%) (Li et al., 2019).
Unsupervised/convolutional learning:
- Adaptive-growth generated convolutional layers outperform supervised counterparts across MNIST, CIFAR-10/100 and in transfer learning: e.g., expanding CIFAR-10-trained layers on CIFAR-100 increased accuracy when growing kernel count from 46 to 84 (Zhu et al., 2023).
Reinforcement learning:
- NE in MuJoCo tasks achieves the highest average active-neuron ratio (0.75 vs. 0.70 for Plasticity-Injection, 0.45 for vanilla TD3) and return (e.g., 3850 ± 140 on Hopper) (Liu et al., 10 Oct 2024).
- Continual DQN Expansion with ARAF accelerates convergence by ~20% in curriculum-based train scheduling tasks, with specialized anchors improving total task completion rate compared to rehearsal-augmented baselines (Jaziri et al., 19 Aug 2024).
Condensed matter systems:
- In crystalline layered magnets (CrI₃, CrCl₃), mechanical expansion-layer plasticity arises due to weak interlayer interactions, enabling large reversible deformations. Bilayers exhibit higher Young’s modulus ( $E_{2L} = 62.1$  GPa for CrCl₃) than thicker stacks (down to $E_{10L} \approx 27$  GPa), with breakdown strains $>6\%$ (Cantos-Prieto et al., 2020). This suggests a continuum between adaptive information processing and physical layer plasticity at the materials level.

6. Theoretical and Computational Implications

Expansion layer plasticity is theoretically linked to:

Pattern separation and code dimensionality: Activity-dependent or reinforcement-gated plasticity within expansion layers increases participation ratio $D$ and reduces correlation among output features (Rudelt et al., 13 Nov 2025).
Sample efficiency and generalization: Plastic adaptation of expansion layers reduces downstream learning complexity by decorrelating input representations and aligning them with task-predictive features.
Unified frameworks: The $L_0$ -regularized, gate-based NPN formalism provides an end-to-end trainable mechanism for both pruning and expansion, interpolating smoothly between static, over-complete, and under-complete architectures (Li et al., 2019).

A plausible implication is that future designs of adaptive neural systems, whether artificial or synthetic biological constructs, will require fine-grained control over expansion layer plasticity to match the adaptive flexibility of natural circuits.

7. Open Questions and Future Directions

Active research areas and open problems include:

Biological–artificial analogies: The detailed mapping between plasticity paradigms in animal expansion layers (STDP, reinforcement-gated, homeostatic, structural) and their ANN analogs remains incomplete (Rudelt et al., 13 Nov 2025).
Optimal plasticity scheduling: How to modulate plasticity parameters ( $k$ , mask relaxations, growth/prune intervals) for arbitrary continual or curriculum regimes.
Expansion/pruning efficiency: Theoretical limits on parameter growth, pruning strategies for sustainability, and explicit trade-off curves for stability versus plasticity.
Cross-domain transfer: Mechanisms by which plastic expansion layers can be expanded, frozen, and then further adapted in new domains without catastrophic interference, as demonstrated in unsupervised and transfer learning scenarios (Zhu et al., 2023).
Material property–information processing ties: As indicated by mechanical plasticity studies in 2D magnets (Cantos-Prieto et al., 2020), there are potential intersections between electronic, mechanical, and informational plasticity in layered devices.
Integration with meta-learning or self-supervision: How self-supervised signals or meta-gradients can regulate expansion layer plasticity for high-level goal-driven continual learning.

Expansion layer plasticity thus constitutes a central axis of ongoing research connecting neural circuit biology, continual/unsupervised learning in modern AI, and adaptive material systems, with converging evidence pointing to its necessity for scalable, robust, and generalizable representation learning.