---
title: Per-Shot Plasticity in Neural Systems
url: https://www.emergentmind.com/topics/per-shot-plasticity
type: topic
---

# Per-Shot Plasticity in Neural Systems

Searching arXiv for the cited work and closely related papers on per-shot plasticity.
Per-shot plasticity denotes a family of mechanisms and evaluation concepts in which a neural system changes as a function of individual shots, episodes, trials, projection rounds, or adaptation examples, rather than remaining a purely static function between offline training updates. In the most direct mechanistic sense, per-shot plasticity means that synaptic efficacy is modified during execution, so the network can accumulate episode-specific information and use the resulting effective weights immediately in inference. In a broader optimization sense, the term also appears in work on pruning, continual learning, and fine-tuning, where plasticity refers to how readily a model recovers or adapts after each discrete intervention or after each added shot of supervision. Across these usages, a common theme is that adaptation is treated as a dynamic property expressed at the timescale of interaction rather than only as a consequence of slow retraining [1609.02228][2106.02681][2606.03843].

## 1. Conceptual scope and major senses of the term

The literature uses closely related notions of plasticity in several technically distinct ways. In online plastic neural networks, the central object is a time-varying synaptic state or trace that evolves within an episode and modifies the effective connection strength seen by subsequent inputs [1609.02228][2106.02681][2206.14048]. In the Assembly Calculus setting, each projection round updates weights among active neurons, and the next round uses those updated weights to select the next top-\(k\) winners, making plasticity explicitly per shot in an iterated winner-selection-plus-learning process [2112.14853]. In continual learning, by contrast, per-shot plasticity is introduced as a metric of adaptation rate as the number of examples from a task increases, rather than as a synaptic update rule [2606.03843].

A second axis of variation concerns what is being adapted. Some works modify fast synaptic states while keeping slow parameters fixed during an episode [1609.02228][2206.14048][2510.21908]. Others jointly optimize slow weights and the parameters governing fast plasticity, so that meta-learning determines not only a representation but also how that representation should change online [1609.02228][2106.02681][2510.21908]. Still other papers use plasticity to describe a model’s ability to recover from pruning during training [2106.10404] or to improve under downstream fine-tuning after pretraining [2602.11137][2106.00042].

| Sense of plasticity | Representative formulation | Example paper |
|---|---|---|
| Online synaptic adaptation within a trial/sequence | Effective weights include a dynamic trace | [1609.02228], [2106.02681], [2206.14048], [2510.21908] |
| Per-round assembly reconfiguration | Weights update after each projection shot | [2112.14853] |
| Recovery/adaptation speed under few-shot evaluation | Adaptation curve summarized by SAUCE | [2606.03843] |
| Recovery after pruning during training | Drop/recovery after one-shot pruning plus continued training | [2106.10404] |
| Downstream retunability after pretraining | Fine-tuned performance as plasticity measure | [2602.11137], [2106.00042] |

This multiplicity of meanings is not merely terminological. It indicates that “plasticity” can refer either to an internal dynamical mechanism or to an externally measured capacity for adaptation. A plausible implication is that encyclopedia treatment of per-shot plasticity must distinguish mechanistic, algorithmic, and evaluative usages rather than treating them as interchangeable.

## 2. Synaptic state as a per-shot memory substrate

A canonical formulation appears in "Learning to learn with backpropagation of Hebbian plasticity" [1609.02228]. There, each synapse has a baseline weight \(w_k\), a plasticity coefficient \(\alpha_k\), and a time-varying Hebbian trace \(Hebb_k(t)\), so the effective contribution becomes
\[
w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).
\]
The Hebbian trace is a running average of pre- and postsynaptic activity,
\[
Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),
\]
and the postsynaptic activity is
\[
y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.
\]
The paper’s central claim is that Hebbian plasticity itself can be trained by backpropagation, allowing a network to learn how its synapses should change online during its lifetime and thereby support one-shot learning, pattern completion, and reversal learning [1609.02228].

"SpikePropamine: Differentiable Plasticity in Spiking Neural Networks" [2106.02681] extends the same principle to SNNs by replacing the static-weight view with a per-execution or per-trial plastic model. The effective membrane input uses a fixed base weight \(W^{(l)}\) plus a dynamic plastic component:
\[
u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).
\]
Here \(E^{(l)}(t)\) is a synaptic trace that evolves online, and \(\boldsymbol\alpha^{(l)}\) scales the plastic contribution. This architecture allows adaptation during the current interaction rather than only after the trial ends, which the paper identifies as the key mechanism enabling per-shot learning [2106.02681].

In "Short-Term Plasticity Neurons Learning to Learn and Forget" [2206.14048], the fast state is written as a synaptic matrix \(\boldsymbol{F}^{(t)}\) added to long-term weights:
\[
\boldsymbol{G}^{(t)} = \boldsymbol{W} + \boldsymbol{F}^{(t)},
\qquad
\boldsymbol{h}^{(t)} = \sigma\!\left(\boldsymbol{G}^{(t)} \boldsymbol{x}^{(t)}\right),
\]
with update
\[
\boldsymbol{F}^{(t+1)}= \boldsymbol{\Gamma} \odot \left(\boldsymbol{x}^{(t)} \otimes \boldsymbol{h}^{(t)}\right) +(\mathds{1}-\boldsymbol{\Lambda}) \odot \boldsymbol{F}^{(t)}.
\]
The distinctive feature is that each synapse has both a learnable plasticity coefficient and a learnable decay coefficient, so the model performs learning followed by forgetting on a per-synapse basis [2206.14048].

A more recent Transformer formulation uses the same decomposition into slow and fast weights. "Enabling Robust In-Context Memory and Rapid Task Adaptation in Transformers with Hebbian and Gradient-Based Plasticity" [2510.21908] augments decoder-only Transformers with fast-weight modules in the FFNs:
\[
W_l(t) = \tilde{W}_l + w_l(t),
\]
where \(\tilde{W}_l\) are static weights and \(w_l(t)\) are fast weights initialized to zero at the start of every sequence and updated online. This explicitly converts otherwise static inference-time computation into token-by-token parameter adaptation within a sequence [2510.21908].

Taken together, these formulations share a structural pattern: a slow parameter component stores task-general regularities, while a fast trace stores shot-specific information. This suggests that per-shot plasticity is best understood not as an alternative to conventional learning, but as a second timescale layered on top of it.

## 3. Learned plasticity rules, decay, and neuromodulation

A central technical issue is the form of the plasticity update. In SpikePropamine, the general differentiable plasticity rule is
\[
E^{(l)}(t+\Delta\tau) = F(\boldsymbol\rho^{(l-1)}(t), \boldsymbol\rho^{(l)}(t), E^{(l)}(t), L^{(l)}),
\]
with a simple DP-Linear Decay instance
\[
E^{(l)}(t+\Delta\tau) = (1-\eta^{(l)})E^{(l)}(t) + \eta^{(l)}(\boldsymbol\rho^{(l)}(t))^\intercal\boldsymbol\rho^{(l-1)}(t).
\]
The same framework learns parameters of BCM, Oja’s, and neuromodulated variants through gradient descent, using BPTT with surrogate gradients via SLAYER, so gradients flow through both network dynamics and plasticity dynamics [2106.02681].

The neuromodulated extension is especially relevant to per-shot plasticity because it allows context-dependent gating:
\[
E^{(l)}(t+\Delta\tau) = G(\boldsymbol\rho^{(l-1)}(t), \boldsymbol\rho^{(l)}(t), E^{(l)}(t), L^{(l)}, M),
\]
where \(M\) denotes neuromodulatory signals that can be local or global and support three-factor learning in the spirit of dopamine-like modulation [2106.02681]. In NDP-Oja’s and NDP-BCM, \(M^{(l)}(t) = W^{(l)}_{m} r^{(l)}(t)\) gates the plastic change, and in the BCM variant \(\boldsymbol\phi^{(l)}\) acts as a learned sliding threshold [2106.02681].

The Transformer fast-weight work also relies on neuromodulatory gating. For Hebbian plasticity,
\[
w_{l}(t+1) = (1-\eta(t))w_{l}(t) + \eta(t)\,\alpha_{l} \circ \big(p_{l}(t)q_{l}^{\top}(t)\big),
\]
where \(\eta(t)\) is a learned global gate and \(\alpha_l\) are per-connection plasticity rates [2510.21908]. The paper reports that learned modulatory signals separate the two update regimes: gradient-based plasticity maintains relatively large and persistent \(\eta(t)\), whereas Hebbian plasticity yields tiny average \(\eta(t)\) values but sharp bursts around salient support events. For copying, the reported \(\eta(t)\) means are approximately \(8.4 \times 10^{-2}\) for gradient plasticity and \(4.3 \times 10^{-4}\) for Hebbian plasticity; for CIFAR-FS they are approximately \(0.116\) and \(2.3 \times 10^{-4}\), respectively [2510.21908].

Decay and forgetting are equally central. In BOHP, the time constant \(\gamma\) controls how fast the Hebbian trace strengthens or decays [1609.02228]. In STPN, forgetting is parameterized directly through \(\boldsymbol{\Lambda}\), so the fast synaptic state can decay quickly or persist, depending on the learned \(\lambda_{ij}\) [2206.14048]. The paper emphasizes that many fast-weight models update memory but do not include a separately learnable decay mechanism per synapse; STPN explicitly learns forgetting dynamics [2206.14048].

A different but related perspective appears in "Reshaping Neural Representation via Associative, Presynaptic Short-Term Plasticity" [2601.10397]. There, the dynamic synapse is written as
\[
w_{ij}(t)=w_{ij}^0 U_{ij} d_{ij}(t),
\]
with \(w_{ij}^0\) the baseline synaptic strength, \(U_{ij}\) the release probability, and \(d_{ij}(t)\) the available resource fraction. The paper derives Fisher-information-based learning rules for both \(w_0\) and \(U\), and shows that the presynaptic term for \(U\) is phase advanced relative to the input rate and selectively detects stimulus onset [2601.10397]. Here per-shot plasticity is not only a memory device; it is a mechanism that changes the temporal transfer function of the synapse itself.

## 4. Computational regimes and experimental demonstrations

The most direct demonstrations of per-shot plasticity are tasks in which a single example or a short episode must alter subsequent behavior. BOHP shows this in pattern completion, one-shot learning of arbitrary patterns, and reversal learning [1609.02228]. In the one-shot task, the network sees two random binary patterns and their labels during the first two time steps, later receives only the patterns with neutral suffix \(00\), and must output the correct labels; one presentation is sufficient to imprint the association [1609.02228]. In reversal learning, the labels swap halfway through the episode, and the trained network adapts after seeing each pattern with its updated label once. The paper reports that the relevant pattern-to-hidden plasticity coefficients become negative, and clipping them to positive values prevents learning in reversal learning while still allowing one-shot learning [1609.02228].

SpikePropamine evaluates online plastic SNNs on a temporal cue-association task and on high-dimensional Half-Cheetah locomotion [2106.02681]. The paper states that a fixed-weight SNN fails on the cue-association task, while DP-BCM and DP-Oja’s solve it and NDP-BCM and NDP-Oja’s do even better [2106.02681]. It also reports that neuromodulation can dramatically reduce spiking activity for Oja’s rule, implying better energy efficiency, and that in locomotion the differentiable plasticity networks show better robustness under novel perturbations than fixed-weight SNNs, especially for joint friction and action noise [2106.02681]. This is presented as evidence for per-shot adaptation in robotics.

STPN broadens the range of domains. In Associative Retrieval, the evaluation table reports \(98.55 \pm 2.76\) for STPNr, and the appendix reports \(99.99 \pm 0.01\) for STPNr with per-synapse STP [2206.14048]. In reinforcement learning, reported inference scores include \(132.7 \pm 1.4\) for Maze Exploration, \(20.7 \pm 0.5\) for Atari Pong, and \(985.0 \pm 15.0\) for MuJoCo Inverted Pendulum [2206.14048]. The paper interprets these results as evidence that trainable short-term plasticity is not confined to spiking models and can support both learning-to-learn and learning-to-forget across supervised and RL settings [2206.14048].

In Transformers, the empirical picture is regime dependent. The 2025 fast-weight study compares plastic and non-plastic decoder-only Transformers on copying, cue–reward association, few-shot regression, and \(5\)-way, \(1\)-shot classification on CIFAR-FS and Omniglot [2510.21908]. On copying, the reported validation losses and recall accuracies are \(0.352 \pm 0.021\), \(0.745 \pm 0.011\) for gradient plasticity; \(0.345 \pm 0.009\), \(0.727 \pm 0.007\) for Hebbian plasticity; and \(0.415 \pm 0.105\), \(0.721 \pm 0.025\) for the non-plastic baseline [2510.21908]. On few-shot regression, the reported query MSE values are \(1.589 \pm 0.038\) for gradient, \(1.546 \pm 0.064\) for Hebbian, and \(1.997 \pm 0.266\) for none [2510.21908]. On CIFAR-FS and Omniglot, Hebbian plasticity has the best validation accuracy: \(0.319 \pm 0.017\) on CIFAR-FS and \(0.237 \pm 0.028\) on Omniglot [2510.21908]. By contrast, on cue–reward association the static baseline performs best, with validation \(0.027 \pm 0.010\) and query \(0.053 \pm 0.020\), versus \(0.035 \pm 0.008\), \(0.064 \pm 0.020\) for gradient and \(0.037 \pm 0.014\), \(0.065 \pm 0.036\) for Hebbian [2510.21908]. The paper therefore identifies a boundary condition: when associations are short and linearly separable, static weights suffice [2510.21908].

The Assembly Calculus work provides a more abstract notion of shot-wise plasticity. In a projection from area \(A\) to area \(B\), winners are selected, synapses for first-time winners are created, and then plasticity is applied to all winners; repeated shots ideally converge to \(y_t = y_{t+1}\) [2112.14853]. The paper studies Hebb’s rule, Oja’s rule, and STDP, and reports that larger \(\beta\) in Hebb’s rule yields faster convergence, higher \(\alpha\) in Oja’s rule yields more steps to convergence and denser assemblies, and inverse-form STDP often causes instability while the step-function version behaves more predictably [2112.14853].

## 5. From local online learning to biologically plausible credit assignment

Per-shot plasticity is often motivated as a way to avoid the static-weight assumption, but it is also closely tied to the credit-assignment problem. BOHP derives analytical activity gradients through Hebbian plastic connections, showing that gradients at time \(t_z\) depend recursively on earlier times because earlier activity changes later traces and outputs [1609.02228]. The paper’s recursive derivatives for \(\partial y(t_z)/\partial w_k\) and \(\partial y(t_z)/\partial \alpha_k\) make explicit that this is not ordinary backpropagation through fixed weights [1609.02228].

SpikePropamine handles the same issue in SNNs through BPTT with surrogate gradients and SLAYER, so the optimization traverses both the spike dynamics and the plasticity dynamics [2106.02681]. The learnable parameters include base weights \(W^{(l)}\), plasticity gains \(\boldsymbol\alpha^{(l)}\), and rule parameters such as \(\eta^{(l)}\), \(\eta_\phi^{(l)}\), \(\boldsymbol\psi^{(l)}\), and \(W_m^{(l)}\) [2106.02681]. This joint optimization is important because it means the rule governing online change is itself a learned object rather than a fixed biological prior.

A different route is taken in "Using local plasticity rules to train recurrent neural networks" [1905.12100]. That paper replaces exact BPTT with a set of local plasticity rules distributed across synaptic sub-populations \(W\), \(A\), and \(J\). Learning occurs in distinct somatic and distal phases: the somatic phase updates the main task synapses \(W\) and recurrence-prediction synapses \(J\), while the distal phase updates the credit-assignment synapses \(A\) [1905.12100]. The explicit update for the dynamics model is
\[
\Delta J_{ij} \propto r_i^{(t)}\left(r_j^{(t+1)} - \sum_k J_{jk} r_k^{(t)}\right),
\]
intended to approximate the Jacobian of recurrent dynamics [1905.12100]. The paper describes this most accurately as per-phase plasticity, with an operationally local per-shot-like interpretation at the level of temporal slices or phase-specific updates [1905.12100].

This line of work suggests that per-shot plasticity need not mean unrestricted online self-modification. It can also denote a constrained local update that substitutes for globally unfolded credit assignment. In that sense, per-shot plasticity and biological plausibility are often coupled through locality, compartmentalization, or neuromodulatory gating.

## 6. Evaluation, boundary conditions, and neighboring meanings of plasticity

The term acquires a distinct evaluative meaning in continual learning. "Re-Evaluating Continual Learning with Few-Shot Adaptation" [2606.03843] argues that standard \(0\)-shot metrics are too coarse and introduces per-shot plasticity as the rate of adaptation as more examples from a task are provided. The paper defines the Area Under the Performance-Adaptation Curve (AUAC) and a normalized score called SAUCE, with lower SAUCE indicating faster adaptation [2606.03843]. It evaluates checkpoints on every task using \(k \in \{0,1,2,5,10\}\) shots and 10 gradient steps, and reports that many apparently catastrophic forgetting effects are partially recoverable with only \(5\) shots [2606.03843]. It also finds that SGD is typically the most plastic under SAUCE, replay methods can have strong eventual performance but slower adaptation, and foresight meta-learning induces a steady increase in plasticity over the sequence on every benchmark except Rot-MNIST-20 [2606.03843].

A neighboring but distinct usage appears in pruning. "Sparse Training via Boosting Pruning Plasticity with Neuroregeneration" [2106.10404] defines pruning plasticity as the drop or recovery in test accuracy caused by pruning and then briefly continuing training with the current learning rate. Formally, for one-shot pruning at epoch \(t\), pruning plasticity is
\[
t_{\mathrm{Contrain}^k(W_t \odot m_t, a_t)} - t_{\mathrm{pre}},
\]
and the paper shows that plasticity decreases as training progresses, especially after the second learning-rate decay, while neuroregeneration improves plasticity across all cases [2106.10404]. Here “per-shot” refers not to a data example but to a discrete pruning event; the relevant question is whether the model can bounce back after each intervention.

In language-model pretraining and transfer learning, plasticity is used again in a broader downstream-adaptability sense. "Weight Decay Improves Language Model Plasticity" [2602.11137] defines plasticity as a pretrained model’s ability to adapt successfully to new data under further training. The paper measures this by fine-tuning pretrained models on six Chain-of-Thought tasks and shows that larger pretraining weight decay tends to yield better downstream performance, with best downstream weight decay values generally above the default \(0.1\) [2602.11137]. In the \(20\) TPP regime, the best downstream weight decay is \(1.0\) for Llama-2-0.5B-20x, Llama-2-1B-20x, Llama-2-4B-20x, and OLMo-2-1B-20x; in the \(140\) TPP regime, it is \(0.3\) for OLMo-2-1B-140x [2602.11137]. The paper links improved plasticity to linearly separable representations, reduced pseudo-rank of attention-related matrices, and reduced train-val gap [2602.11137].

By contrast, "A study on the plasticity of neural networks" [2106.00042] emphasizes loss of plasticity under pretraining. It defines plasticity as the ability of a network to keep learning new tasks without its eventual performance being harmed by previously seen tasks, and studies a generalisation gap in which a pretrained model, even on the same distribution, reaches worse test performance than a fresh model trained from scratch [2106.00042]. On ResNet-18 and CIFAR-10, the gap appears after only \(5\)–\(10\) epochs of pretraining with Adam, persists across Adam, RMSprop, SGD, and SGD with momentum, worsens across multiple stages, and is substantially reduced by increasing the tuning learning rate by \(10\times\) [2106.00042]. This provides an important caution: prior learning can either facilitate or impair later adaptation, depending on the optimization regime.

These adjacent usages show that per-shot plasticity is not a single formalism but a spectrum of related concerns: immediate synaptic adaptation, rapid recoverability, and retained capacity for future change. A plausible implication is that the field increasingly treats plasticity as an end-to-end systems property that depends jointly on architecture, learning rule, timescale separation, and evaluation protocol.

## 7. Scientific significance and recurrent points of tension

Several themes recur across the literature. First, per-shot plasticity is repeatedly presented as a remedy for the limits of static inference. BOHP contrasts fixed weights with online Hebbian adaptation [1609.02228]; SpikePropamine shows that fixed-weight SNNs fail on tasks solved by differentiable plastic SNNs [2106.02681]; the Transformer fast-weight work frames plasticity as making adaptation explicit and persistent within a sequence rather than leaving it implicit in activations and attention patterns [2510.21908].

Second, the relative value of plasticity is task dependent. The 2025 Transformer study states that static weights suffice when associations are short and linearly separable [2510.21908]. The continual-learning evaluation paper similarly shows that methods with strong \(0\)-shot stability are not necessarily those with the best few-shot adaptation rate [2606.03843]. In pruning and pretraining studies, plasticity depends on the learning-rate regime, pruning severity, weight decay, and degree of overfitting [2106.10404][2602.11137][2106.00042]. This indicates that there is no uniform claim that “more plasticity” is always better.

Third, fast adaptation and controlled forgetting are often coupled. In STPN, the learnable decay term is a primary mechanism rather than an implementation detail [2206.14048]. In BOHP, negative plasticity coefficients are necessary for reversal learning [1609.02228]. In the presynaptic STP theory, release probability plasticity alters the temporal code and biases learned connectivity toward anti-causal associations, yielding ramp-like sustained representations and reverse replay after drive removal [2601.10397]. These results suggest that per-shot plasticity is not just about rapid acquisition; it is equally about selective erasure, gating, and temporal asymmetry.

Finally, the concept remains scientifically heterogeneous. In some papers, per-shot plasticity is a local synaptic law executed inside the forward pass [1609.02228][2106.02681][2206.14048][2510.21908]. In others, it is a measurement of how quickly performance changes with each additional shot of evidence [2606.03843]. In still others, plasticity refers to recovery after pruning or adaptability after pretraining [2106.10404][2602.11137][2106.00042]. The shared denominator is the treatment of learning as something that can and should be analyzed at the granularity of discrete interactions. That broad convergence, across spiking systems, recurrent networks, Transformers, assembly models, continual learners, and sparse-training methods, is the clearest unifying feature of the modern literature on per-shot plasticity.

Source: https://www.emergentmind.com/topics/per-shot-plasticity