Papers
Topics
Authors
Recent
Search
2000 character limit reached

Per-Shot Plasticity in Neural Systems

Updated 5 July 2026
  • Per-Shot Plasticity is the capability of neural networks to dynamically update synaptic weights during each interaction, enabling rapid adaptation.
  • It spans various mechanisms from Hebbian updates and neuromodulatory gating in spiking models to metrics in continual learning and pruning.
  • Experimental results show enhanced one-shot learning, pattern completion, and robustness across neural architectures through adaptive weight dynamics.

Searching arXiv for the cited work and closely related papers on per-shot plasticity. Per-shot plasticity denotes a family of mechanisms and evaluation concepts in which a neural system changes as a function of individual shots, episodes, trials, projection rounds, or adaptation examples, rather than remaining a purely static function between offline training updates. In the most direct mechanistic sense, per-shot plasticity means that synaptic efficacy is modified during execution, so the network can accumulate episode-specific information and use the resulting effective weights immediately in inference. In a broader optimization sense, the term also appears in work on pruning, continual learning, and fine-tuning, where plasticity refers to how readily a model recovers or adapts after each discrete intervention or after each added shot of supervision. Across these usages, a common theme is that adaptation is treated as a dynamic property expressed at the timescale of interaction rather than only as a consequence of slow retraining (Miconi, 2016, Schmidgall et al., 2021, Inamdar et al., 2 Jun 2026).

1. Conceptual scope and major senses of the term

The literature uses closely related notions of plasticity in several technically distinct ways. In online plastic neural networks, the central object is a time-varying synaptic state or trace that evolves within an episode and modifies the effective connection strength seen by subsequent inputs (Miconi, 2016, Schmidgall et al., 2021, Rodriguez et al., 2022). In the Assembly Calculus setting, each projection round updates weights among active neurons, and the next round uses those updated weights to select the next top-kk winners, making plasticity explicitly per shot in an iterated winner-selection-plus-learning process (Constantinides et al., 2021). In continual learning, by contrast, per-shot plasticity is introduced as a metric of adaptation rate as the number of examples from a task increases, rather than as a synaptic update rule (Inamdar et al., 2 Jun 2026).

A second axis of variation concerns what is being adapted. Some works modify fast synaptic states while keeping slow parameters fixed during an episode (Miconi, 2016, Rodriguez et al., 2022, Chaudhary, 24 Oct 2025). Others jointly optimize slow weights and the parameters governing fast plasticity, so that meta-learning determines not only a representation but also how that representation should change online (Miconi, 2016, Schmidgall et al., 2021, Chaudhary, 24 Oct 2025). Still other papers use plasticity to describe a model’s ability to recover from pruning during training (Liu et al., 2021) or to improve under downstream fine-tuning after pretraining (Han et al., 11 Feb 2026, Berariu et al., 2021).

Sense of plasticity Representative formulation Example paper
Online synaptic adaptation within a trial/sequence Effective weights include a dynamic trace (Miconi, 2016, Schmidgall et al., 2021, Rodriguez et al., 2022, Chaudhary, 24 Oct 2025)
Per-round assembly reconfiguration Weights update after each projection shot (Constantinides et al., 2021)
Recovery/adaptation speed under few-shot evaluation Adaptation curve summarized by SAUCE (Inamdar et al., 2 Jun 2026)
Recovery after pruning during training Drop/recovery after one-shot pruning plus continued training (Liu et al., 2021)
Downstream retunability after pretraining Fine-tuned performance as plasticity measure (Han et al., 11 Feb 2026, Berariu et al., 2021)

This multiplicity of meanings is not merely terminological. It indicates that “plasticity” can refer either to an internal dynamical mechanism or to an externally measured capacity for adaptation. A plausible implication is that encyclopedia treatment of per-shot plasticity must distinguish mechanistic, algorithmic, and evaluative usages rather than treating them as interchangeable.

2. Synaptic state as a per-shot memory substrate

A canonical formulation appears in "Learning to learn with backpropagation of Hebbian plasticity" (Miconi, 2016). There, each synapse has a baseline weight wkw_k, a plasticity coefficient αk\alpha_k, and a time-varying Hebbian trace Hebbk(t)Hebb_k(t), so the effective contribution becomes

wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).

The Hebbian trace is a running average of pre- and postsynaptic activity,

Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),

and the postsynaptic activity is

y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.

The paper’s central claim is that Hebbian plasticity itself can be trained by backpropagation, allowing a network to learn how its synapses should change online during its lifetime and thereby support one-shot learning, pattern completion, and reversal learning (Miconi, 2016).

"SpikePropamine: Differentiable Plasticity in Spiking Neural Networks" (Schmidgall et al., 2021) extends the same principle to SNNs by replacing the static-weight view with a per-execution or per-trial plastic model. The effective membrane input uses a fixed base weight W(l)W^{(l)} plus a dynamic plastic component: u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t). Here E(l)(t)E^{(l)}(t) is a synaptic trace that evolves online, and wkw_k0 scales the plastic contribution. This architecture allows adaptation during the current interaction rather than only after the trial ends, which the paper identifies as the key mechanism enabling per-shot learning (Schmidgall et al., 2021).

In "Short-Term Plasticity Neurons Learning to Learn and Forget" (Rodriguez et al., 2022), the fast state is written as a synaptic matrix wkw_k1 added to long-term weights: wkw_k2 with update

wkw_k3

The distinctive feature is that each synapse has both a learnable plasticity coefficient and a learnable decay coefficient, so the model performs learning followed by forgetting on a per-synapse basis (Rodriguez et al., 2022).

A more recent Transformer formulation uses the same decomposition into slow and fast weights. "Enabling Robust In-Context Memory and Rapid Task Adaptation in Transformers with Hebbian and Gradient-Based Plasticity" (Chaudhary, 24 Oct 2025) augments decoder-only Transformers with fast-weight modules in the FFNs: wkw_k4 where wkw_k5 are static weights and wkw_k6 are fast weights initialized to zero at the start of every sequence and updated online. This explicitly converts otherwise static inference-time computation into token-by-token parameter adaptation within a sequence (Chaudhary, 24 Oct 2025).

Taken together, these formulations share a structural pattern: a slow parameter component stores task-general regularities, while a fast trace stores shot-specific information. This suggests that per-shot plasticity is best understood not as an alternative to conventional learning, but as a second timescale layered on top of it.

3. Learned plasticity rules, decay, and neuromodulation

A central technical issue is the form of the plasticity update. In SpikePropamine, the general differentiable plasticity rule is

wkw_k7

with a simple DP-Linear Decay instance

wkw_k8

The same framework learns parameters of BCM, Oja’s, and neuromodulated variants through gradient descent, using BPTT with surrogate gradients via SLAYER, so gradients flow through both network dynamics and plasticity dynamics (Schmidgall et al., 2021).

The neuromodulated extension is especially relevant to per-shot plasticity because it allows context-dependent gating: wkw_k9 where αk\alpha_k0 denotes neuromodulatory signals that can be local or global and support three-factor learning in the spirit of dopamine-like modulation (Schmidgall et al., 2021). In NDP-Oja’s and NDP-BCM, αk\alpha_k1 gates the plastic change, and in the BCM variant αk\alpha_k2 acts as a learned sliding threshold (Schmidgall et al., 2021).

The Transformer fast-weight work also relies on neuromodulatory gating. For Hebbian plasticity,

αk\alpha_k3

where αk\alpha_k4 is a learned global gate and αk\alpha_k5 are per-connection plasticity rates (Chaudhary, 24 Oct 2025). The paper reports that learned modulatory signals separate the two update regimes: gradient-based plasticity maintains relatively large and persistent αk\alpha_k6, whereas Hebbian plasticity yields tiny average αk\alpha_k7 values but sharp bursts around salient support events. For copying, the reported αk\alpha_k8 means are approximately αk\alpha_k9 for gradient plasticity and Hebbk(t)Hebb_k(t)0 for Hebbian plasticity; for CIFAR-FS they are approximately Hebbk(t)Hebb_k(t)1 and Hebbk(t)Hebb_k(t)2, respectively (Chaudhary, 24 Oct 2025).

Decay and forgetting are equally central. In BOHP, the time constant Hebbk(t)Hebb_k(t)3 controls how fast the Hebbian trace strengthens or decays (Miconi, 2016). In STPN, forgetting is parameterized directly through Hebbk(t)Hebb_k(t)4, so the fast synaptic state can decay quickly or persist, depending on the learned Hebbk(t)Hebb_k(t)5 (Rodriguez et al., 2022). The paper emphasizes that many fast-weight models update memory but do not include a separately learnable decay mechanism per synapse; STPN explicitly learns forgetting dynamics (Rodriguez et al., 2022).

A different but related perspective appears in "Reshaping Neural Representation via Associative, Presynaptic Short-Term Plasticity" (Shimizu et al., 15 Jan 2026). There, the dynamic synapse is written as

Hebbk(t)Hebb_k(t)6

with Hebbk(t)Hebb_k(t)7 the baseline synaptic strength, Hebbk(t)Hebb_k(t)8 the release probability, and Hebbk(t)Hebb_k(t)9 the available resource fraction. The paper derives Fisher-information-based learning rules for both wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).0 and wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).1, and shows that the presynaptic term for wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).2 is phase advanced relative to the input rate and selectively detects stimulus onset (Shimizu et al., 15 Jan 2026). Here per-shot plasticity is not only a memory device; it is a mechanism that changes the temporal transfer function of the synapse itself.

4. Computational regimes and experimental demonstrations

The most direct demonstrations of per-shot plasticity are tasks in which a single example or a short episode must alter subsequent behavior. BOHP shows this in pattern completion, one-shot learning of arbitrary patterns, and reversal learning (Miconi, 2016). In the one-shot task, the network sees two random binary patterns and their labels during the first two time steps, later receives only the patterns with neutral suffix wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).3, and must output the correct labels; one presentation is sufficient to imprint the association (Miconi, 2016). In reversal learning, the labels swap halfway through the episode, and the trained network adapts after seeing each pattern with its updated label once. The paper reports that the relevant pattern-to-hidden plasticity coefficients become negative, and clipping them to positive values prevents learning in reversal learning while still allowing one-shot learning (Miconi, 2016).

SpikePropamine evaluates online plastic SNNs on a temporal cue-association task and on high-dimensional Half-Cheetah locomotion (Schmidgall et al., 2021). The paper states that a fixed-weight SNN fails on the cue-association task, while DP-BCM and DP-Oja’s solve it and NDP-BCM and NDP-Oja’s do even better (Schmidgall et al., 2021). It also reports that neuromodulation can dramatically reduce spiking activity for Oja’s rule, implying better energy efficiency, and that in locomotion the differentiable plasticity networks show better robustness under novel perturbations than fixed-weight SNNs, especially for joint friction and action noise (Schmidgall et al., 2021). This is presented as evidence for per-shot adaptation in robotics.

STPN broadens the range of domains. In Associative Retrieval, the evaluation table reports wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).4 for STPNr, and the appendix reports wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).5 for STPNr with per-synapse STP (Rodriguez et al., 2022). In reinforcement learning, reported inference scores include wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).6 for Maze Exploration, wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).7 for Atari Pong, and wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).8 for MuJoCo Inverted Pendulum (Rodriguez et al., 2022). The paper interprets these results as evidence that trainable short-term plasticity is not confined to spiking models and can support both learning-to-learn and learning-to-forget across supervised and RL settings (Rodriguez et al., 2022).

In Transformers, the empirical picture is regime dependent. The 2025 fast-weight study compares plastic and non-plastic decoder-only Transformers on copying, cue–reward association, few-shot regression, and wkxk(t)+αkHebbk(t)xk(t).w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t).9-way, Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),0-shot classification on CIFAR-FS and Omniglot (Chaudhary, 24 Oct 2025). On copying, the reported validation losses and recall accuracies are Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),1, Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),2 for gradient plasticity; Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),3, Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),4 for Hebbian plasticity; and Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),5, Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),6 for the non-plastic baseline (Chaudhary, 24 Oct 2025). On few-shot regression, the reported query MSE values are Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),7 for gradient, Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),8 for Hebbian, and Hebbk(t)=(1γ)Hebbk(t1)+γxk(t)y(t),Hebb_k(t) = (1-\gamma) * Hebb_k(t-1) + \gamma * x_k(t) * y(t),9 for none (Chaudhary, 24 Oct 2025). On CIFAR-FS and Omniglot, Hebbian plasticity has the best validation accuracy: y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.0 on CIFAR-FS and y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.1 on Omniglot (Chaudhary, 24 Oct 2025). By contrast, on cue–reward association the static baseline performs best, with validation y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.2 and query y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.3, versus y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.4, y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.5 for gradient and y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.6, y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.7 for Hebbian (Chaudhary, 24 Oct 2025). The paper therefore identifies a boundary condition: when associations are short and linearly separable, static weights suffice (Chaudhary, 24 Oct 2025).

The Assembly Calculus work provides a more abstract notion of shot-wise plasticity. In a projection from area y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.8 to area y(t)=tanh{kinputs[wkxk(t)+αkHebbk(t)xk(t)]+b}.y(t) = \tanh\left\{\sum_{k \in inputs}[w_k x_k(t) + \alpha_k Hebb_k(t) x_k(t)] + b\right\}.9, winners are selected, synapses for first-time winners are created, and then plasticity is applied to all winners; repeated shots ideally converge to W(l)W^{(l)}0 (Constantinides et al., 2021). The paper studies Hebb’s rule, Oja’s rule, and STDP, and reports that larger W(l)W^{(l)}1 in Hebb’s rule yields faster convergence, higher W(l)W^{(l)}2 in Oja’s rule yields more steps to convergence and denser assemblies, and inverse-form STDP often causes instability while the step-function version behaves more predictably (Constantinides et al., 2021).

5. From local online learning to biologically plausible credit assignment

Per-shot plasticity is often motivated as a way to avoid the static-weight assumption, but it is also closely tied to the credit-assignment problem. BOHP derives analytical activity gradients through Hebbian plastic connections, showing that gradients at time W(l)W^{(l)}3 depend recursively on earlier times because earlier activity changes later traces and outputs (Miconi, 2016). The paper’s recursive derivatives for W(l)W^{(l)}4 and W(l)W^{(l)}5 make explicit that this is not ordinary backpropagation through fixed weights (Miconi, 2016).

SpikePropamine handles the same issue in SNNs through BPTT with surrogate gradients and SLAYER, so the optimization traverses both the spike dynamics and the plasticity dynamics (Schmidgall et al., 2021). The learnable parameters include base weights W(l)W^{(l)}6, plasticity gains W(l)W^{(l)}7, and rule parameters such as W(l)W^{(l)}8, W(l)W^{(l)}9, u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).0, and u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).1 (Schmidgall et al., 2021). This joint optimization is important because it means the rule governing online change is itself a learned object rather than a fixed biological prior.

A different route is taken in "Using local plasticity rules to train recurrent neural networks" (Marschall et al., 2019). That paper replaces exact BPTT with a set of local plasticity rules distributed across synaptic sub-populations u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).2, u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).3, and u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).4. Learning occurs in distinct somatic and distal phases: the somatic phase updates the main task synapses u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).5 and recurrence-prediction synapses u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).6, while the distal phase updates the credit-assignment synapses u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).7 (Marschall et al., 2019). The explicit update for the dynamics model is

u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).8

intended to approximate the Jacobian of recurrent dynamics (Marschall et al., 2019). The paper describes this most accurately as per-phase plasticity, with an operationally local per-shot-like interpretation at the level of temporal slices or phase-specific updates (Marschall et al., 2019).

This line of work suggests that per-shot plasticity need not mean unrestricted online self-modification. It can also denote a constrained local update that substitutes for globally unfolded credit assignment. In that sense, per-shot plasticity and biological plausibility are often coupled through locality, compartmentalization, or neuromodulatory gating.

6. Evaluation, boundary conditions, and neighboring meanings of plasticity

The term acquires a distinct evaluative meaning in continual learning. "Re-Evaluating Continual Learning with Few-Shot Adaptation" (Inamdar et al., 2 Jun 2026) argues that standard u(l)(t)=(W(l)+α(l)E(l)(t))a(l)(t)+(vs(l))(t).u^{(l)}(t) = \left(W^{(l)} + \boldsymbol\alpha^{(l)}\odot E^{(l)}(t)\right)a^{(l)}(t) + (v*s^{(l)})(t).9-shot metrics are too coarse and introduces per-shot plasticity as the rate of adaptation as more examples from a task are provided. The paper defines the Area Under the Performance-Adaptation Curve (AUAC) and a normalized score called SAUCE, with lower SAUCE indicating faster adaptation (Inamdar et al., 2 Jun 2026). It evaluates checkpoints on every task using E(l)(t)E^{(l)}(t)0 shots and 10 gradient steps, and reports that many apparently catastrophic forgetting effects are partially recoverable with only E(l)(t)E^{(l)}(t)1 shots (Inamdar et al., 2 Jun 2026). It also finds that SGD is typically the most plastic under SAUCE, replay methods can have strong eventual performance but slower adaptation, and foresight meta-learning induces a steady increase in plasticity over the sequence on every benchmark except Rot-MNIST-20 (Inamdar et al., 2 Jun 2026).

A neighboring but distinct usage appears in pruning. "Sparse Training via Boosting Pruning Plasticity with Neuroregeneration" (Liu et al., 2021) defines pruning plasticity as the drop or recovery in test accuracy caused by pruning and then briefly continuing training with the current learning rate. Formally, for one-shot pruning at epoch E(l)(t)E^{(l)}(t)2, pruning plasticity is

E(l)(t)E^{(l)}(t)3

and the paper shows that plasticity decreases as training progresses, especially after the second learning-rate decay, while neuroregeneration improves plasticity across all cases (Liu et al., 2021). Here “per-shot” refers not to a data example but to a discrete pruning event; the relevant question is whether the model can bounce back after each intervention.

In language-model pretraining and transfer learning, plasticity is used again in a broader downstream-adaptability sense. "Weight Decay Improves LLM Plasticity" (Han et al., 11 Feb 2026) defines plasticity as a pretrained model’s ability to adapt successfully to new data under further training. The paper measures this by fine-tuning pretrained models on six Chain-of-Thought tasks and shows that larger pretraining weight decay tends to yield better downstream performance, with best downstream weight decay values generally above the default E(l)(t)E^{(l)}(t)4 (Han et al., 11 Feb 2026). In the E(l)(t)E^{(l)}(t)5 TPP regime, the best downstream weight decay is E(l)(t)E^{(l)}(t)6 for Llama-2-0.5B-20x, Llama-2-1B-20x, Llama-2-4B-20x, and OLMo-2-1B-20x; in the E(l)(t)E^{(l)}(t)7 TPP regime, it is E(l)(t)E^{(l)}(t)8 for OLMo-2-1B-140x (Han et al., 11 Feb 2026). The paper links improved plasticity to linearly separable representations, reduced pseudo-rank of attention-related matrices, and reduced train-val gap (Han et al., 11 Feb 2026).

By contrast, "A study on the plasticity of neural networks" (Berariu et al., 2021) emphasizes loss of plasticity under pretraining. It defines plasticity as the ability of a network to keep learning new tasks without its eventual performance being harmed by previously seen tasks, and studies a generalisation gap in which a pretrained model, even on the same distribution, reaches worse test performance than a fresh model trained from scratch (Berariu et al., 2021). On ResNet-18 and CIFAR-10, the gap appears after only E(l)(t)E^{(l)}(t)9–wkw_k00 epochs of pretraining with Adam, persists across Adam, RMSprop, SGD, and SGD with momentum, worsens across multiple stages, and is substantially reduced by increasing the tuning learning rate by wkw_k01 (Berariu et al., 2021). This provides an important caution: prior learning can either facilitate or impair later adaptation, depending on the optimization regime.

These adjacent usages show that per-shot plasticity is not a single formalism but a spectrum of related concerns: immediate synaptic adaptation, rapid recoverability, and retained capacity for future change. A plausible implication is that the field increasingly treats plasticity as an end-to-end systems property that depends jointly on architecture, learning rule, timescale separation, and evaluation protocol.

7. Scientific significance and recurrent points of tension

Several themes recur across the literature. First, per-shot plasticity is repeatedly presented as a remedy for the limits of static inference. BOHP contrasts fixed weights with online Hebbian adaptation (Miconi, 2016); SpikePropamine shows that fixed-weight SNNs fail on tasks solved by differentiable plastic SNNs (Schmidgall et al., 2021); the Transformer fast-weight work frames plasticity as making adaptation explicit and persistent within a sequence rather than leaving it implicit in activations and attention patterns (Chaudhary, 24 Oct 2025).

Second, the relative value of plasticity is task dependent. The 2025 Transformer study states that static weights suffice when associations are short and linearly separable (Chaudhary, 24 Oct 2025). The continual-learning evaluation paper similarly shows that methods with strong wkw_k02-shot stability are not necessarily those with the best few-shot adaptation rate (Inamdar et al., 2 Jun 2026). In pruning and pretraining studies, plasticity depends on the learning-rate regime, pruning severity, weight decay, and degree of overfitting (Liu et al., 2021, Han et al., 11 Feb 2026, Berariu et al., 2021). This indicates that there is no uniform claim that “more plasticity” is always better.

Third, fast adaptation and controlled forgetting are often coupled. In STPN, the learnable decay term is a primary mechanism rather than an implementation detail (Rodriguez et al., 2022). In BOHP, negative plasticity coefficients are necessary for reversal learning (Miconi, 2016). In the presynaptic STP theory, release probability plasticity alters the temporal code and biases learned connectivity toward anti-causal associations, yielding ramp-like sustained representations and reverse replay after drive removal (Shimizu et al., 15 Jan 2026). These results suggest that per-shot plasticity is not just about rapid acquisition; it is equally about selective erasure, gating, and temporal asymmetry.

Finally, the concept remains scientifically heterogeneous. In some papers, per-shot plasticity is a local synaptic law executed inside the forward pass (Miconi, 2016, Schmidgall et al., 2021, Rodriguez et al., 2022, Chaudhary, 24 Oct 2025). In others, it is a measurement of how quickly performance changes with each additional shot of evidence (Inamdar et al., 2 Jun 2026). In still others, plasticity refers to recovery after pruning or adaptability after pretraining (Liu et al., 2021, Han et al., 11 Feb 2026, Berariu et al., 2021). The shared denominator is the treatment of learning as something that can and should be analyzed at the granularity of discrete interactions. That broad convergence, across spiking systems, recurrent networks, Transformers, assembly models, continual learners, and sparse-training methods, is the clearest unifying feature of the modern literature on per-shot plasticity.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Per-Shot Plasticity.