Inference-time Plasticity Explained

• Inference-time plasticity is the online adaptation of synaptic efficacy or fast weights, allowing models to learn on the fly during inference.
• It encompasses mechanisms like short-term synaptic dynamics, Hebbian/gradient-based updates in Transformers, and reservoir-based adjustments to cope with nonstationarity.
• Empirical studies show that such plasticity enhances mixing, speeds up adaptation, and improves performance in dynamic environments without separate offline training.

Inference-time plasticity denotes regimes in which synaptic efficacies or other adaptive parameters continue to change during ongoing operation, so that learning-like state change is part of the inference process rather than being confined to an offline training stage. In the literature, the plastic variable may be the momentary synaptic efficacy $W_i(t)\propto w U R$ induced by short-term synaptic plasticity in spiking Boltzmann-style samplers, a short-term component $\boldsymbol{F}^{(k)}_t$ added to resting weights $\boldsymbol{W}^{(k)}$, continuously evolving STDP weights in unsupervised spiking networks, readout weights updated millisecond-by-millisecond in a timing decoder, or fast weights $w_l(t)$ added to static Transformer parameters $\tilde W_l$ within a sequence (Leng et al., 2017, Moraitis et al., 2020, Gebhardt et al., 2024, Yamada et al., 16 Oct 2025, Chaudhary, 24 Oct 2025). The unifying feature is that the system’s effective input–output map is altered online by local or structured plastic dynamics; the main exclusions are settings in which plasticity is used only to construct a fixed inference circuit prior to evaluation (Adamiat et al., 19 Dec 2025).

1. Conceptual scope and definitional boundaries

The term covers several distinct but related operational regimes. In the strongest sense, synapses or fast weights are updated while samples, tokens, or sensorimotor streams are being processed, with no separation between “learning” and “using” the model. Short-term synaptic plasticity in spiking generative networks is an instance of this strict form: the long-term weight matrix remains fixed, but instantaneous synaptic efficacy changes online and transiently during sampling, thereby reshaping the current energy landscape (Leng et al., 2017). Sequence-level fast weights in decoder-only Transformers are another direct instance: a plastic component $w_l(t)$ is initialized at the start of each sequence and updated at each step, so adaptation occurs inside the forward pass (Chaudhary, 24 Oct 2025).

A weaker, hybrid sense appears in models that retain an explicit train/test distinction within trials yet still rely on cumulative online plasticity across nonstationary operating regimes. In the reservoir spiking model for joint “what” and “when” prediction, the “when” decoder learns online within trials, the “what” decoder is consolidated offline at trial end, and cue-only evaluation disables learning during the test phase; nevertheless, the weights are carried forward across blocks, permitting adaptation after block switches without resetting the architecture (Yamada et al., 16 Oct 2025). This suggests that inference-time plasticity can be localized to one computational pathway rather than being a property of the entire model.

The boundary with adjacent notions is important. “Spike-Timing-Dependent Plasticity for Bernoulli Message Passing” uses STDP to train spiking modules that later implement sum-product updates for Bernoulli messages, but after training the “training layer” is removed and the network is used as a fixed inference circuit; this is plasticity for constructing an inference machine, not plasticity as the inference process itself (Adamiat et al., 19 Dec 2025). “A study on the plasticity of neural networks” uses the word plasticity in a different sense again: the ability of a pretrained network to remain capable of reaching a well-generalizing solution after further adaptation, with the central issue being loss of plasticity under pretraining rather than online synaptic change during inference (Berariu et al., 2021).

A further extension appears in work that treats plasticity itself as the latent object to be inferred. The STEER framework separates fast within-session neural dynamics from slow session-to-session plastic change and formalizes the latter by a latent recurrence $z_{k+1}=g_\theta(z_k,u_k)$, so that the plasticity rule becomes identifiable, interpretable, and predictive under new stimulation schedules (Liang et al., 27 Feb 2026). This broadens the topic from online parameter updates to principled inference over the law governing adaptation.

Regime	Plastic locus	Operational role
Local short-term synaptic dynamics	$W_i(t)\propto w U R$	Local reshaping of the current attractor landscape during sampling
Hybrid online/offline predictive coding	“when” readout weights	Online within-trial timing adaptation with offline “what” consolidation
Sequence-level fast weights	$W_l(t)=\tilde W_l+w_l(t)$	Rapid task-specific adaptation inside a sequence

2. Short-term synaptic dynamics in probabilistic inference

A central line of work treats inference-time plasticity as transient modification of effective synaptic strength during probabilistic sampling. In “Spiking neurons with short-term synaptic plasticity form superior generative networks,” the reference probabilistic model is a Boltzmann machine with binary states $z_k\in\{0,1\}$ and distribution

$\boldsymbol{F}^{(k)}_t$0

The spiking implementation uses conductance-based leaky integrate-and-fire neurons in the high-conductance state, where strong balanced Poisson background input makes firing probability approximately logistic. The membrane potential obeys

$\boldsymbol{F}^{(k)}_t$1

with synaptic input

$\boldsymbol{F}^{(k)}_t$2

and activation in the high-conductance state approximates

$\boldsymbol{F}^{(k)}_t$3

Within this mapping, short-term plasticity is modeled by the Tsodyks–Markram equations, with postsynaptic impact $\boldsymbol{F}^{(k)}_t$4,

$\boldsymbol{F}^{(k)}_t$5

The key claim is that the plastic variable is the instantaneous synaptic efficacy during processing, not the long-term learned weight matrix; STP therefore acts as a local, spike-triggered, transient inference-time mechanism (Leng et al., 2017).

Its functional role is formulated as a local alternative to tempering. In multimodal generative models trained on structured data, deep attractor basins and high energy barriers impair mixing. Simulated tempering, adaptive simulated tempering, and CAST address this by globally altering inverse temperature $\boldsymbol{F}^{(k)}_t$6, but they require extra computation, global state changes, and many samples that are not valid at the target temperature. STP instead weakens the attractor currently being visited by affecting only the efferent connections of simultaneously active neurons. Rather than flattening the entire energy landscape, it locally reshapes the landscape around the current state, functioning as an activity-dependent self-tempering process (Leng et al., 2017).

The quantitative results support that interpretation. In a 10-neuron target distribution, the Kullback–Leibler divergence between sampled and target distributions was minimized for STP settings around $\boldsymbol{F}^{(k)}_t$7 ms, close to the synaptic time constant $\boldsymbol{F}^{(k)}_t$8 ms, and static synapses were not optimal. On the hard bar-image dataset, the STP network could still mix whereas ordinary Gibbs sampling often remained stuck; in the easy case, both mixed, but Gibbs spent about $\boldsymbol{F}^{(k)}_t$9 times longer in the same mode before switching. On MNIST, a 3-layer hierarchical network with $\boldsymbol{W}^{(k)}$0 visible, $\boldsymbol{W}^{(k)}$1 hidden, and $\boldsymbol{W}^{(k)}$2 label units, using $\boldsymbol{W}^{(k)}$3 and $\boldsymbol{W}^{(k)}$4 ms, showed faster growth of the indirect sampling likelihood, more diverse samples after $\boldsymbol{W}^{(k)}$5 samples (about $\boldsymbol{W}^{(k)}$6 s biological time), and similar long-run ISL to Gibbs after $\boldsymbol{W}^{(k)}$7 samples. Discriminative performance changed little: Gibbs reached $\boldsymbol{W}^{(k)}$8 test accuracy and the STP spiking network $\boldsymbol{W}^{(k)}$9. Under imbalanced digit frequencies, the spiking network generated a more even class distribution over $w_l(t)$0 samples and more balanced ambiguous completions than Gibbs or AST, indicating faster access to minority modes in finite-time inference (Leng et al., 2017).

A second formalization links inference-time plasticity directly to Bayes-optimal prediction in dynamic environments. “Optimality of short-term synaptic plasticity in modelling certain dynamic environments” decomposes synaptic efficacy as

$w_l(t)$1

where $w_l(t)$2 is a resting weight and $w_l(t)$3 is a dynamic short-term component updated online by

$w_l(t)$4

The central claim is that for environments whose observations transform continuously but randomly over time, the Bayes-optimal predictor or inferencer is equivalent to short-term spike-timing-dependent plasticity. In this view, recent input pulls the synapse toward the current observation, after which the synapse relaxes back toward its resting state. On OMNIST, an SNN trained on static MNIST achieved $w_l(t)$5 on MNIST test data, dropped to $w_l(t)$6 on OMNIST without ST-STDP, and rose to $w_l(t)$7 on OMNIST with ST-STDP enabled during inference; the same setup outperformed MLP, CNN, RNN, and LSTM baselines reported for that benchmark (Moraitis et al., 2020).

3. Predictive timing, probability, and hybrid online plasticity in spiking reservoirs

Inference-time plasticity also appears as pathway-specific online readout learning. In “Joint encoding of ‘what’ and ‘when’ predictions through error-modulated plasticity in reservoir spiking networks,” the substrate is a $w_l(t)$8-neuron heterogeneous Izhikevich spiking reservoir with about $w_l(t)$9 excitatory and $\tilde W_l$0 inhibitory neurons, fixed recurrent weights, sparse Dale-consistent connectivity, double-exponential synapses, and a $\tilde W_l$1 ms integration step. The reservoir itself is not plastic; plasticity resides in the readouts. The “when” pathway uses an error-modulated, attention-gated three-factor Hebbian rule built from presynaptic state $\tilde W_l$2, signed timing error $\tilde W_l$3, an attention gate $\tilde W_l$4, and a mask $\tilde W_l$5. The “what” pathway is updated offline at trial end from the post-cue average reservoir state $\tilde W_l$6 (Yamada et al., 16 Oct 2025).

The model explicitly splits the prediction object into fast and slow learning timescales. “When” learns online within trials; “what” learns offline at trial end. The combined prediction is

$\tilde W_l$7

so identity selects the channel, timing selects the moment, and the product yields a probability-weighted, time-localized expectation. Probability is encoded in readout magnitude rather than in a separate explicit scalar module: higher probability corresponds to stronger readout amplitude near the target window, and lower probability to weaker amplitude (Yamada et al., 16 Oct 2025).

The operational significance is adaptation under nonstationarity. After a block switch, RMSE briefly rises and then rapidly reconverges; predictions relock to new timing windows and channel amplitudes shift to match new probabilities. The model rapidly adapts to timing changes such as $\tilde W_l$8 ms to $\tilde W_l$9 ms, to probability changes such as $w_l(t)$0 flipping from $w_l(t)$1 to $w_l(t)$2 or $w_l(t)$3 to $w_l(t)$4, and to combined timing/probability changes across blocks. This is contrasted with a “Gated FORCE” baseline trained by recursive least squares, which can fit stationary regimes but adapts poorly under repeated switches and may need explicit resets or forgetting. The local gated rule, by contrast, updates only synapses implicated by the active channel and time window and preserves drift-free recalibration across blocks (Yamada et al., 16 Oct 2025).

The model is not uniformly inference-time-plastic. Each trial contains a training phase with cue plus teacher signal and learning enabled, immediately followed by a cue-only testing phase with learning disabled, and the reservoir state is never reset between trials. Thus the strongest inference-time-plastic component is the timing pathway during online operation, whereas identity/probability learning remains a trial-level consolidation step (Yamada et al., 16 Oct 2025). A plausible implication is that inference-time plasticity may be computationally most useful when restricted to variables that demand rapid temporal credit assignment.

4. Continuous unsupervised adaptation and recurrent self-organization

A more fully online formulation is provided by “Time-Integrated Spike-Timing-Dependent-Plasticity,” which proposes TI-STDP as a rule for continuous unsupervised adaptation to sensory input streams. TI-STDP is defined in terms of the last presynaptic spike time $w_l(t)$5, the last postsynaptic spike time $w_l(t)$6, the current time $w_l(t)$7, and the current synaptic weight $w_l(t)$8. The post-synaptic event-based scale factor is

$w_l(t)$9

When there is a postsynaptic spike but the presynaptic neuron has been silent, the synapse decays according to

$z_{k+1}=g_\theta(z_k,u_k)$0

When there is presynaptic activity, the update becomes

$z_{k+1}=g_\theta(z_k,u_k)$1

These are explicitly continuous-time evolution laws rather than event-local increments (Gebhardt et al., 2024).

The theorem-level contribution is to establish closed-form synaptic trajectories between spike events. With both pre- and postsynaptic activity, the weight satisfies

$z_{k+1}=g_\theta(z_k,u_k)$2

with corollaries stating potentiation for $z_{k+1}=g_\theta(z_k,u_k)$3 and depression for $z_{k+1}=g_\theta(z_k,u_k)$4. With only postsynaptic activity,

$z_{k+1}=g_\theta(z_k,u_k)$5

and the corresponding corollary yields decay when $z_{k+1}=g_\theta(z_k,u_k)$6. The rule therefore preserves the causal asymmetry of STDP while integrating updates over time and without maintaining explicit pre/post traces or large spike-history windows (Gebhardt et al., 2024).

Its empirical emphasis is one-pass online learning. On MNIST, the model uses a Poisson spike encoding sensory layer, two hidden recurrent LIF layers, excitatory/inhibitory pairs with fixed local recurrent wiring, and plastic feedforward synapses. In a multi-pass regime of $z_{k+1}=g_\theta(z_k,u_k)$7 epochs and about $z_{k+1}=g_\theta(z_k,u_k)$8 image presentations, TR-STDP, EV-STDP, and TI-STDP achieved $z_{k+1}=g_\theta(z_k,u_k)$9, $W_i(t)\propto w U R$0, and $W_i(t)\propto w U R$1 accuracy, respectively. In the one-pass online regime of $W_i(t)\propto w U R$2 image presentations, TI-STDP performed best with $W_i(t)\propto w U R$3, compared with $W_i(t)\propto w U R$4 for TR-STDP and $W_i(t)\propto w U R$5 for EV-STDP. The same work also reports deeper hierarchical self-organization in a patch-based model in which $W_i(t)\propto w U R$6 patches are processed locally and assembled into part-whole representations (Gebhardt et al., 2024).

Related recurrent self-organization is analyzed in “Resonances induced by Spiking Time Dependent Plasticity.” There the network is repeatedly driven by a periodic input $W_i(t)\propto w U R$7, and weights evolve under STDP plus homeostatic normalization. The crucial equilibrium relation is

$W_i(t)\propto w U R$8

together with the self-consistency equations

$W_i(t)\propto w U R$9

For sparse peaked stimuli, the learned structure becomes phase-organized and feed-forward, increasing signal-to-noise ratio and sharpening temporal responses. This is an inference-time-plasticity setting in the sense that synapses change during ongoing exposure to the same stimulus, improving representation without labels or offline retraining (Aceituno, 2020).

5. Dynamical-systems, information-theoretic, and latent-rule formulations

Inference-time plasticity can also be formulated as coupled neural–synaptic dynamics whose computational role depends on timescale separation. “Plasticity-induced multistability on fast and slow timescales enables optimal information encoding and spontaneous sequence discrimination” studies a stochastic rate model with excitatory and inhibitory population activities $W_l(t)=\tilde W_l+w_l(t)$0, time-varying stimulus $W_l(t)=\tilde W_l+w_l(t)$1, and a plasticity-controlled gain parameter $W_l(t)=\tilde W_l+w_l(t)$2. Neural dynamics obey

$W_l(t)=\tilde W_l+w_l(t)$3

with

$W_l(t)=\tilde W_l+w_l(t)$4

Here $W_l(t)=\tilde W_l+w_l(t)$5 is Hebbian and $W_l(t)=\tilde W_l+w_l(t)$6 anti-Hebbian. In the slow-plasticity regime $W_l(t)=\tilde W_l+w_l(t)$7, plasticity acts as a global modulator steering the system toward a mutual-information optimum. In the fast-plasticity regime $W_l(t)=\tilde W_l+w_l(t)$8, plasticity becomes stimulus-conditioned and can generate multistability, so that sequence order is encoded in the synaptic trajectory itself (Barzon et al., 17 Sep 2025).

The information-theoretic analysis is explicit. For slow plasticity, mutual information between neural state and input is

$W_l(t)=\tilde W_l+w_l(t)$9

with analytic lower and upper bounds derived from pairwise divergences of Gaussian mixture components. The information-maximizing operating point is summarized by

$z_k\in\{0,1\}$0

The phase diagram shows that either Hebbian or anti-Hebbian plasticity may become optimal depending on input projection $z_k\in\{0,1\}$1. In the fast regime, input-conditioned fixed points satisfy $z_k\in\{0,1\}$2, and sequence information

$z_k\in\{0,1\}$3

is nonzero only when the sequence traverses a multistable region. The same work identifies an optimal variability $z_k\in\{0,1\}$4 for sequence discrimination at a given plasticity strength (Barzon et al., 17 Sep 2025).

A complementary perspective treats plasticity not as an update rule chosen a priori but as a latent dynamical law inferred from longitudinal data. STEER separates within-session activity from across-session plastic change by representing session-specific recurrent connectivity as

$z_k\in\{0,1\}$5

with low-dimensional plasticity embedding

$z_k\in\{0,1\}$6

and slow recurrence

$z_k\in\{0,1\}$7

The model is trained with a composite loss containing within-session prediction, slow-rule consistency, and smoothness terms, and can be rolled forward under counterfactual stimulation schedules by iterating $z_k\in\{0,1\}$8 on latent plasticity states (Liang et al., 27 Feb 2026).

Its benchmarks establish that long-term plasticity can be treated as an identifiable dynamical object rather than as unconstrained parameter drift. On synthetic Lorenz systems, BCM-based networks, stimulation-induced task learning, and longitudinal Parkinsonian rat DBS recordings, STEER is reported to recover interpretable update equations, predict network adaptation under unseen stimulation schedules, and support improved intervention design (Liang et al., 27 Feb 2026). This suggests a broader interpretation of inference-time plasticity: not only synapses changing during processing, but also formal inference over the rule by which repeated stimulation will continue to reshape the circuit.

6. Fast weights in Transformers and real-time hardware plasticity

The topic is not confined to spiking models. “Enabling Robust In-Context Memory and Rapid Task Adaptation in Transformers with Hebbian and Gradient-Based Plasticity” augments decoder-only Transformers with fast-weight modules in selected feed-forward layers: $z_k\in\{0,1\}$9 where $\boldsymbol{F}^{(k)}_t$00 are static meta-trained weights and $\boldsymbol{F}^{(k)}_t$01 are sequence-level fast weights initialized to zero at the start of each sequence. The Hebbian rule is

$\boldsymbol{F}^{(k)}_t$02

with learnable plasticity coefficients $\boldsymbol{F}^{(k)}_t$03 and neuromodulatory gate

$\boldsymbol{F}^{(k)}_t$04

The gradient-based variant instead updates fast weights and biases using local gradients of an internally generated auxiliary loss $\boldsymbol{F}^{(k)}_t$05 (Chaudhary, 24 Oct 2025).

The paper’s central distinction is between associative memory and online optimization. Hebbian plasticity writes correlations between presynaptic and postsynaptic activity into transient weights, whereas gradient-based plasticity performs a local gradient descent step on a self-generated objective. Both are trained in an outer loop while operating in an inner loop during the sequence. The reported results are task-dependent. On copying, gradient plasticity achieved loss $\boldsymbol{F}^{(k)}_t$06 and recall $\boldsymbol{F}^{(k)}_t$07, Hebbian plasticity loss $\boldsymbol{F}^{(k)}_t$08 and recall $\boldsymbol{F}^{(k)}_t$09, and the non-plastic baseline loss $\boldsymbol{F}^{(k)}_t$10 and recall $\boldsymbol{F}^{(k)}_t$11. On few-shot regression, gradient and Hebbian plasticity achieved MSE $\boldsymbol{F}^{(k)}_t$12 and $\boldsymbol{F}^{(k)}_t$13, versus $\boldsymbol{F}^{(k)}_t$14 without plasticity. On 5-way 1-shot CIFAR-FS and Omniglot classification, the table values are $\boldsymbol{F}^{(k)}_t$15, $\boldsymbol{F}^{(k)}_t$16, and $\boldsymbol{F}^{(k)}_t$17 for CIFAR-FS, and $\boldsymbol{F}^{(k)}_t$18, $\boldsymbol{F}^{(k)}_t$19, and $\boldsymbol{F}^{(k)}_t$20 for Omniglot, for gradient, Hebbian, and non-plastic variants respectively. By contrast, on cue–reward association the static Transformer performed best, with validation/query losses $\boldsymbol{F}^{(k)}_t$21 and $\boldsymbol{F}^{(k)}_t$22, compared with $\boldsymbol{F}^{(k)}_t$23 and $\boldsymbol{F}^{(k)}_t$24 for gradient plasticity and $\boldsymbol{F}^{(k)}_t$25 and $\boldsymbol{F}^{(k)}_t$26 for Hebbian plasticity (Chaudhary, 24 Oct 2025).

The mechanistic analysis concerns the learned modulatory signal. Gradient plasticity maintains relatively large, persistent $\boldsymbol{F}^{(k)}_t$27, while Hebbian plasticity is sharply gated around salient events and near zero otherwise. Reported mean values include $\boldsymbol{F}^{(k)}_t$28 for gradient plasticity and $\boldsymbol{F}^{(k)}_t$29 for Hebbian plasticity on copying, and approximately $\boldsymbol{F}^{(k)}_t$30 and $\boldsymbol{F}^{(k)}_t$31 on CIFAR-FS. The same work reports that extending the copying task to $\boldsymbol{F}^{(k)}_t$32 layers makes gradient plasticity unstable, with divergence after about $\boldsymbol{F}^{(k)}_t$33 steps and plastic norms exploding above $\boldsymbol{F}^{(k)}_t$34, whereas Hebbian plasticity remains stable but saturates (Chaudhary, 24 Oct 2025).

A hardware-oriented realization appears in “FireFly-P: FPGA-Accelerated Spiking Neural Network Plasticity for Robust Adaptive Control.” There the plasticity rule itself is optimized offline and then executed online on FPGA. The learned local rule combines associative, presynaptic, postsynaptic, and decay terms, with exponentially decaying spike traces

$\boldsymbol{F}^{(k)}_t$35

The architecture uses a Forward Engine for SNN inference, a Plasticity Engine for online weight updates, a Scheduler, and BRAM-based state storage. Neuron dynamics are implemented as

$\boldsymbol{F}^{(k)}_t$36

with $\boldsymbol{F}^{(k)}_t$37, FP16 arithmetic, and pipelined overlap of inference and plasticity so that the forward engine sees the latest weights without double buffering (Li et al., 29 Jan 2026).

The reported hardware metrics establish that inference-time plasticity can be implemented as a deterministic real-time pipeline. On a Xilinx Artix-7 Cmod A7-35T synthesized at $\boldsymbol{F}^{(k)}_t$38 MHz with $\boldsymbol{F}^{(k)}_t$39 processing elements, the design achieves an end-to-end latency of $\boldsymbol{F}^{(k)}_t$40s for both inference and plasticity updates, consumes $\boldsymbol{F}^{(k)}_t$41 W, and uses about $\boldsymbol{F}^{(k)}_t$42k LUTs, $\boldsymbol{F}^{(k)}_t$43k registers, $\boldsymbol{F}^{(k)}_t$44 BRAMs, and $\boldsymbol{F}^{(k)}_t$45 DSPs. For MNIST with a $\boldsymbol{F}^{(k)}_t$46-$\boldsymbol{F}^{(k)}_t$47-$\boldsymbol{F}^{(k)}_t$48 SNN, it achieves $\boldsymbol{F}^{(k)}_t$49 accuracy and sustains $\boldsymbol{F}^{(k)}_t$50 FPS end-to-end at $\boldsymbol{F}^{(k)}_t$51 MHz, with the reported FPS including forward and learning stages pipelined together. In Brax continuous-control benchmarks on ant, half cheetah, and ur5e, the authors report faster adaptation and higher performance than SNNs with directly trained fixed synaptic weights (Li et al., 29 Jan 2026).

7. Adjacent concepts, limitations, and recurrent misconceptions

A common misconception is to treat any use of plasticity in an inference model as inference-time plasticity. “Spike-Timing-Dependent Plasticity for Bernoulli Message Passing” is an explicit counterexample. It implements Bayesian inference on Forney-style factor graphs by training LIF modules with STDP to approximate Bernoulli-message updates for AND, OR, NOT, XOR, and equality factors, and the resulting outputs closely match analytical sum-product solutions. In a coding-theory example with binary symmetric channel cross-over probability $\boldsymbol{F}^{(k)}_t$52, the learned SNN approximates several exact messages closely, such as $\boldsymbol{F}^{(k)}_t$53 with SP $\boldsymbol{F}^{(k)}_t$54 versus SNN $\boldsymbol{F}^{(k)}_t$55 and $\boldsymbol{F}^{(k)}_t$56 with SP $\boldsymbol{F}^{(k)}_t$57 versus SNN $\boldsymbol{F}^{(k)}_t$58. Yet the paper is explicit that after training the network is run as a fixed inference circuit, so it is better understood as a precursor or adjacent work than as a direct example of online inference-time adaptation (Adamiat et al., 19 Dec 2025).

A second misconception is to conflate inference-time plasticity with the broader question of whether pretrained networks remain plastic. “A study on the plasticity of neural networks” defines plasticity as the ability to keep learning without degrading final generalization after pretraining. In its ResNet-18 experiment, pretraining on half of CIFAR-10 and then fine-tuning on the full training set yields worse test accuracy than training from scratch, even though both reach $\boldsymbol{F}^{(k)}_t$59 training accuracy. The gap appears across Adam, RMSprop, SGD, and SGD with momentum; it can emerge after only $\boldsymbol{F}^{(k)}_t$60–$\boldsymbol{F}^{(k)}_t$61 epochs of pretraining; it persists under smooth distribution shift and with larger architectures; and it is reduced by a $\boldsymbol{F}^{(k)}_t$62 larger learning rate or by reinitializing the top part of the model (Berariu et al., 2021). This is not inference-time plasticity in the usual online-update sense, but it is relevant because repeated adaptation can consume the capacity for future adaptation.

The literature also contains broader or less standardized probabilistic formalisms. “Application and Computation of Probabilistic Neural Plasticity” proposes an additive short-term memory equation

$\boldsymbol{F}^{(k)}_t$63

defines a degree of neural plasticity as

$\boldsymbol{F}^{(k)}_t$64

and uses $\boldsymbol{F}^{(k)}_t$65 as the central inferential quantity. The same source notes that several Bayesian expressions are nonstandard or internally inconsistent. It is therefore best read as a conceptually related attempt to formalize input-conditioned plastic change, rather than as a settled mathematical foundation for inference-time plasticity (Hossain, 2019).

The principal limitations reported across direct inference-time-plasticity models are heterogeneous. Short-term plasticity in spiking samplers primarily improves finite-time inference and does not preserve exact equivalence to the static Gibbs distribution over short runs, although discrepancies diminish for very long sampling times (Leng et al., 2017). The reservoir prediction model is only partially online-plastic because the “what” pathway is consolidated offline and cue-only test evaluation disables learning (Yamada et al., 16 Oct 2025). FireFly-P relies on a simple local rule, careful scheduling, BRAM management, and mostly simulated robotics benchmarks, with the authors noting that more complex tasks may require richer plasticity and larger memories (Li et al., 29 Jan 2026). Plastic Transformers exhibit task-dependent benefits, clear boundary conditions where static weights suffice, and stability problems for deeper gradient-plastic stacks (Chaudhary, 24 Oct 2025). TI-STDP is motivated by online adaptation, but its classification evaluation disables plasticity at test time after unsupervised learning and label binding, so the reported gains primarily reflect better streaming representation learning rather than continual label-conditioned online correction (Gebhardt et al., 2024).

Taken together, these works define inference-time plasticity not as a single mechanism but as a family of adaptive computation regimes. In some cases it is a local transient reshaping of an energy landscape; in others it is a Bayesian tracking rule, a continuous STDP process, a fast-weight inner loop, a real-time hardware update engine, or an inferred slow law of circuit reorganization. What unifies them is that the adaptive state is computationally active during operation, and the consequences are measured in mixing speed, robustness to nonstationarity, sequence discrimination, probability-weighted prediction, or rapid task adaptation rather than in offline training loss alone.