Bayesian Metaplasticity Overview

• Bayesian metaplasticity is a framework where synapses store both efficacy and uncertainty, using probabilistic inference to gate plasticity.
• The approach employs online Bayesian updates, variational Bayes, and posterior sampling to balance learning with controlled forgetting in dynamic environments.
• It underpins continual learning algorithms, demonstrating improved performance on benchmarks like permuted MNIST and real-time edge computing scenarios.

Bayesian metaplasticity is a family of theories and algorithms in which the variable that regulates plasticity is itself represented probabilistically, typically as posterior variance, entropy, or a latent continuous field, and updated online by Bayesian inference, variational Bayes, or posterior sampling. In this formulation, a synapse does not store only an efficacy estimate; it also stores a measure of confidence, and that confidence modulates how readily the synapse changes. The framework spans normative theories of synaptic learning in neuroscience, fully Bayesian state-space models for plasticity from spike trains, variational continual learning in binary neural networks, and bounded-memory methods for non-stationary streams in edge settings (Aitchison et al., 2014, Kappel et al., 2015, Linderman et al., 2014, Li et al., 2022, Bonnet et al., 18 Apr 2025, Cottart et al., 28 May 2026).

1. Conceptual definition and lineage

Metaplasticity denotes a higher-order regulation of plasticity: the “state of the synapse” controls its susceptibility to future change. In Bayesian metaplasticity, that state is not an auxiliary heuristic variable but a probabilistic object. A synapse maintains either a posterior over its weight or a latent parameterization from which a posterior is induced, and the uncertainty of that posterior becomes the metaplastic resource that gates subsequent updates. In the formulation of Aitchison and colleagues, each synapse carries both an efficacy estimate and “error bars,” and the effective learning rate increases with posterior variance; synaptic sampling then identifies trial-to-trial PSP variability with posterior uncertainty (Aitchison et al., 2014).

This basic idea appears in several mathematically distinct traditions. In synaptic-sampling models, network plasticity is treated as posterior sampling over latent synaptic parameters $\theta$, with Langevin dynamics combining likelihood gradients, prior gradients, and diffusion noise. The stationary distribution is

$p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$

so ongoing stochasticity is functional rather than incidental (Kappel et al., 2015). In variational continual learning for binary networks, metaplasticity is implemented by training factorized Bernoulli posteriors in “field space,” so that certainty reduces the gradient magnitude and consolidated synapses become resistant to task interference (Li et al., 2022). In more recent continual-learning work, the same principle is adapted to Gaussian or Bernoulli posteriors with explicit forgetting terms that prevent uncertainty collapse in long streams (Bonnet et al., 18 Apr 2025, Cottart et al., 28 May 2026).

A common misconception is that Bayesian metaplasticity is merely a probabilistic restatement of adaptive learning rates. The literature is more specific. The metaplastic controller is derived from an explicit probabilistic model—posterior variance in Gaussian families, entropy or $\cosh^{-2}(\lambda)$ in Bernoulli families, or a latent state-space prior in neural spike-train models—rather than being inserted as an optimizer heuristic (Aitchison et al., 2014, Li et al., 2022).

2. Probabilistic state representations

Across the literature, the metaplastic state is instantiated through different posterior families and latent-variable parameterizations.

Framework	Synaptic state	Plasticity control
"Synaptic plasticity as Bayesian inference" (Aitchison et al., 2014)	Gaussian posterior over log-weight $z_{\mathrm{tar},i}$ with mean $m_i$ and variance $s_i^2$	Effective learning rate scales with $s_i^2$; PSP variability reflects uncertainty
"Network Plasticity as Bayesian Inference" (Kappel et al., 2015)	Continuous $\theta_i$ encoding connectivity and efficacy	Langevin drift from $\log p_S(\theta)+\log p_N(x\mid\theta)$ plus diffusion samples the posterior
"Statistical mechanics of continual learning: variational principle and mean-field potential" (Li et al., 2022)	Bernoulli $$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$	Gradient is gated by $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$0
"Bayesian continual learning and forgetting in neural networks" (Bonnet et al., 18 Apr 2025)	Mean-field Gaussian $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$1	$p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$2 scales both learning and forgetting
"Active Continual Learning with Metaplastic Binary Bayesian Neural Networks" (Cottart et al., 28 May 2026)	Mean-field Bernoulli $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$3 with natural parameter $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$4	Uncertainty-gated relaxation and a bounded metaplastic step size prevent saturation

Despite these formal differences, the shared mechanism is direct: uncertainty modulates update amplitude. In the field-space Bernoulli formulation,

$p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$5

and the continual-learning gradient takes the form

$p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$6

so the same quantity $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$7 simultaneously gates plasticity and regularization toward the previous posterior (Li et al., 2022). In Gaussian MESU, the mean update is preconditioned by $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$8, and the uncertainty update itself is regularized toward a prior scale, which yields an online approximation to diagonal Newton steps without explicit Hessian construction (Bonnet et al., 2023). In BiMU, the Bernoulli family is parameterized by natural parameters $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$9, and uncertainty is tied to

$\cosh^{-2}(\lambda)$0

which becomes central to preventing posterior degeneracy on long streams (Cottart et al., 28 May 2026).

This suggests that “Bayesian metaplasticity” is better understood as a design principle than as a single algorithm: the metaplastic state may be variance, entropy, precision, or latent field uncertainty, but it must be updated by the same probabilistic machinery that governs learning.

3. Continual learning, bounded memory, and controlled forgetting

A central theme in the modern literature is that posterior accumulation alone is insufficient for continual learning. In field-space variational continual learning, the posterior from the previous task becomes the prior for the next task, and the negative ELBO takes the form

$\cosh^{-2}(\lambda)$1

For binary networks, this yields a posterior-as-prior scheme in which certain synapses become progressively less plastic because their variance shrinks, thereby preserving old knowledge (Li et al., 2022). The same work shows that the KL term reduces locally to an EWC-like quadratic penalty with diagonal Fisher $\cosh^{-2}(\lambda)$2, making the EWC connection explicit.

MESU modifies this standard recursive Bayes picture by introducing a truncated posterior over a finite memory window. The learning–forgetting balance is written through a variational free energy

$\cosh^{-2}(\lambda)$3

The resulting updates,

$\cosh^{-2}(\lambda)$4

$\cosh^{-2}(\lambda)$5

use uncertainty to scale both adaptation and relaxation toward the prior. In the i.i.d. analysis, the uncertainty dynamics admit a nonzero steady state

$\cosh^{-2}(\lambda)$6

whereas $\cosh^{-2}(\lambda)$7 causes $\cosh^{-2}(\lambda)$8, producing overconfidence and “catastrophic remembering” (Bonnet et al., 18 Apr 2025).

BiMU addresses the same problem in mean-field Bernoulli BNNs, where long streams can drive $\cosh^{-2}(\lambda)$9 monotonically upward. Since

$z_{\mathrm{tar},i}$0

large $z_{\mathrm{tar},i}$1 makes $z_{\mathrm{tar},i}$2 nearly deterministic, and

$z_{\mathrm{tar},i}$3

The paper interprets this as posterior saturation: epistemic uncertainty vanishes, samples stop disagreeing, and synapses effectively freeze. BiMU derives a bounded-memory objective in which plasticity, stability, and forgetting are separated into a data term, a KL-to-previous term with weight $z_{\mathrm{tar},i}$4, and a KL-to-prior term with weight $z_{\mathrm{tar},i}$5. The update then combines a prior relaxation proportional to $z_{\mathrm{tar},i}$6 with a bounded, asymmetric step size that favors consolidation when gradients reinforce the current sign and damps abrupt de-consolidation unless gradients persist (Cottart et al., 28 May 2026).

A recurring conclusion across these frameworks is that forgetting is not an external correction layered on top of Bayesian learning. It is part of the probabilistic definition of the effective posterior when the environment is non-stationary or memory is deliberately bounded.

4. Algorithmic realizations in modern neural networks

The MESU line of work casts Bayesian metaplasticity as an online optimizer for continual learning. In the 2023 formulation, the key identity

$z_{\mathrm{tar},i}$7

connects uncertainty to curvature, and the updates

$z_{\mathrm{tar},i}$8

yield an asymptotic diagonal-Newton preconditioner with damping $z_{\mathrm{tar},i}$9. On permuted MNIST without task boundaries, MESU maintains learning across 100 tasks, while the ablation without the $m_i$0-regularizer exhibits vanishing plasticity; after 100 tasks, BGD suffers an $m_i$1 accuracy drop, whereas MESU learns the last 10 tasks nearly as well as the first 10 (Bonnet et al., 2023).

The 2025 extension broadens this framework to streaming Gaussian BNNs with explicit bounded forgetting. On 200 sequential permuted MNIST tasks, the average accuracy on the last 5 tasks after 200 tasks is reported as MESU $m_i$2, compared with EWC Online $m_i$3 and SI $m_i$4, while FOO-VB Diagonal plateaus around $m_i$5 because vanishing $m_i$6 produces catastrophic remembering. On the Animals benchmark, after learning three tasks, BNN+MESU retains $m_i$7 on Task 1 and generalizes $m_i$8 to unseen Task 4, whereas the SGD baseline drops to $m_i$9 and $s_i^2$0; MESU’s epistemic uncertainty also separates whales as OOD data from the in-distribution stream (Bonnet et al., 18 Apr 2025).

BiMU transfers Bayesian metaplasticity to binary networks and edge-like online settings. On 1000-tasks Permuted-MNIST, in an online no-replay batch-size-1 regime with a compact MLP of 100 hidden units, BiMU reaches mean accuracy over the last five tasks of $s_i^2$1, compared with BayesBiNN $s_i^2$2, Synaptic Metaplasticity $s_i^2$3, and STE $s_i^2$4, while also achieving OOD AUC $s_i^2$5. On OpenLORIS-Object, using frozen ImageNet VGG19 features and a linear head, BiMU reports $s_i^2$6 at 1,024 dimensions, $s_i^2$7 at 8,192, and $s_i^2$8 at 25,088; epistemic OOD AUC is $s_i^2$9 at 1,024 and 8,192 dimensions and $s_i^2$0 at 25,088 (Cottart et al., 28 May 2026).

The same non-degenerate posterior supports one-pass active learning. BiMU samples $s_i^2$1 binary networks, computes disagreement scores such as Variation Ratio,

$s_i^2$2

and queries labels only when the disagreement exceeds a threshold. On imbalanced Animals, VR attains $s_i^2$3 accuracy at $s_i^2$4 labels and $s_i^2$5 at $s_i^2$6 labels, slightly above the $s_i^2$7-update baseline of $s_i^2$8. On OpenLORIS-Object at 8,192 dimensions, BiMU with VR reaches $s_i^2$9 accuracy while updating on only $\theta_i$0 of samples, approximately $\theta_i$1 fewer labels and updates, and $\theta_i$2 at $\theta_i$3, approximately $\theta_i$4 fewer labels and updates (Cottart et al., 28 May 2026).

These results are accompanied by systems-level claims. BiMU stores only the current Bernoulli natural parameters and no buffers; in the PMNIST setup the training-state overhead is $\theta_i$5 MB, compared with $\theta_i$6 MB for BayesBiNN. On an STM32 NUCLEO-64 at 216 MHz, binary inference is approximately $\theta_i$7 cheaper than training, and in a projected OpenLORIS $\theta_i$8 linear head, BiMU with VR and $\theta_i$9 at $\log p_S(\theta)+\log p_N(x\mid\theta)$0 queried samples yields an expected per-sample compute of $\log p_S(\theta)+\log p_N(x\mid\theta)$1 ms versus $\log p_S(\theta)+\log p_N(x\mid\theta)$2 ms for full BiMU updates, with near-matched accuracy $\log p_S(\theta)+\log p_N(x\mid\theta)$3 versus $\log p_S(\theta)+\log p_N(x\mid\theta)$4 (Cottart et al., 28 May 2026).

5. Biological interpretation and data-driven inference

In the biological literature, Bayesian metaplasticity is motivated as a normative account of how synapses should learn under noise and non-stationarity. In the linear-feedback model of Aitchison and colleagues, the synapse maintains a Gaussian posterior over a log-weight $\log p_S(\theta)+\log p_N(x\mid\theta)$5 with mean $\log p_S(\theta)+\log p_N(x\mid\theta)$6 and variance $\log p_S(\theta)+\log p_N(x\mid\theta)$7, and the online assumed-density updates are

$\log p_S(\theta)+\log p_N(x\mid\theta)$8

$\log p_S(\theta)+\log p_N(x\mid\theta)$9

Here the effective learning rate is proportional to $$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$0, so larger uncertainty implies larger plasticity. The synaptic-sampling hypothesis then posits

$$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$1

which yields the experimentally testable relation

$$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$2

In reanalyzed mouse V1 L2/3 data, the observed log–log slope between normalized variability and firing rate is $$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$3, statistically indistinguishable from $$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$4 (Aitchison et al., 2014).

Kappel and colleagues extend the Bayesian view from individual synapses to network plasticity and structural plasticity. Synaptic parameters $$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$5 jointly encode connectivity and efficacy, with negative $$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$6 corresponding to non-functional contacts and positive $$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$7 to functional synapses. Under the exponential map

$$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$8

a Gaussian prior over $$q(w_i\mid \theta_i)=e^{\beta\theta_i w_i}/(2\cosh(\beta\theta_i))$$9 induces log-normal weights, and the online dynamics

$p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$00

implement posterior sampling rather than point estimation. This framework predicts log-normal weight distributions, power-law survival of newly formed synapses, ongoing drift in synaptic states, and self-repair after perturbations because noise continuously explores new posterior modes (Kappel et al., 2015).

A separate line of work asks how metaplasticity can be inferred directly from population spike trains. In a fully Bayesian GLM with time-varying synaptic weights, the conditional intensity for neuron $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$01 is

$p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$02

and the weight dynamics are governed by a learning rule embedded as a latent state-space model. The framework explicitly allows a metaplastic latent state $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$03 with dynamics

$p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$04

which modulates plasticity parameters over time. Inference is performed with particle MCMC and conditional SMC. On synthetic NEURON data, posterior edge probabilities yield ROC AUC $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$05, and the method recovers weight trajectories and underlying learning rules. At the same time, it identifies a principled failure mode: when weights saturate near $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$06, additive and multiplicative STDP become difficult to distinguish from predictive likelihood alone (Linderman et al., 2014).

Taken together, these studies locate Bayesian metaplasticity at the interface of normative theory and experimental analysis: it is both a proposal about what synapses should compute and a latent-variable framework for testing plasticity hypotheses against data.

6. Relations to adjacent methods, misconceptions, and open problems

Bayesian metaplasticity is closely related to curvature-based continual-learning methods, but the relation is neither purely terminological nor exact. In the binary variational framework, a local quadratic expansion makes the KL term reduce to an EWC-like penalty with $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$07, so synaptic uncertainty supplies a Bayesian interpretation of elastic consolidation (Li et al., 2022). In MESU, the online uncertainty dynamics induce a Hessian-weighted stabilizer automatically through $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$08, but forgetting prevents these importance weights from diverging indefinitely (Bonnet et al., 18 Apr 2025). This differs from methods that accumulate penalties monotonically across tasks.

Another misconception is that Bayesian metaplasticity is equivalent to “being more Bayesian” in the sense of indefinitely accumulating posterior evidence. Recent continual-learning results argue the opposite. Standard recursive Bayes in streaming settings can over-constrain the posterior, collapse uncertainty, and eliminate the epistemic signal needed both for adaptation and for OOD detection. BiMU identifies this explicitly as posterior saturation in Bernoulli BNNs, while MESU characterizes the analogous Gaussian failure mode as catastrophic remembering (Bonnet et al., 18 Apr 2025, Cottart et al., 28 May 2026).

The main open problems are methodological rather than conceptual. Mean-field and diagonal posteriors ignore inter-parameter correlations, so richer posterior families remain a natural extension (Bonnet et al., 18 Apr 2025). Replica-symmetric and Gaussian-field approximations may fail at very large $p^*(\theta)=Z^{-1}p(\theta\mid x)^{1/T},$09, and finite-size SGD can become trapped in suboptimal basins, especially outside the shallow or large-width regimes where theory is most explicit (Li et al., 2022). Binary weights limit representational capacity and may require architectural adaptations such as Reverse Binary Gate when scaling to larger models and datasets (Cottart et al., 28 May 2026). In biological settings, the physical substrate that stores and exposes synaptic uncertainty remains incompletely specified, even when PSP variability offers an empirical proxy (Aitchison et al., 2014). Likewise, posterior-sampling accounts rely on local likelihood gradients and biologically interpretable noise processes whose exact implementation may vary by circuit (Kappel et al., 2015).

There are also unresolved issues specific to non-stationary streams. The bounded-memory formulations assume ongoing drift; abrupt domain shifts or label-function changes may require adaptive thresholds or budget controllers, and equal-likelihood assumptions in forgetting surrogates may be violated when task difficulty varies sharply (Bonnet et al., 18 Apr 2025, Cottart et al., 28 May 2026). This suggests that the next stage of the field will likely combine Bayesian metaplasticity with richer priors, structured posteriors, or explicit data-selection mechanisms rather than treating uncertainty-gated plasticity as a complete solution on its own.

In the literature to date, Bayesian metaplasticity is best understood as a unifying principle: plasticity, consolidation, and forgetting are controlled by probabilistic synaptic state, and that state is updated continuously from evidence. The principle has yielded local update rules, thermodynamic analyses, online variational algorithms, spike-train inference procedures, active querying schemes, and strong OOD behavior in long non-stationary streams, while also clarifying that preserving uncertainty is not ancillary to continual learning but one of its central computational requirements (Aitchison et al., 2014, Linderman et al., 2014, Li et al., 2022, Bonnet et al., 18 Apr 2025, Cottart et al., 28 May 2026).