---
title: Parametric Memory Law Overview
url: https://www.emergentmind.com/topics/parametric-memory-law
type: topic
---

# Parametric Memory Law Overview

Parametric memory law denotes a memory rule whose present effect is governed by explicit parameterization rather than by an unstructured dependence on the past. Across the cited literatures, the term has several technically distinct realizations. In fractional dynamics, it refers to power-law kernels indexed by a real order parameter, typically \(\alpha\), so that past states are weighted by \((t-t_i)^{\alpha-1}\) or \((n-k+1)^{\alpha-1}\). In contemporary machine learning, it refers to knowledge stored in parameters or auxiliary parametric modules, together with laws that quantify scaling, gating, or selective consolidation of that memory. In economics, human recall, memristive circuits, and quantum associative memory, the same expression denotes explicit functional rules linking memory strength, storage, or retrieval to tunable parameters or latent state variables [1306.6361][2601.22887][1905.02403].

## 1. Conceptual scope and terminological variants

A parametric memory law is not a single theorem shared by all fields; rather, it is a family of constructions in which the contribution of past information is modulated by one or more explicit parameters. In nonlinear fractional maps, the parameter is usually the fractional order \(\alpha\), and the law takes the form of a long-range kernel \(w_\alpha(n-k)\propto(n-k+1)^{\alpha-1}\). In that setting, changing \(\alpha\) changes the strength, range, and effective dimensionality of memory [1612.01174].

In parametric neural memory, the same phrase shifts meaning. The memory is the information stored in learned parameters, and the law is a quantitative relation or update rule governing how that storage scales or is rewritten. Examples include the power law \(\Delta \mathcal{L}(r,\ell)= C r^\alpha \ell^{-\beta}+b\) for LoRA-based exact memorization, and memory-only update rules in which a sparse subset of parameters is selected and modified while the backbone is frozen [2605.30260].

A further distinction, emphasized in long-running agent work, separates **memory access** from **memory depth**. Retrieval systems provide access to stored facts at inference time, whereas deep parametric memory supplies durable goal-conditioned tendencies that persist after working context is unloaded. This suggests that “parametric memory law” in agentic systems often denotes a rule for *which* experiences are written into parameters and *how strongly* they are consolidated [2606.26806].

## 2. Fractional dynamics and power-law kernels

The most classical use of the term arises in systems with power-law memory described by fractional calculus. In this literature, the law is explicit: the weight of a past state at time \(t_i\) in defining the present state at time \(t\) is proportional to \((t-t_i)^{\alpha-1}\). For kicked nonlinear systems, this yields \(\alpha\)-families of maps in which the next state depends on all previous states, for example
\[
x_{n+1}=x_0-\frac{1}{\Gamma(\alpha)}\sum_{k=0}^n G_K(x_k)(n-k+1)^{\alpha-1},
\]
with analogous Riemann–Liouville and Caputo forms. Integer \(\alpha\) recovers familiar finite-dimensional maps, while non-integer \(\alpha\) gives genuine long-range power-law memory [1306.6361].

The continuous-time origin of these maps is the fractional derivative, represented by a Volterra-type kernel. In Caputo form,
\[
{}_0^C D_t^\alpha x(t)=\frac{1}{\Gamma(n-\alpha)}\int_0^t \frac{D_\tau^n x(\tau)\,d\tau}{(t-\tau)^{\alpha-n+1}},
\]
so the operator is itself a memory law. Discretization yields long-term memory maps, fractional universal maps, and fractional difference maps with either exact power-law weights or asymptotically power-law falling-factorial weights. In all cases, \(\alpha\) is the memory parameter controlling how slowly the kernel decays [1612.01174].

Varying \(\alpha\) changes both local stability and global bifurcation structure. The resulting phase spaces may contain periodic sinks, attracting slow-diverging trajectories, attracting accelerator mode trajectories, chaotic attractors, and cascade of bifurcations type trajectories (CBTT). In these systems, universality persists, but with new features: critical values of \(K\) become functions of \(\alpha\), convergence is power-law rather than exponential, and period-doubling can occur along a single trajectory in time rather than only under parameter sweeps [1709.00093].

## 3. Physical, biological, economic, and recall formulations

In intersectoral dynamics, the parametric memory law is implemented by replacing the standard accelerator \(I(t)=B\,dX(t)/dt\) with a Caputo derivative:
\[
I(t)=B\,(D_{0+}^\alpha X)(t).
\]
For \(0<\alpha<1\), this introduces a fading kernel \((t-\tau)^{-\alpha}\); for \(\alpha=1\), the classical memoryless model is recovered; and for \(1<\alpha<2\), the model acquires additional inertial effects. The resulting closed and open Leontief-type systems have solutions in terms of Mittag–Leffler functions, and effective growth rates are reparametrized as \(\lambda_{k,\mathrm{eff}}(\alpha)=\lambda_k^{1/\alpha}\) [1712.09087].

Human free recall yields a different but equally explicit law. In the associative-search model of random lists, the average number of recalled items \(R\) is related to the number of items effectively stored in memory \(M\) by the parameter-free formula
\[
R=\sqrt{\frac{3\pi}{2}\,M}.
\]
Here the law is not a tunable kernel but a fixed scaling relation between storage and recall capacity. The prefactor \(\sqrt{3\pi/2}\) is derived rather than fitted, and the model predicts that recall grows sublinearly as \(\sqrt{M}\), separating acquisition from retrieval dynamics [1905.02403].

In purely memristive circuits, the law governs internal state variables \(w_i(t)\) under topological constraints. The network-level evolution can be written as
\[
\frac{d \vec W}{dt}=\alpha \vec W-\frac{1}{\beta}\bigl(I+\xi\,\bar\Omega W\bigr)^{-1}\bar\Omega\,\bar S,
\]
where \(\xi=(R_{off}/R_{on})-1\), \(\bar\Omega\) is a topology-dependent projector, and only \(M-N+1\) memory degrees of freedom are independent. In the weakly nonlinear regime, the dynamics becomes a constrained gradient descent; in the strongly nonlinear regime, a conservation law for squared memory variables appears [1611.02104].

A quantum associative version appears in parametric probabilistic quantum memory. There the parameter is a phase-scale factor \(t\), and the retrieval law for binary patterns becomes
\[
P_t(c=0\mid x)=\frac{1}{r}\sum_{p=1}^r \cos^2\!\left(\frac{\pi}{2nt}\,d_H(x,x^{(p)})\right).
\]
The parameter \(t\) rescales the Hamming-distance phase and thereby sharpens or softens discrimination. In this formulation, the law is a distance-to-probability mapping governed by a single continuous control variable [2001.04798].

## 4. Scaling parametric memory in neural sequence models

In autoregressive modeling, parametric memory is treated as the repository of factual knowledge or visual patterns stored in model parameters. MoVE introduces a global bank of learnable value embeddings
\[
\mathcal{E}\in\mathbb{R}^{N_{\text{vocab}}\times M\times d}
\]
shared across all attention layers, together with a soft gating mechanism that mixes retrieved value embeddings into the value stream. The central scaling variable is the number of slots \(M\). Because the additional FLOP term
\[
C_{\text{MoVE}}\approx 2dH(M+1)
\]
is small relative to the dense backbone cost
\[
C_{\text{std}}\approx 24d^2+4Td,
\]
the paper argues that parametric memory can be scaled independently of depth; for \(d=2048\), \(H=16\), \(M=32\), and \(T=2048\), the reported overhead is \(\approx 1.8\%\). Empirically, validation loss improves monotonically as \(M\) increases over the tested range [2601.22887].

A distinct formulation appears in LoRA finetuning, where parametric memory is measured as exact verbatim content written into low-rank updates. The reported Parametric Memory Law is
\[
\Delta \mathcal{L}(r,\ell)= C\cdot r^\alpha\cdot \ell^{-\beta}+b,
\]
where \(r\) is LoRA rank, \(\ell\) is sequence length, and \(\Delta \mathcal{L}\) is loss reduction over answer tokens. At token level, the paper identifies a deterministic phase transition: under greedy decoding, \(p_{\text{target}}>0.5\) is a sufficient condition for correct recall, equivalently \(\mathcal{L}_t<\ln 2\approx 0.693\). On that basis it proposes MemFT, which reallocates optimization toward sub-threshold tokens [2605.30260].

These two lines of work use different memory objects—global value banks in one case and low-rank residuals in the other—but they share a common thesis: parametric memory can be treated as an independent scaling axis with its own quantitative laws, rather than as a by-product of model width or depth.

## 5. Retrieval, agents, and selective parametric consolidation

In retrieval-augmented language models, the central question is not only how much the model knows parametrically, but when that knowledge is overridden by retrieved context. In Atlas-style RAG, causal mediation analysis shows that when the model can choose between parametric and non-parametric information, it relies more on context than on parametric knowledge. Early layers use subject and relation tokens to judge relevance, while later layers focus on object tokens and support copying. Parametric memory thus acts as a fallback when context is judged irrelevant or damaged, rather than as the dominant source whenever retrieval is present [2410.05162].

In self-evolving agents, TMEM recasts parametric memory as fast weights. The policy is sampled from \(\pi_{\theta_0+\Delta_t}\), where \(\Delta_t\) is an episode-local LoRA delta updated online from extraction actions that produce QA supervision. The fast-weight update is constrained to a fixed low-rank subspace, and SVD-based initialization of the LoRA projection improves online adaptation. This formulation turns memory writing into part of the agentic decision process: extraction quality determines future parametric adaptation within the same rollout [2606.04536].

ParamMem and ParamAgent use a different agentic route. Here the parametric module is a LoRA-finetuned language model \(p_\psi(r^g\mid x)\) whose output is not the task answer but a reflection signal. The module is trained on synthetic reflection pairs \((x_i,r_i^g)\), then sampled at low temperature in the first iteration and higher temperature later to inject reflective diversity. Across five datasets and three reflection frameworks, the reported average Pearson correlation between reflective diversity and performance is \(0.76\), and ParamAgent consistently improves over Reflexion, DoT, and DoT-bank on programming and multi-hop QA [2602.23320].

Selective parametric consolidation for long-running agents is formalized even more explicitly in EVAF. Each event receives a surprise score and a valence score, and the write gate is
\[
g_t=\sigma(k_s(s_t-\tau_s))\cdot \sigma(k_v(v_t-\tau_v)).
\]
Only events with \(g_t>\tau_w\) are admitted to the write buffer, after which sparse LoRA updates are applied with replay and an L2 anchor. In the loop-drift protocol, retrieval is strongest on short-fact accuracy (\(0.956\)–\(0.973\)), whereas EVAF is strongest on goal persistence and post-unload recovery (\(0.812\)–\(0.904\)) with only \(2\)–\(3\) parametric writes per \(200\) events. Mechanism controls further show that selective consolidation factorizes into **selection** and **actuation**, and that stale-memory invalidation remains unresolved [2606.26806].

## 6. Continual learning, modular stabilization, and unresolved boundaries

Continual learning work extends the idea of a parametric memory law from single-model scaling to long-horizon consolidation. Semi-parametric Memory Consolidation introduces a dual memory per task: non-parametric low-entropy cues \(\mathbf{v}\) and a parametric pattern-completion network \(N_C^t\). During wake, the system constructs and stores compressed cues; during sleep, it replays reconstructed samples to consolidate the backbone and classifier. This biomimetic wake–sleep mechanism approximates joint training while avoiding raw exemplar storage, and on ImageNet-100 the method approaches joint-training performance with a memory footprint comparable to replay-based baselines [2504.14727].

A related but more modular design appears in continual generative retrieval. PAMT adds a Parametric Memory Head to a frozen adapted GenIR backbone. The PMH is a product-key memory with fixed addressing; decoder hidden states retrieve sparse value vectors that yield residual hidden-space corrections, and those corrections are mapped to score adjustments only over trie-valid tokens. To reduce cross-slice interference, PAMT protects historically important rows and updates only a fixed budget of value entries chosen by current access frequency multiplied by inverse historical frequency. The result is a memory-only stabilization stage that improves retention on earlier slices while minimally affecting performance on newly added documents [2604.23388].

Across these literatures, a recurring misconception is that parametric memory is simply “more parameters.” The surveyed work indicates a narrower and more structured picture. In fractional systems, memory is a kernel law whose order changes stability and universality. In neural sequence models, it is often a separate scaling variable or low-rank store. In agents and continual learners, it is a controlled write process governed by gating, replay, protection, and bounded drift. A plausible synthesis is that parametric memory becomes useful when three conditions are met simultaneously: the write rule is selective, the substrate is modular enough to localize interference, and the memory signal complements rather than duplicates retrieval. Open boundaries remain explicit in the source literature, including the missing exchange rate between memory slots and transformer depth, the lack of principled stale-memory invalidation in selective consolidation, and unresolved universality questions for low-\(\alpha\) fractional dynamics [2604.23388][2606.26806][1709.00093].

Source: https://www.emergentmind.com/topics/parametric-memory-law