Parametric Memory Block (PMB)

• Parametric Memory Block (PMB) is a dual-use term that denotes both a LoRA-based modular memory unit for language models and a Poisson multi-Bernoulli filter in multi-target tracking.
• In LoRA-based external parametric memory systems, PMBs serve as addressable memory primitives parameterized by low-rank matrices, enabling efficient encoding of task-specific knowledge and document content.
• Advanced routing techniques, such as PMDRouter, decode query-conditioned activations over PMB banks to achieve zero-shot retrieval without relying on separately trained classifiers.

Searching arXiv for the cited papers and closely related PMB usages to ground the article. Parametric Memory Block (PMB) most specifically denotes, in recent large-language-model literature, an individual LoRA module used as a modular, pluggable unit of parametric memory in a LoRA-based External Parametric Memory (EPM) bank. In that setting, a PMB is attached to a frozen backbone LLM, parameterized by low-rank matrices, and used to encode specific knowledge, task skills, or document content for modular inference (Ji et al., 5 Jul 2026). The same acronym, however, is also established in multi-object tracking for Poisson multi-Bernoulli models and filters, which are unrelated to neural parametric memory (Xia et al., 2024). The term therefore requires domain-specific interpretation.

1. Terminological scope

Recent work uses the PMB label in at least two technically distinct ways. In language-model memory systems, PMB refers to a LoRA memory unit inside an external bank of adapters. In multi-target Bayesian filtering, PMB denotes a Poisson multi-Bernoulli density or filter. This dual usage is not merely stylistic; the underlying mathematical objects, training procedures, and applications differ completely (Ji et al., 5 Jul 2026, García-Fernández et al., 8 May 2026).

Usage	Meaning	Representative source
PMB in EPM	Individual LoRA module as modular parametric memory	(Ji et al., 5 Jul 2026)
PMB in tracking	Poisson multi-Bernoulli density/filter	(Xia et al., 2024)

A related family of papers does not always use the exact PMB label, but studies closely adjacent constructs: LoRA as a probe of exact parametric memory, parametric reflective memory for agents, parametric memory heads for continual generative retrieval, and parametric memory networks for semantic communication (Xu et al., 28 May 2026, Yao et al., 26 Feb 2026, Mekonnen et al., 25 Apr 2026, Chen et al., 3 May 2026). Taken together, these works place PMB within a broader research program on storing task-relevant information directly in parameters rather than in explicit retrieval buffers.

2. PMB as a LoRA memory unit in external parametric memory

In LoRA-based EPM, a PMB is defined as an individual LoRA module that acts as a modular, pluggable unit of parametric memory when attached to a frozen backbone LLM. Each PMB is parameterized by low-rank matrices, typically denoted as $\theta_k=(A_k,B_k)$, and can encode specific knowledge, task skills, or document content. A bank of such PMBs is maintained externally and can be dynamically attached to or detached from a base model for modular inference, with the stated purpose of enabling flexible, updatable, and scalable memory architectures compared to a monolithic model (Ji et al., 5 Jul 2026).

This formulation makes the PMB an addressable memory primitive rather than merely a parameter-efficient adaptation device. The bank abstraction separates the frozen backbone from memory-bearing LoRA modules, so memory access becomes a routing problem: for a given query, the system must determine which PMB should be activated. A plausible implication is that PMB-based systems inherit both the modularity of external memory and the latency profile of adapter-style inference, provided routing can be done without a separate learned router.

The paper “Parametric Memory Decoding for Zero-Shot Routing in LoRA-Based External Parametric Memory” organizes this problem explicitly around EPM access and proposes PMD-Bench, covering document-level, domain-level knowledge, and task-skill (Ji et al., 5 Jul 2026). In that benchmark design, each LoRA or PMB is trained on a disjoint unit, and the routing task is to select the correct PMB without any additional routing data.

3. Routing and decoding over PMB banks

Parametric Memory Decoding (PMD) reframes PMB routing as decoding query-conditioned activations over external parametric memory. For an input query $q$, each PMB $L_k$ produces a query-conditioned response $\rho_k(q)$, and routing is expressed as

$$s_k(q) = D\left( \mathrm{Resp}\big(h(q), \theta_k\big) \right), \qquad k^*(q) = \arg\max_{k\in[K]} s_k(q).$$

Here $h(q)$ is an aggregated, frozen-backbone representation of the input, and $\theta_k=(A_k,B_k)$ are PMB parameters (Ji et al., 5 Jul 2026).

PMDRouter is the concrete parameter-free instantiation of PMD. It uses a single backbone prefill to obtain an activation vector $u(x)$, computes the LoRA-induced linear response

$$\rho_{\ell,k}(x) = \Delta W_{\ell,k} u(x) = B_{\ell,k}A_{\ell,k} u(x),$$

and then decodes a scale-normalized response energy

$$E_{\ell,k}(x) = \frac{ \left\| \rho_{\ell,k}(x) \right\|_2^2 }{ \left\| u(x) \right\|_2^2 \left\| \Delta W_{\ell,k} \right\|_F^2 + \epsilon }.$$

Layerwise energies are aggregated and calibrated, and the selected PMB is

$q$0

The reported pathway is

$q$1

This involves only a single backbone forward pass per query and no per-PMB fine-tuning or additional routing model (Ji et al., 5 Jul 2026).

On PMD-Bench, PMDRouter is reported to achieve the strongest internal-signal performance across multiple zero-shot routing settings. The detailed results list PaperQA accuracies of 0.600–0.658 across backbones, Task-LoRA gains up to approximately 0.92, and NQ-DomainLoRA best zero-shot results in most cases, such as 0.769–0.790. The same study reports that higher response margin predicts routing success and that PMD reduces the routing design space by more than 40x compared to prior static matching approaches (Ji et al., 5 Jul 2026).

A common misconception in adapter routing is that zero-shot access necessarily requires an auxiliary router. The PMD formulation is offered precisely as a counterexample: it treats routing as a readout problem over PMB responses rather than as a separately trained classification module (Ji et al., 5 Jul 2026).

4. Capacity laws and exact recall in LoRA-based parametric memory

Because the PMB in EPM is a LoRA module, the quantitative study of LoRA memory capacity provides a direct lens on PMB behavior. The paper “How LoRA Remembers? A Parametric Memory Law for LLM Finetuning” introduces a power law for exact parametric memory,

$q$2

where $q$3 is loss reduction after fine-tuning, $q$4 is LoRA rank, and $q$5 is sequence length. The constants $q$6 are dataset/model-dependent positive constants; $q$7 measures how efficiently rank increases memory, while $q$8 captures the increasing difficulty with longer sequences (Xu et al., 28 May 2026).

This law is reported to hold robustly across Qwen3-8B and Llama3.1-8B, and across both semantic and random data, with $q$9 and low mean absolute percentage errors. The stated implication is predictive power: given a pair $L_k$0, the law predicts the achievable loss reduction (Xu et al., 28 May 2026). For PMB design, this suggests that adapter budget and memory load can be analyzed quantitatively rather than only through downstream task scores.

The same work argues that low average loss is not sufficient for exact verbatim recall. At the token level, it identifies a deterministic phase transition: for any token, if its probability under greedy decoding satisfies $L_k$1, the correct token will always be generated. The corresponding token-level cross-entropy threshold is

$L_k$2

An ordered phase is defined by $L_k$3, and a disordered phase by $L_k$4; successful verbatim recall requires every token’s probability to cross this threshold, not merely a low average loss (Xu et al., 28 May 2026).

MemFT, or Memorization-oriented Fine-Tuning, is introduced as a threshold-guided optimization strategy that reallocates the training budget toward sub-threshold tokens. The variants MemFT-OT and MemFT-SW focus optimization on tokens with loss above the critical threshold and on tokens near the first predicted failure. Reported evaluations on the Long-Context Memorization Stress Test and PhoneBook show higher exact match and token-level accuracy than standard SFT at the same or lower parameter budget, and perfect recall with lower LoRA ranks in difficult settings (Xu et al., 28 May 2026). For PMBs, a plausible implication is that routing alone is not sufficient; write-time optimization must also account for tokenwise failure modes if the memory unit is expected to support exact recall.

5. Related parametric-memory architectures in LLMs and agents

A closely related line of work uses parametric memory modules without always adopting the PMB label. “ParamMem: Augmenting Language Agents with Parametric Reflective Memory” defines a lightweight, finetuned LLM module that encodes cross-sample reflection patterns into parameters. The PMB or ParamMem module is trained on pairs $L_k$5 by minimizing

$L_k$6

typically with Llama-3.1-8B and LoRA, and is then sampled with temperature control to provide diverse reflection signals inside a reflection-based agent loop (Yao et al., 26 Feb 2026). The paper reports consistent improvements on code generation, mathematical reasoning, and multi-hop question answering, and further states that ParamMem is sample-efficient, enables weak-to-strong transfer across model scales, and supports self-improvement without reliance on a stronger external model (Yao et al., 26 Feb 2026).

“Memory Depth, Not Memory Access: Selective Parametric Consolidation for Long-Running Language Agents” separates retrieval access from memory depth, defined as durable goal-conditioned tendencies written into a small parametric store. Its EVAF mechanism uses surprise- and valence-gated LoRA consolidation, with gate

$L_k$7

and a parametric update proportional to

$L_k$8

In the loop-drift protocol, retrieval is strongest on shallow factual recall, with short-fact accuracy 0.956–0.973, while EVAF is strongest on goal persistence and post-unload recovery, at 0.812–0.904, using only 2–3 parametric writes per 200 events (Han, 25 Jun 2026). This provides an explicit corrective to the misconception that parametric memory and retrieval are interchangeable: the reported strengths are complementary rather than identical.

“A Parametric Memory Head for Continual Generative Retrieval” proposes post-adaptation memory tuning (PAMT), which freezes the backbone and attaches a modular parametric memory head with product-key memory and fixed addressing. During prefix-trie constrained decoding, decoder hidden states sparsely query the memory head to produce residual corrections in hidden space; these are mapped to score adjustments through the frozen output embedding matrix and computed only over trie-valid tokens. To limit cross-slice interference, the method updates only a fixed budget of memory values selected using decoding-time access statistics, prioritizing entries frequently activated by the current slice and rarely used in prior sessions (Mekonnen et al., 25 Apr 2026). Although this module is termed a parametric memory head rather than a PMB, it occupies the same design space: compact, modular, parameter-resident memory added after backbone adaptation.

6. Other scientific uses and boundary cases

Outside language modeling, parametric memory appears in distinct architectures. “Evolving Token Communication with Parametric Memory Network” introduces a receiver-side parametric memory network for semantic token communication over MIMO fading channels. Only an equal-length prefix of each semantic token is transmitted; a GPT-2-based recovery module, trained using codebook labels and kNN-based teacher distributions, reconstructs the missing suffix information from the received token prefixes, with semantic memory stored implicitly in network parameters. The system includes an online evolution strategy that periodically updates the parametric memory network and the entire system using newly observed test samples, and the reported gain over the evolving memory benchmark reaches up to 1.09 dB PSNR under different channel conditions and channel bandwidth ratios (Chen et al., 3 May 2026). This usage is adjacent to PMB research in that memory is again encoded in parameters, but the object is a receiver-side reconstruction network rather than a LoRA memory unit.

By contrast, in multitarget filtering the acronym PMB denotes Poisson multi-Bernoulli, not parametric memory. In “Hybrid PHD-PMB Trajectory Smoothing Using Backward Simulation,” the PMB filtering density is the intermediate multi-object representation produced by the PHD update before projection back to a Poisson point process, and the hybrid PHD-PMB trajectory smoother uses these extracted PMB densities in a backward pass to estimate the set of all trajectories without labeling or tagging (Xia et al., 2024). In “Variational PMB filter via coordinate descent Kullback-Leibler divergence minimisation,” PMB denotes a variational approximation to a PMBM posterior in augmented space, with the V-PMB projection performing coordinate descent KLD minimisation and preserving the probability hypothesis density (García-Fernández et al., 8 May 2026). These works are central in tracking, but they are terminologically, mathematically, and application-wise separate from PMB as a parametric memory block in neural systems.

The coexistence of these meanings is itself a practical consideration for literature search and citation. In current arXiv usage, “PMB” is not a univocal term: in one line of work it names a LoRA-based memory unit for modular neural memory, while in another it names a set-valued Bayesian filtering object (Ji et al., 5 Jul 2026, García-Fernández et al., 8 May 2026).