Unified Memory Budget Protocol

Updated 2 January 2026

Unified Memory Budget Protocol is a formal mechanism that allocates, compresses, and enforces memory usage to optimize output quality under strict system-wide resource budgets.
It employs two-stage scoring and selection methods, such as BudgetMem and LAVa, to dynamically manage memory retention and cache compression in LLMs and blockchain consensus.
The protocol’s deployment involves trade-offs in memory savings and accuracy, with ongoing research exploring adaptive policies and integration into real-world high-demand systems.

A Unified Memory Budget Protocol is a formalized mechanism for allocating, compressing, and enforcing memory utilization constraints in computational systems, most notably in the domains of LLM inference, cache compression in sequence models, and memory–work trade-off blockchain consensus. Such protocols are motivated by the need to maintain high-quality outputs—e.g., next-token predictions, QA accuracy, or system security—while operating under strict, system-wide resource budgets on memory or combined time-memory usage.

1. Formal Definition and Problem Setting

A Unified Memory Budget Protocol encodes the optimization of a system’s output quality subject to an explicit, quantifiable memory or memory–time budget. The canonical LLM memory budgeting formulation is as follows:

Let $D = (d_1, \ldots, d_N)$ be an input sequence (e.g., document, dialog) with $N \gg C$ tokens, where $C$ is the LLM’s context window; $q$ a query; $a$ a reference answer; $M$ the set of pre-chunked memory units (e.g., L tokens each; total $M$ chunks).
The protocol restricts the storage to a subset $S \subseteq \{1,\ldots,M\}$ with $\mathrm{Cost}(M_S) = \sum_{i \in S} \ell(c_i) \le B$ , maximizing expected answer quality:

$\max \;\; \mathbb{E}_{(q,a)}\left[F_1(a, \hat a(q; M_S))\right]$

subject to the memory budget $N \gg C$ 0.

In transformer cache compression, the problem is posed as

Selecting masks $N \gg C$ 1 to indicate "keep/evict" for token $N \gg C$ 2 in layer $N \gg C$ 3, head $N \gg C$ 4, such that

$N \gg C$ 5

and information-loss on future outputs is minimized.

In blockchain consensus, the protocol unifies proof-of-work and proof-of-space via a time-memory-data trade-off puzzle, enforcing global constraints $N \gg C$ 6 on the required work, memory, and unique inputs per mining round (Mihaljevic, 2019).

2. Algorithmic Protocols and Scoring Mechanisms

Protocols implement budget enforcement through learned or analytically derived scoring and eviction policies:

BudgetMem (selective memory policy for LLMs):
- For each candidate memory chunk $N \gg C$ 7, compute a retention score $N \gg C$ 8, with feature vector $N \gg C$ 9 including entity density, average TF–IDF, discourse markers, positional bias, and semantic novelty.
- Store $C$ 0 chunks, either by thresholding or top- $C$ 1 selection.
- Write-policy and ranking-margin losses supervise learning in trained gates; feature weighting enables effective zero-shot setups (Alla et al., 7 Nov 2025).
LAVa (layer-wise KV cache eviction):
- Compute importance score $C$ 2, reflecting each key–value (KV) entry’s contribution to the final layer residual, using recent attention matrices and value norms.
- Eviction selects top entries per layer, with both per-head ( $C$ 3) and per-layer ( $C$ 4) budgets dynamically allocated via cross-layer entropy normalization (Shen et al., 11 Sep 2025).
Consensus via TMD-TO puzzle:
- Nodes precompute a table of memory size $C$ 5 (seeds), use $C$ 6 CPU steps (cipher calls) per live challenge, under the global constraint $C$ 7.
- Honest majority is enforced by maintaining $C$ 8 given system-wide declared budgets (Mihaljevic, 2019).

3. Retrieval and Eviction Protocols

Central to unified memory protocols is the separation between write (storage) and read (retrieval) policies:

In BudgetMem, once the budgeted memory store $C$ 9 is built, retrieval for each query $q$ 0 is performed using BM25 over $q$ 1. Optionally, a fusion score $q$ 2 is used for hybrid ranking, although $q$ 3 (pure BM25) suffices in practice for strong performance (Alla et al., 7 Nov 2025).
LAVa enforces recent-token retention $q$ 4 in all transformer heads, followed by greedy layer-head-top- $q$ 5 selection and dynamic redistribution of budgets as streaming attention patterns change (Shen et al., 11 Sep 2025).
In TMD-TO consensus, the mining algorithm consists of a preprocessing step (table construction under chosen memory-budget), followed by an on-line loop that finds a suitable nonce and attempts to invert the challenge within the prescribed per-challenge time budget (Mihaljevic, 2019).

A distinctive characteristic is the two-stage (scoring then allocation/eviction) structure shared across these domains.

4. Metrics, Budget Sensitivity, and Pareto Trade-offs

Evaluation metrics and budget curves are fundamental for protocol tuning:

Memory Utilization: $q$ 6.
F $q$ 7 Degradation: $q$ 8 (Alla et al., 7 Nov 2025).
Memory Savings: $q$ 9.
In cache compression, importance is measured as reduced perturbation on residual streams ( $a$ 0), empirically linked to overall model accuracy (Shen et al., 11 Sep 2025).
For blockchain TMD-TO, the core metric is the success probability per round ( $a$ 1) and honest/miner expected chains grown per slot, subject to security inequalities (Mihaljevic, 2019).

Budget sensitivity analysis in BudgetMem shows, for long documents ( $a$ 2 tokens), setting $a$ 3 preserves $a$ 4 of baseline F $a$ 5 with $a$ 6 memory savings. Pareto trade-offs are smooth, with diminishing returns for $a$ 7 (Alla et al., 7 Nov 2025). LAVa’s ablation studies emphasize that both dynamic head and dynamic layer budgets are essential: static splitting costs 1–2 accuracy points at low budgets. In large-context model inference, LAVa achieves $a$ 8 speedup and $a$ 9 memory reduction on $M$ 0-token contexts (Shen et al., 11 Sep 2025).

Protocol/Task	Budget Ratio $M$ 1	F $M$ 2/Performance	Memory Utilization or Saving
BudgetMem (short)	0.30	F $M$ 3=0.7232 (-9.7%)	U=84.5% (S=15.5%)
BudgetMem (long)	0.30	F $M$ 4=0.8042 (-1%)	U=27.6% (S=72.4%)
LAVa (LongBench)	$M$ 5	Score=40.12 (+3.6)	$M$ 6 speedup, $M$ 7 GB saved
TMD-TO consensus	tune $M$ 8, $M$ 9	$M$ 0	$M$ 1 table size

5. Deployment Guidelines, Constraints, and Practical Considerations

Deployment recommendations are domain-specific:

BudgetMem: For resource-constrained LLM applications (e.g., single 24 GB GPU), the recommended memory ratio is $M$ 2– $M$ 3 on long inputs ( $M$ 45K tokens). Zero-shot, feature-weighted gating performs competitively without labeled data (entity density 0.2, TF–IDF 0.2, positional 0.15, numeric 0.15, discourse 0.1). BudgetMem incurs $M$ 520% additional retrieval latency, but realizes substantial 72% memory savings (Alla et al., 7 Nov 2025).
Limitations of BudgetMem include reliance on synthetic documents (further evaluation needed on Qasper, GovReport, LongBench), limited margin for short contexts ( $M$ 6500 tokens), and degradation on queries spanning multiple low-salience memory chunks.
LAVa: The protocol is lightweight, adding less than 1% inference overhead compared to SnapKV, and supports on-the-fly budget enforcement with no hyperparameter tuning. Dynamic tailoring matters: generation tasks require adaptive per-layer allocation; extraction tasks benefit from adaptive per-head allocation (Shen et al., 11 Sep 2025).
Consensus protocols: In TMD-TO, explicit memory–time trade-off lines are declared or cryptographically pledged per miner (“resource certificate”), ensuring system-level enforcement. Protocol parameters (challenge bit lengths, max budgets) are periodically adjusted by on-chain governance to retain both performance and security as hardware evolves (Mihaljevic, 2019).

6. Protocol Generalization and Theoretical Underpinnings

Unified Memory Budget Protocols share unifying theoretical motifs:

All enforce a hard system-wide budget constraint ( $M$ 7 or $M$ 8). This is achieved via write-time gating (for context storage), dynamic allocation (cache retention), or joint time–memory pledge (consensus).
Scoring functions reflect explicit or surrogate utility proxies: e.g., analytic F $M$ 9, attention-induced residual loss, or block success probability.
Two-stage protocol: scoring/valuation of candidate items (chunks, cache entries, puzzle steps) followed by greedy selection/eviction to meet the strict budget constraint.
In cache/KV compression, dynamic head budgets arise from per-head variance in information flow, while dynamic layer allocations are justified by cross-layer entropy in importance metrics, linking architecture and allocation (Shen et al., 11 Sep 2025).
In consensus, the protocol interpolates between proof-of-work ( $S \subseteq \{1,\ldots,M\}$ 0) and proof-of-space ( $S \subseteq \{1,\ldots,M\}$ 1), offering a continuous space of security–resource trade-offs.

A plausible implication is that unified budgeting schemes, when carefully individualized by architecture or task, enable cost–accuracy Pareto optimal deployment across a spectrum of high-memory and distributed-computation domains.

7. Future Directions and Open Questions

Open research avenues include:

End-to-end learning of write-gating policies for memory selection (moving beyond hand-tuned features) and per-document adaptive budget sizing in LLM systems (Alla et al., 7 Nov 2025).
Extension to multimodal content storage and retrieval (e.g., tables, code), beyond pure text environments.
Deployment, benchmarking, and human evaluation in real-world scenarios (e.g., Qasper, GovReport, LongBench for BudgetMem).
Enhanced theoretical analysis of budget allocation—especially for tasks with nonuniform, evolving resource demands—and security parameterization for consensus protocols under adversarial model.
Integration of temporal decay, frequency, or use-based heuristics for eviction beyond static least-salience or most-recent approaches.

These directions will likely advance unified memory budgeting as a paradigm for practical, scalable, and robust system design in both centralized and distributed computational environments.