---
title: 'FadeMem: Adaptive Memory for LLM Agents'
url: https://www.emergentmind.com/topics/fademem
type: topic
---

# FadeMem: Adaptive Memory for LLM Agents

FadeMem denotes a biologically-inspired memory architecture for large language model (LLM) agents that addresses long-term retention, adaptive forgetting, and conflict resolution in agent memory systems. The architecture integrates differential decay, semantic relevance, recency, and frequency modulation within a dual-layer memory hierarchy, supported by LLM-guided memory fusion and contradiction handling. Unlike fixed retention or binary memory protocols, FadeMem operationalizes nuanced, context-driven forgetting, mirroring aspects of human cognitive memory mechanisms [2601.18642].

## 1. Dual-Layer Memory Hierarchy and Adaptive Exponential Decay

FadeMem uses a tiered structure consisting of short-term memory (SML) and long-term memory (LML). Each memory slot $m_n$ at time $t$ is a tuple
$$
m_n(t) = (c_n, s_n, v_n(t), \tau_n, f_n)
$$
where $c_n \in \mathbb{R}^d$ is the embedding, $s_n$ is the raw text, $v_n(t) \in [0,1]$ is the memory strength, $\tau_n$ the creation timestamp, and $f_n$ the raw access count.

Importance for each memory is continuously evaluated:
$$
I_n(t) = \alpha\,\mathrm{rel}(c_n, Q_t) + \beta\,\frac{\bar{f}_n(t)}{1 + \bar{f}_n(t)} + \gamma\,\exp[-\delta(t - \tau_n)]
$$
where $Q_t$ is the current context embedding, $\mathrm{rel}(\cdot)$ is cosine similarity (or attention), and $\bar{f}_n(t)$ is the exponentially decayed access frequency. The thresholds $\theta_\mathrm{promote}$ and $\theta_\mathrm{demote}$, with hysteresis, determine migration of memories between SML and LML:
$$
\mathrm{Layer}(m_n) = \begin{cases}
\mathrm{LML} & \text{if } I_n(t)\ge \theta_\mathrm{promote} \\
\mathrm{SML} & \text{if } I_n(t)< \theta_\mathrm{demote}
\end{cases}
$$

Forgetting is governed by a parameterized exponential decay:
$$
v_n(t) = v_n(0)\,\exp\left[-\lambda_n(t - \tau_n)^{\beta_n}\right]
$$
with $\lambda_n = \lambda_{\text{base}}\,\exp(-\mu I_n(t))$ and
$$
\beta_n = \begin{cases}
0.8, & m_n \in \mathrm{LML} \quad (\text{sub-linear decay}) \\
1.2, & m_n \in \mathrm{SML} \quad (\text{super-linear decay})
\end{cases}
$$
Strengthening on recall, $v_n(t^+)$, incorporates diminishing returns from repeated accesses. Pruning occurs if $v_n(t)<\epsilon_{\text{prune}}$ or upon prolonged dormancy. The architecture yields half-lives of approximately $11.25$ days (LML) and $5.02$ days (SML) at $I_n=0$.

## 2. Modulation by Semantic Relevance, Frequency, and Temporal Patterns

Semantic relevance is quantitatively assessed via $\mathrm{rel}(c_n, Q_t)$, using cosine similarity or LLM attention scores. Access frequency replaces raw counts with the decayed frequency $\bar{f}_n(t) = \sum_j \exp[-\kappa(t-t_j)]$, emphasizing recency and dampening extreme values via $\bar{f}/(1+\bar{f})$. Recency is incorporated as an exponential term in $I_n(t)$. The importance-weighted decay rate $\lambda_n$ slows forgetting for crucial memories, with $\mu$ controlling the sensitivity of this adaptation.

## 3. LLM-Guided Conflict Resolution and Adaptive Memory Fusion

Incoming memories $m_{\text{new}}$ trigger content-based retrieval of potentially conflicting or related slots, $\mathcal{S}$, defined by similiarity above $\theta_\mathrm{sim}$. For each pair $(s_\text{new}, s_i)$, a compact LLM (GPT-4o-mini) classifies the relationship as compatible, contradictory, subsumes, or subsumed.

- **Compatible:** Redundant importance is penalized by $I_i \leftarrow I_i (1 - \omega\,\mathrm{sim}(c_{\text{new}}, c_i))$.
- **Contradictory:** Older memories are suppressed based on time difference: $v_i(t) \leftarrow v_i(t)\exp[-\rho \mathrm{clip}((\tau_{\text{new}}-\tau_i)/W_{\mathrm{age}},0,1)]$.
- **Subsumes/Subsumed:** LLM fuses or consolidates the memory contents.

Clusters exceeding a similarity and temporal window are merged by an LLM, with the fused memory strength $v_{\text{fused}}(0) = \max_{i \in \mathcal{C}} v_i(t) + \varepsilon \operatorname{Var}(\{v_i\})$ and an adjusted decay constant:
$$
\lambda_{\text{fused}} = \frac{\lambda_\text{base}}{1+\log |\mathcal{C}_k|}
$$
A subsequent LLM validation ensures no loss of distinct high-information facts above $\theta_\text{preserve}$.

## 4. Empirical Evaluation and Benchmarks

FadeMem was evaluated across Multi-Session Chat (MSC), LoCoMo, and LTI-Bench. Baselines include fixed-window (4K–16K tokens), retrieval-augmented generation (RAG), Mem0 (unified memory), and MemGPT (hierarchical).

| System           | Retention (Critical/Context) | Storage (%) | RP@10/TCS (MSC) | F1 (LoCoMo) | FCR (LoCoMo) | SRR |
|------------------|-----------------------------|-------------|-----------------|-------------|--------------|-----|
| FadeMem          | 82.1 % / 71.0 %             | 55.0 %      | 77.2 % / 0.82   | 29.43       | 85.9 %       | 0.45|
| Mem0             | 78.4 % / 69.1 %             | 100 %       | 74.8 % / 0.79   | 28.37       | 83.6 %       | 0   |
| MemGPT           | 75.6 % / 62.8 %             | 85.3 %      | -               | -           | -            | -   |
| Fixed 16K window | ~50.2 % / 44.8 %            | 100 %       | -               | -           | -            | -   |

Conflict resolution (LTI-Bench, 4,075 injected conflicts):
- FadeMem macro-averaged accuracy: $68.9\%$, consistency: $80.4\%$.
- Mem0: $64.2\%$ (accuracy), $75.8\%$ (consistency).
- MemGPT: $62.4\%$ (accuracy), $74.6\%$ (consistency).

FadeMem reduces memory usage by approximately $45\%$ with retrieval and reasoning metrics that meet or surpass contemporary baselines.

## 5. Ablation Studies and Limitations

Ablation on LoCoMo (multi-hop F1):
- Full model: $29.43$
- Without dual-layer: $19.45$ ($-33.9\%$)
- Without fusion: $13.63$ ($-53.7\%$)
- Without conflict resolution: $22.88$ ($-22.4\%$)

Omission of key modules most strongly impacts temporal reasoning and open-domain scenarios. Limitations include:
- Dependence on LLM inference quality and resulting latency for fusion and conflict adjudication.
- Fixed schedules for important decay parameters ($\mu$, $\lambda_\text{base}$); meta-learning for these is suggested as future work.
- Instances of over-compression in memory fusion can occur, potentially eliminating fine-grained causal relationships when $\theta_\text{preserve}$ is not optimally set.

## 6. Context Within Memory-Efficient AI and Future Perspectives

FadeMem operates in the domain of efficient agent memory management, complementing and improving upon fixed-window and hierarchical memory protocols by embedding cognitive principles of selective forgetting [2601.18642]. Its adoption of dual-layer hierarchy, adaptive decay, and LLM-mediated consolidation reflects emerging trends toward bio-inspired, information-centric memory systems for foundation model agents. Prospective work includes dynamic meta-learning of decay schedules and more granular memory clustering for enhanced preservation of complex, temporally distributed knowledge.

## 7. Comparison: "FadeMem" in Edge Training Contexts

It is notable that the term "FadeMem" is also referenced in the context of federated adversarial decoupled learning (FADE) for edge device training [2209.03839]. In this paradigm, "FadeMem" characterizes aggressive memory savings achieved by modularizing model training; clients only load one small model module at a time rather than the entire network, yielding $40$–$75\%$ RAM reduction and enabling adversarial training where it was previously infeasible due to memory constraints. This separate usage highlights a distinct application domain (model training on heterogeneous hardware), but converges with agent memory FadeMem on the theme of resource-efficient retention and selective processing.

FadeMem, across both domains, is defined by differentiated, importance-aware management of memory—whether in model weights for federated training or in symbolic/content memory for LLM agents—using principled architectural and algorithmic mechanisms to optimize capability under strict resource bounds.

Source: https://www.emergentmind.com/topics/fademem