LiCoMemory: Hierarchical Memory for LLMs

• LiCoMemory is an end-to-end memory framework that uses a hierarchical CogniGraph to enhance LLM reasoning and maintain multi-session consistency.
• It employs a real-time agentic memory controller to efficiently update and retrieve contextual data for improved dialogue and reasoning benchmarks.
• Empirical results show LiCoMemory outperforms traditional memory systems with higher accuracy and reduced latency across multiple evaluation benchmarks.

LiCoMemory is an end-to-end agentic memory framework designed to address the persistent memory limitations in LLM agents. By interposing a lightweight, hierarchical external memory (CogniGraph) and real-time controller between users and LLMs, LiCoMemory enables efficient long-term reasoning, multi-session consistency, and improved retrieval accuracy. This architecture outperforms existing graph-based and flat memory systems on key dialogue and reasoning benchmarks while reducing retrieval and update latencies.

1. System Architecture

LiCoMemory is structured as a modular agentic memory system for LLM-based agents, comprising two primary components:

1. CogniGraph: A lightweight hierarchical graph index serving as external agentic memory.
2. Agentic Memory Controller: Orchestrates the flow between user queries, CogniGraph retrievals/updates, prompt assembly, and LLM invocation.

On every user-agent interaction, the controller extracts entities and temporal cues from the user input, requests contextual retrievals from CogniGraph (session summaries, triples, and chunks), assembles the context-rich prompt for the LLM, and subsequently updates CogniGraph with the dialogue chunk, maintaining both summary and semantic layers. Updates and retrievals operate on a unified in-memory index, allowing for the immediate integration of new information.

2. Hierarchical CogniGraph Representation

CogniGraph models external memory as a three-layer, directed graph $G = (V, E)$, where:

• Session Nodes $\mathbf{V_{session}}$: Store a summary $s_j$, distilled keywords $K_j$, and timestamp $\tau_j$.
• Triple Nodes $\mathbf{V_{entity}}$: Represent (head entity, relation, tail entity) triples $t_i = (e_h, r, e_t)$, each hyperlinked to session nodes and chunk nodes.
• Chunk Nodes $\mathbf{V_{chunk}}$: Store raw dialogue text $c_k$ and timestamp $\tau_k$.

Edges $\mathbf{V_{session}}$0 connect session nodes to triple nodes (extracted for a session) and triple nodes to chunk nodes (extracted from dialogue chunks). No intra-layer edges are present, minimizing index redundancy and promoting efficient retrieval.

Formal Construction

Let:

• $\mathbf{V_{session}}$1
• $\mathbf{V_{session}}$2
• $\mathbf{V_{session}}$3

Triple nodes $\mathbf{V_{session}}$4 carry timestamps $\mathbf{V_{session}}$5 and maintain cross-layer links:

• $\mathbf{V_{session}}$6
• $\mathbf{V_{session}}$7

Update Pseudocode

$\tau_j$0

Triple deduplication leverages type-aware and semantic similarity matching, ensuring memory compactness and reducing redundant fact storage.

3. Real-Time Update and Hierarchical Retrieval Mechanisms

Update Complexity

For each incoming chunk:

• Session node update: $\mathbf{V_{session}}$8 lookup/insert (hash table)
• Triple extraction/deduplication: $\mathbf{V_{session}}$9 operations ($s_j$0: triples per chunk)
• Similarity checks: $s_j$1 (approximate nearest neighbor indexing among $s_j$2 triples)

Empirically, single-chunk update latency is approximately $s_j$3 seconds on A100 GPUs, with token consumption per session $s_j$4k.

Retrieval Workflow

LiCoMemory's retrieval pipeline is hierarchy- and temporal-aware:

1. Entity Extraction: $s_j$5 from the query.
2. Session-Level Ranking: Compute semantic similarity $s_j$6; select top-$s_j$7 sessions.
3. Triple-Level Scoring: For candidate session $s_j$8 and each neighbor triple $s_j$9, $K_j$0.
4. Integrated Relevance Scoring: Harmonic mean

$K_j$1

Temporal weighting

$K_j$2

Final score

$K_j$3

1. Reranking and Prompt Construction: Sort triples by $K_j$4, fetch linked session summaries and chunks, and select top-$K_j$5 units for the LLM prompt.

Reranking Algorithm Pseudocode

$\tau_j$1

4. Empirical Evaluation

LiCoMemory was quantitatively assessed on the LongMemEval and LoCoMo benchmarks, testing multi-session, single-hop, multi-hop, temporal, open-domain, and adversarial reasoning.

Performance Comparison Table

Method	LongMemEval Acc.	Rec.@15	T_R	LoCoMo Acc.	Rec.@15	T_R
LoCoMo (RAG)	17.6 %	22.0 %	4.9s	23.6 %	25.5 %	4.9s
Mem0	56.8 %	61.2 %	2.7s	53.2 %	57.1 %	3.3s
A-MEM	57.4 %	62.2 %	3.0s	43.8 %	49.2 %	3.1s
Zep	60.2 %	62.7 %	2.8s	40.3 %	51.1 %	3.5s
LiCoMemory	69.2 %	72.4 %	2.6s	63.0 %	64.5 %	2.4s

LiCoMemory achieves an absolute improvement of +9.0 pp accuracy over the second-best baseline (Zep) and exhibits lowest observed query latency.

Ablation Insights

• Removing hierarchical retrieval leads to a –22 pp accuracy reduction.
• Omitting temporal weighting yields –22 pp on temporal QA.
• Excluding session-level summaries causes –12 pp overall accuracy drop.

Real-Time Simulation

Chunk-wise insertion on LongMemEval demonstrates:

• Update latency $K_j$6 s/session
• Retrieval latency $K_j$7 s/query
• Token usage: $K_j$8k/session, $K_j$9k/query
• QA accuracy is maintained at 67.4% (only 1.8 pp less relative to static full-history context).

A plausible implication is that LiCoMemory maintains robust performance even under incremental, streaming update scenarios.

5. Applications and Limitations

LiCoMemory functions as a cognitive scaffold for LLM agents, supporting:

• Multi-session consistency via session-level summaries.
• Temporal query handling through decay weighting.
• Grounding specific facts with hyperlinks to raw source chunks.

In practical deployment, LiCoMemory enables agents to adaptively retrieve and update persistent knowledge, yielding coherent long-term dialogues and improving agentic reasoning capacity.

Limitations include linear memory growth with session volume and the single-agent design of CogniGraph. Future work may address multi-agent knowledge graph sharing, adaptive compression of historical data, and improved learned summary abstraction. Extraction and summarization fidelity remain as bottlenecks for maximal retrieval precision.

6. Significance in Agentic Long-Term Reasoning

LiCoMemory demonstrates that a semantically indexed, lightweight hierarchical graph—in combination with temporal and hierarchy-aware retrieval mechanisms—substantially surpasses previous external memory architectures in both efficiency and reasoning accuracy. These design principles suggest a path forward for scalable, persistent, agentic memory solutions compatible with real-time LLM deployments.