---
title: 'LiCoMemory: Hierarchical Memory for LLMs'
url: https://www.emergentmind.com/topics/licomemory
type: topic
---

# LiCoMemory: Hierarchical Memory for LLMs

LiCoMemory is an end-to-end agentic memory framework designed to address the persistent memory limitations in large language model (LLM) agents. By interposing a lightweight, hierarchical external memory (CogniGraph) and real-time controller between users and language models, LiCoMemory enables efficient long-term reasoning, multi-session consistency, and improved retrieval accuracy. This architecture outperforms existing graph-based and flat memory systems on key dialogue and reasoning benchmarks while reducing retrieval and update latencies.

## 1. System Architecture

LiCoMemory is structured as a modular agentic memory system for LLM-based agents, comprising two primary components:

1. **CogniGraph**: A lightweight hierarchical graph index serving as external agentic memory.
2. **Agentic Memory Controller**: Orchestrates the flow between user queries, CogniGraph retrievals/updates, prompt assembly, and LLM invocation.

On every user-agent interaction, the controller extracts entities and temporal cues from the user input, requests contextual retrievals from CogniGraph (session summaries, triples, and chunks), assembles the context-rich prompt for the LLM, and subsequently updates CogniGraph with the dialogue chunk, maintaining both summary and semantic layers. Updates and retrievals operate on a unified in-memory index, allowing for the immediate integration of new information.

## 2. Hierarchical CogniGraph Representation

CogniGraph models external memory as a three-layer, directed graph $G = (V, E)$, where:

- **Session Nodes $\mathbf{V_{session}}$**: Store a summary $s_j$, distilled keywords $K_j$, and timestamp $\tau_j$.
- **Triple Nodes $\mathbf{V_{entity}}$**: Represent (head entity, relation, tail entity) triples $t_i = (e_h, r, e_t)$, each hyperlinked to session nodes and chunk nodes.
- **Chunk Nodes $\mathbf{V_{chunk}}$**: Store raw dialogue text $c_k$ and timestamp $\tau_k$.

Edges $E$ connect session nodes to triple nodes (extracted for a session) and triple nodes to chunk nodes (extracted from dialogue chunks). No intra-layer edges are present, minimizing index redundancy and promoting efficient retrieval.

### Formal Construction

Let:
- $V_{session} = \{s_j \mid j=1\dots N_{sessions}\}$
- $V_{entity} = \{t_i \mid i=1\dots N_{triples}\}$
- $V_{chunk} = \{c_k \mid k=1\dots N_{chunks}\}$

Triple nodes $t_i$ carry timestamps $\tau_i$ and maintain cross-layer links:
- $link_{session}: V_{entity} \rightarrow V_{session}$
- $link_{chunk}: V_{entity} \rightarrow \mathcal{P}(V_{chunk})$

### Update Pseudocode

```python
Initialize CogniGraph G = (V_session={}, V_entity={}, V_chunk={}, E={})

def process_chunk(chunk_text, session_id, timestamp):
    # 1. Session-level summary
    if session_id in V_session:
        s = V_session[session_id]
        s.summary = update_summary(s.summary, chunk_text)
        s.keys    = update_keywords(s.keys, chunk_text)
        s.timestamp = timestamp
    else:
        s = SessionNode(id=session_id, summary=make_summary(chunk_text),
                        keys=extract_keywords(chunk_text), timestamp=timestamp)
        V_session.add(s)

    # 2. Triple extraction and deduplication
    triples = extract_triples(chunk_text)
    for tri in triples:
        if not exists_similar_triple(tri):
            t_node = TripleNode(tri, timestamp)
            V_entity.add(t_node)
            E.add((s, t_node))
            c_node = ChunkNode(chunk_text, timestamp)
            V_chunk.add(c_node)
            E.add((t_node, c_node))
        else:
            t_exist = find_similar_triple(tri)
            c_node = ChunkNode(chunk_text, timestamp)
            V_chunk.add(c_node)
            E.add((t_exist, c_node))
```

Triple deduplication leverages type-aware and semantic similarity matching, ensuring memory compactness and reducing redundant fact storage.

## 3. Real-Time Update and Hierarchical Retrieval Mechanisms

### Update Complexity

For each incoming chunk:
- **Session node update**: $O(1)$ lookup/insert (hash table)
- **Triple extraction/deduplication**: $O(M)$ operations ($M$: triples per chunk)
- **Similarity checks**: $O(M \cdot \log N)$ (approximate nearest neighbor indexing among $N$ triples)

Empirically, single-chunk update latency is approximately $T_G \approx 5.2$ seconds on A100 GPUs, with token consumption per session $K_G \approx 13.5$k.

### Retrieval Workflow

LiCoMemory's retrieval pipeline is hierarchy- and temporal-aware:

1. **Entity Extraction**: $Q_{entities} = \{e^q_1, ..., e^q_m\}$ from the query.
2. **Session-Level Ranking**: Compute semantic similarity $S_s(j) = sim_{emb}(K_j, Q_{entities})$; select top-$K_s$ sessions.
3. **Triple-Level Scoring**: For candidate session $s_j$ and each neighbor triple $t_i$, $S_t(i) = sim_{emb}((e_h, e_t, r)_i, Q_{entities})$.
4. **Integrated Relevance Scoring**: Harmonic mean
   $$
   S_{sem}(i) = \frac{2 \cdot S_s(j) \cdot S_t(i)}{S_s(j) + S_t(i)}
   $$
   Temporal weighting
   $$
   w(\Delta \tau_i) = \exp\left[ -\left( \frac{\Delta \tau_i}{\hat{\tau}} \right)^k \right],\quad 0 < k < 1
   $$
   Final score
   $$
   R(i) = S_{sem}(i) \cdot w(\Delta \tau_i)
   $$
5. **Reranking and Prompt Construction**: Sort triples by $R(i)$, fetch linked session summaries and chunks, and select top-$N$ units for the LLM prompt.

#### Reranking Algorithm Pseudocode

```python
def retrieve_and_rerank(query):
    Q_ents = extract_entities(query)
    for s_j in V_session:
        S_s[j] = semantic_sim(K_j, Q_ents)
    S_s_sorted = topK_sessions(S_s)

    candidates = []
    for s in S_s_sorted:
        for t in neighbors(s):
            S_t = semantic_sim(triple_repr(t), Q_ents)
            delta_tau = abs(now - t.timestamp)
            candidates.append((s, t, S_s[s], S_t, delta_tau))

    tau_hat = median([delta_tau for _,_,_,_,delta_tau in candidates])
    for (s, t, S_s_, S_t_, delta_tau) in candidates:
        S_sem = 2 * S_s_ * S_t_ / (S_s_ + S_t_)
        w     = exp(- (delta_tau / tau_hat) ** k)
        R     = S_sem * w
        store(score=R, triple=t, session=s)

    best = topN_by_score(candidates)
    return assemble_prompt(best)
```

## 4. Empirical Evaluation

LiCoMemory was quantitatively assessed on the LongMemEval and LoCoMo benchmarks, testing multi-session, single-hop, multi-hop, temporal, open-domain, and adversarial reasoning.

### Performance Comparison Table

| Method         | LongMemEval Acc. | Rec.@15 | T_R  | LoCoMo Acc. | Rec.@15 | T_R  |
|----------------|------------------|---------|------|-------------|---------|------|
| LoCoMo (RAG)   | 17.6 %           | 22.0 %  | 4.9s | 23.6 %      | 25.5 %  | 4.9s |
| Mem0           | 56.8 %           | 61.2 %  | 2.7s | 53.2 %      | 57.1 %  | 3.3s |
| A-MEM          | 57.4 %           | 62.2 %  | 3.0s | 43.8 %      | 49.2 %  | 3.1s |
| Zep            | 60.2 %           | 62.7 %  | 2.8s | 40.3 %      | 51.1 %  | 3.5s |
| **LiCoMemory** | **69.2 %**       |**72.4 %**|**2.6s**|**63.0 %**  |**64.5 %**|**2.4s**|

LiCoMemory achieves an absolute improvement of +9.0 pp accuracy over the second-best baseline (Zep) and exhibits lowest observed query latency.

### Ablation Insights

- Removing hierarchical retrieval leads to a –22 pp accuracy reduction.
- Omitting temporal weighting yields –22 pp on temporal QA.
- Excluding session-level summaries causes –12 pp overall accuracy drop.

### Real-Time Simulation

Chunk-wise insertion on LongMemEval demonstrates:

- Update latency $T_G \approx 5.21$ s/session
- Retrieval latency $T_R \approx 2.63$ s/query
- Token usage: $K_G \approx 13.52$k/session, $K_R \approx 1.22$k/query
- QA accuracy is maintained at 67.4% (only 1.8 pp less relative to static full-history context).

A plausible implication is that LiCoMemory maintains robust performance even under incremental, streaming update scenarios.

## 5. Applications and Limitations

LiCoMemory functions as a cognitive scaffold for LLM agents, supporting:

- Multi-session consistency via session-level summaries.
- Temporal query handling through decay weighting.
- Grounding specific facts with hyperlinks to raw source chunks.

In practical deployment, LiCoMemory enables agents to adaptively retrieve and update persistent knowledge, yielding coherent long-term dialogues and improving agentic reasoning capacity.

**Limitations** include linear memory growth with session volume and the single-agent design of CogniGraph. Future work may address multi-agent knowledge graph sharing, adaptive compression of historical data, and improved learned summary abstraction. Extraction and summarization fidelity remain as bottlenecks for maximal retrieval precision.

## 6. Significance in Agentic Long-Term Reasoning

LiCoMemory demonstrates that a semantically indexed, lightweight hierarchical graph—in combination with temporal and hierarchy-aware retrieval mechanisms—substantially surpasses previous external memory architectures in both efficiency and reasoning accuracy. These design principles suggest a path forward for scalable, persistent, agentic memory solutions compatible with real-time LLM deployments.

Source: https://www.emergentmind.com/topics/licomemory