SHIMI: Semantic Hierarchical Memory Index

• SHIMI is a decentralized semantic memory architecture that organizes data into a hierarchical tree of concepts, enabling abstract and precise retrieval.
• It employs top-down semantic traversal and CRDT-style merging to ensure explainability, scalability, and eventual consistency in multi-agent systems.
• Integration of Bloom filters and Merkle-DAG hashes minimizes synchronization costs while supporting efficient memory updates across decentralized agents.

SHIMI, or Semantic Hierarchical Memory Index, is a proposed memory architecture for AI agents that replaces flat, embedding-only retrieval with a hierarchical, semantically organized memory tree that can also operate in decentralized, multi-agent settings. It models memory as a rooted tree of semantic concepts rather than a flat list of vectors; retrieval works by top-down semantic traversal from abstract intent to specific entities; and, in decentralized deployments, each agent keeps its own local semantic tree and shares updates asynchronously through a lightweight sync protocol that uses Merkle-DAG hashes, Bloom filters, and CRDT-style merging to exchange only the minimal divergent pieces of memory (Helmi, 8 Apr 2025).

1. Motivation and problem setting

SHIMI was proposed from the claim that standard Retrieval-Augmented Generation and dense vector retrieval systems are useful but fundamentally limited when agents need memory that is not only relevant, but also abstractable, interpretable, and synchronizable across peers. The paper identifies four limitations of the common RAG/vector-search paradigm: flat representation, weak abstraction; semantic drift and low precision; poor interpretability; and centralization mismatch (Helmi, 8 Apr 2025).

In this framing, vector databases typically store memory as an unstructured collection of embeddings, making it hard to represent conceptual hierarchy, abstraction layers, or compositional meaning. Retrieval becomes “nearest neighbors in embedding space,” not “reasoning through a concept hierarchy.” Similar embeddings may retrieve content that looks related but does not match the intended concept well, especially in complex or heterogeneous domains. Vector search is also described as largely a black box, which makes it difficult to explain why a particular result was retrieved. Finally, many RAG/vector systems assume a single central index, whereas decentralized AI ecosystems require multiple agents to own local memory, evolve independently, and reach eventual consistency without a global coordinator.

SHIMI is proposed as an answer to all four limitations: it makes memory hierarchical, meaning-driven, explainable, and native to decentralized synchronization. This suggests that the architecture is intended not merely as a retrieval subsystem, but as a memory model for decentralized cognitive systems.

2. Semantic hierarchy and memory representation

SHIMI represents memory as a rooted directed tree,

$T = (V, E)$

where each node $v \in V$ is a semantic concept and each edge $(v_i, v_j) \in E$ indicates a parent-child relation (Helmi, 8 Apr 2025).

The paper defines several key parameters:

• $T$: maximum number of children per node, i.e. branching factor
• $L$: maximum abstraction depth
• $\delta \in [0,1]$: similarity threshold for placement or retrieval
• $\gamma \in [0,1]$: semantic compression ratio used when abstracting upward

Each node contains a semantic summary $s(v)$, a child list $C(v)$, a set of entities $\mathcal{E}_v$, and a parent pointer $v \in V$0. The structure is intended to mimic cognitive memory organization: broad concepts near the top, more specific ones deeper down. SHIMI can insert new information by locating the appropriate semantic branch, attaching an entity at the right level, and, if needed, introducing new abstraction nodes.

The hierarchy is kept compact through semantic merging. If a parent exceeds the child limit $v \in V$1, SHIMI merges the two most similar children into a new parent,

$v \in V$2

The paper also defines semantic compression upward through the hierarchy,

$v \in V$3

where $v \in V$4 is the number of words in a node summary at level $v \in V$5. This ensures that higher levels are progressively more abstract rather than just longer paraphrases. A plausible implication is that SHIMI treats abstraction as a first-class constraint on memory organization, not as a post hoc summarization step.

3. Retrieval and insertion mechanics

SHIMI’s retrieval algorithm is explicitly not global nearest-neighbor search. Instead, retrieval is a semantic descent: start at the root nodes; at each level, compare the query with the node summary $v \in V$6; only expand branches whose semantic similarity exceeds the threshold $v \in V$7; continue until leaves are reached; and return the entities stored at relevant leaves (Helmi, 8 Apr 2025).

The central decision rule is

$v \in V$8

In simplified form, the retrieval procedure initializes the frontier with all root nodes, expands to the next level, checks the similarity condition for each node, collects entities if the node is a leaf, and otherwise continues descending into its children. The paper states that retrieval is explainable because the system can report the exact path taken through the hierarchy. It also allows fallback to embeddings if no semantic path is found, though the main design is hierarchy-first.

Insertion begins by matching an incoming entity to candidate root buckets, then descending the tree semantically. The algorithmic structure given in the paper uses MatchToBucket(e) to find likely root domains, DescendTree(R, x) to traverse downward using semantic relation checks, and AddNode(...) to attach the new entity to the best parent node. If the parent already has a semantically equivalent child, the entity is merged there.

This organization makes the retrieval path itself a human-readable reasoning trace. The paper’s interpretation is that hierarchical semantic retrieval is both more precise and more explainable, because the chosen traversal records how the system moved from abstract user intent to specific stored items.

4. Decentralized synchronization protocol

One of SHIMI’s most distinctive contributions is that it is designed from the beginning for decentralized agent systems. Rather than one shared global memory index, each agent maintains its own local tree $v \in V$9. These trees evolve independently as agents learn, add entities, or reorganize local knowledge. Synchronization is asynchronous and partial, not full replication (Helmi, 8 Apr 2025).

The sync process is conceptually defined as follows: compute a hash summary of the local tree; exchange root hashes with a peer; if roots differ, identify the divergent subtree; produce a compact probabilistic summary of that subtree using a Bloom filter; send the Bloom filter to the peer; the peer responds with the nodes it is missing or where conflicts exist; conflicting nodes are merged using CRDT-like rules; and the local tree is updated.

The protocol has three technical components.

First, each tree has a Merkle root hash,

$(v_i, v_j) \in E$0

When two peers communicate, they compare root hashes $(v_i, v_j) \in E$1 and $(v_i, v_j) \in E$2. If they differ, SHIMI recursively identifies the smallest divergent subtree $(v_i, v_j) \in E$3 by comparing structural hashes.

Second, after detecting the divergent subtree, the sender generates a Bloom filter,

$(v_i, v_j) \in E$4

which compactly represents the contents of the divergent subtree and lets the other peer infer which nodes it is missing without scanning the full tree.

Third, for conflicting nodes, SHIMI uses a merge function,

$(v_i, v_j) \in E$5

with the stated CRDT-style properties

$(v_i, v_j) \in E$6

$(v_i, v_j) \in E$7

$(v_i, v_j) \in E$8

If semantic summaries disagree, the protocol favors the one with greater abstraction depth or observed usage. The overall effect is eventual convergence without centralized coordination. This suggests that SHIMI treats decentralized memory synchronization as part of the memory architecture itself, rather than as an external replication layer.

5. Complexity and empirical evaluation

The paper provides approximate complexity reasoning for insertion, retrieval, and synchronization. For a balanced tree with $(v_i, v_j) \in E$9 root domains and branching factor $T$0, with $T$1 entities,

$T$2

which yields

$T$3

So the hierarchy depth grows logarithmically with the number of entities (Helmi, 8 Apr 2025).

Let $T$4 be the average number of nodes visited per level due to semantic overlap. The paper gives the following estimate for insertion and retrieval:

$T$5

$T$6

In practice, retrieval is usually cheaper than insertion because it skips merging/abstraction and can terminate early once relevant leaves are found. For a divergent subtree $T$7 and local edit operations $T$8, the synchronization cost is

$T$9

so synchronization is proportional to the changed region, not the full tree size.

The evaluation was conducted in a simulated decentralized setting on four main axes: retrieval accuracy, traversal efficiency, synchronization cost, and scalability.

Metric	SHIMI	RAG baseline
Top-1 accuracy	90%	65%
Mean Precision@3	92.5%	68.0%
Interpretability score	4.7 / 5	2.1 / 5

The retrieval benchmark used 20 semantically non-trivial queries involving agents described with different lexical forms but similar functions. Traversal cost, measured as average nodes visited per query at increasing tree depths, was reported as follows: Depth 2: SHIMI 3, RAG 8.1; Depth 3: SHIMI 4.2, RAG 12.3; Depth 4: SHIMI 5.6, RAG 17.0; Depth 5: SHIMI 7.1, RAG 21.4.

Evaluation item	Result
Full-state replication vs SHIMI partial sync, 3 nodes	118 KB vs 1320 KB, about 91% savings
Full-state replication vs SHIMI partial sync, 4 nodes	162 KB vs 1740 KB, about 90.7% savings
Full-state replication vs SHIMI partial sync, 5 nodes	204 KB vs 2210 KB, about 90.8% savings
Full-state replication vs SHIMI partial sync, 6 nodes	248 KB vs 2650 KB, about 90.6% savings
Conflict resolution time, 5% conflict	20 ms
Conflict resolution time, 15% conflict	65 ms
Conflict resolution time, 30% conflict	130 ms
SHIMI latency, 100 / 500 / 1000 / 2000 entities	10 ms / 13.5 ms / 17.8 ms / 22.3 ms
RAG flat-scan baseline, 100 / 500 / 1000 / 2000 entities	9 ms / 45 ms / 88 ms / 180 ms

The paper interprets these results as showing advantages in retrieval accuracy, semantic fidelity, and scalability. Because the evaluation is simulation-only, these results establish a benchmarked design claim rather than a deployment result in live heterogeneous networks.

6. Applications, advantages, and limitations

The paper presents SHIMI as infrastructure for several decentralized collaboration settings: decentralized agent markets; federated knowledge graphs; autonomous multi-agent systems; and blockchain-based task orchestration (Helmi, 8 Apr 2025).

In decentralized agent markets, agents advertise skills such as compute, legal summarization, or sensor analysis, and SHIMI helps match tasks to the right agent based on meaning rather than wording while providing a traceable path for auditing. In federated knowledge graphs, independent institutions can maintain local ontologies and synchronize partial views without requiring global agreement on one canonical ontology. In autonomous multi-agent systems, SHIMI allows each agent to manage local memory and share updates without centralized indexing. In blockchain-native task platforms, it can serve as a semantic index for task descriptions, bids, and milestone records while preserving transparency and deterministic matching.

The main advantages claimed in the paper are semantic precision, explainability, scalability, decentralization, efficient sync, and eventual consistency. In the paper’s bottom-line formulation, SHIMI does not simply retrieve by similarity; it organizes memory into a hierarchy of concepts, searches by top-down semantic narrowing, and synchronizes memory between agents using delta-based, CRDT-inspired replication.

The paper is equally explicit about limitations. SHIMI currently assumes a strict tree, which makes it simpler and more efficient but causes difficulty for knowledge that is naturally polyhierarchical or graph-like. Generalization, similarity checks, and merging rely on language-model-driven judgments, which are powerful but opaque and hard to formally verify. Experiments are conducted in simulated decentralized settings, not in live heterogeneous networks with real latency, failures, and adversarial behavior. The benchmark may also favor semantically rich queries, so behavior on simpler surface-level tasks may differ.

The stated future directions are to evaluate SHIMI in real-world decentralized agent networks, generalize beyond trees to graphs and cyclic ontologies, use lightweight on-device LLMs for local semantic reasoning, and formalize semantic generalization and merge procedures as an interoperable protocol layer. This suggests that SHIMI is best understood as a proposed decentralized semantic memory protocol whose current contribution lies in the unification of hierarchical semantic retrieval and asynchronous partial synchronization, rather than in a finalized universal memory substrate.