ByteRover Architecture: Agent-Native Memory

• ByteRover Architecture is an agent-native design where the LLM curates, organizes, and retrieves its own knowledge, storing information in human-readable Markdown files.
• It inverts traditional memory-augmented pipelines by making memory operations a core part of the LLM's processing loop, achieving sub-100 ms query responses.
• The system emphasizes crash safety, explicit provenance, and adaptive knowledge lifecycle management, ensuring robust long-term reasoning without external memory infrastructure.

ByteRover is an agent-native memory architecture for LLMs in which the LLM itself curates, organizes, and retrieves episodic and semantic knowledge without relying on external memory infrastructure. The architecture inverts the prevailing Memory-Augmented Generation (MAG) pipeline: instead of treating memory as an external black-box database, all memory operations—such as addition, update, and retrieval—are first-class tools within the LLM’s active loop. Knowledge is anchored in a hierarchical Context Tree of human-readable Markdown files on the local file system, with an explicit relation graph, provenance, and adaptive lifecycle metadata. Retrieval is handled by a five-tier progressive strategy designed to resolve most queries at sub-100 ms latency without invoking the LLM. Empirical results on benchmark datasets demonstrate state-of-the-art long-term memory performance, competitive accuracy, and robust fault tolerance, all without vector or graph databases (Nguyen et al., 2 Apr 2026).

1. Design Principles and Architectural Goals

ByteRover is based on three primary architectural inversions:

• Agent-Native Memory Loop: The LLM is responsible for not only reasoning but also for the full lifecycle of knowledge curation—eliminating semantic drift caused by decoupled storage pipelines.
• Local, Human-Readable Storage: All knowledge is stored as Markdown files, eliminating the dependency on vector databases, graph databases, or external embedding services.
• Crash Safety and Fault Tolerance: File operations employ atomic write-to-temp-then-rename semantics, guaranteeing easy recovery and preventing data corruption.

The core design goals are:

• Eliminate semantic drift between the agent’s memory intent and storage outcome.
• Preserve provenance and agent coordination context with explicit metadata (“why,” “when,” and “by whom” for each entry).
• Achieve fast sub-100 ms retrieval for the majority of queries without invoking the LLM unless escalation is required.
• Require zero external infrastructure, maximizing portability and inspectability via Markdown.

2. Hierarchical Context Tree and Knowledge Graph

The Context Tree is the central data structure. It is a file-based, hierarchical knowledge graph defined as a directed graph $\mathcal{G}=(\mathcal{N},\mathcal{E})$:

• Nodes $\mathcal{N}$ are entries (Markdown files).
• Edges $\mathcal{E}$ are explicit “@relation” links to other entries.

Each entry $n_i$ is a tuple:

$$n_i = \langle \mathcal{R}_i, \mathcal{C}_i, \mathcal{V}_i, \mathcal{S}_i, \mathcal{L}_i \rangle$$

• $\mathcal{R}_i:$ Relation set (bidirectional links via @path/to/other.md)
• $\mathcal{C}_i:$ Raw concept and provenance (task, sources, timestamp, author)
• $\mathcal{V}_i:$ Narrative (dependencies, rules, examples, diagrams)
• $\mathcal{S}_i:$ Snippets (code blocks, formulas, raw data)
• $\mathcal{L}_i:$ Lifecycle metadata (importance, maturity, recency, counters)

The hierarchy spans five symbol kinds: Domain, Topic, Subtopic, Context, Summary. Each file is indexed for O(1) forward/backward traversal. The directory structure (Domain > Topic > Subtopic) is injected into the LLM system prompt to provide ambient awareness.

3. Adaptive Knowledge Lifecycle (AKL)

AKL governs the evolution and prioritization of entries via importance scores, maturity, and recency:

• Importance ($\mathcal{N}$0): $\mathcal{N}$1, increased by $\mathcal{N}$2 on access, $\mathcal{N}$3 on update, decays daily by $\mathcal{N}$4.
• Maturity Tiers: Entries are promoted/demoted through {draft, validated, core} with hysteresis. Promotion/demotion thresholds:
  • Draft → Validated at $\mathcal{N}$5, demote if $\mathcal{N}$6.
  • Validated → Core at $\mathcal{N}$7, demote if $\mathcal{N}$8.
• Recency Decay: $\mathcal{N}$9, with $\mathcal{E}$0 (days since last update).
• Compound Retrieval Score: For query $\mathcal{E}$1,

$\mathcal{E}$2

with tunable weights, BM25 for lexical matching, normalized importance $\mathcal{E}$3, and recency $\mathcal{E}$4.

This multi-criteria ranking integrates relevance, importance from history, and freshness, yielding adaptive prioritization for retrieval scenarios.

4. Five-Tier Progressive Retrieval Pipeline

ByteRover employs a five-tier retrieval system that prioritizes speed and minimizes unnecessary LLM calls. Each tier is applied successively until a confident result is obtained:

Tier	Retrieval Mechanism	Typical Latency	Escalation Condition
0	Exact cache hit	$\mathcal{E}$50 ms	Hash/cache fingerprint match
1	Fuzzy cache (Jaccard)	$\mathcal{E}$650 ms	$\mathcal{E}$7
2	MiniSearch (BM25 etc.)	$\mathcal{E}$8100 ms	Score $\mathcal{E}$9, gap $n_i$0
3	Single LLM call w/ prefetch	$n_i$15 s	Score $n_i$2
4	Full agentic multi-turn	8–15 s	All other queries

Pseudocode for retrieval:

$n_i$6

Score normalization for BM25: $n_i$3

5. Knowledge Management and Operational Layers

All entries reside in Markdown files within a directory tree: /Domain/Topic/Subtopic/entry.md. Each file begins with YAML frontmatter: title, tags, keywords, related (@ annotations), and lifecycle statistics (importance, maturity, recency, counts, timestamps). The body is structured into Relations, Raw Concept, Narrative, and Snippets.

• Curation: Executes sandboxed in a CurateExecutor with a ToolsSDK for operations such as curate(), searchKnowledge(), and readFile(). Pre-compaction uses LLM summarization and deterministic truncation for input size constraints.
• Execution Layer: Ensures serial task order, eliminating write–write conflicts without requiring file-locking.
• Knowledge Layer: Maintains the Context Tree, a MiniSearch full-text index (BM25, fuzzy, prefix searching), and an in-process cache.
• Crash Safety: Achieved by atomic write-to-temp-then-rename file operations.

6. Empirical Evaluation and Ablation

ByteRover’s architecture has been evaluated across canonical memory benchmarks:

• LoCoMo (35 sessions, 1,982 questions): Single-Hop 97.5%, Multi-Hop 93.3%, Open-Domain 85.9%, Temporal 97.8%, Overall 96.1% (outperforming HonCho at 89.9%).
• LongMemEval-S (500 questions, 23,867 docs): Knowledge Update 98.7%, Single-Session User 98.6%, Single-Session Assistant 98.2%, Single-Session Preference 96.7%, Temporal Reasoning 91.7%, Multi-Session 84.2%, Overall 92.8% (Chronos-Low 92.6%, Hindsight 91.4%).
• Latency Profile (cold queries): LoCoMo p50 $n_i$4, LongMemEval-S p50 $n_i$5.
• Ablation (LongMemEval-S): w/o Tiered Retrieval drops overall accuracy to 63.4% (–29.4 pp), w/o OOD Detection 92.4% (–0.4 pp), w/o Relation Graph 92.4% (–0.4 pp).

The empirical results indicate that the combination of a hierarchical agent-curated Context Tree, adaptive lifecycle scoring, and a five-tier retrieval pipeline is an effective approach for agent-native long-context reasoning and robust episodic memory without external infrastructure (Nguyen et al., 2 Apr 2026).