ByteRover: Agent-Native Memory for LLMs

• ByteRover is an agent-native memory architecture designed to enable high-fidelity, long-context reasoning in LLMs by internalizing memory curation and retrieval.
• It uses a hierarchical Context Tree and adaptive knowledge lifecycle to maintain semantic integrity, robust crash recovery, and superior performance across benchmarks.
• Its progressive retrieval cascade and in-process management eliminate semantic drift and coordination issues common in traditional external memory systems.

ByteRover is an agent-native memory architecture designed to extend LLMs with high-fidelity, long-context reasoning capabilities by internalizing the memory curation and retrieval process within the agent itself. Unlike traditional Memory-Augmented Generation (MAG) systems, which treat memory as an opaque, external service mediated via chunking, embedding, and graph extraction pipelines, ByteRover employs the same LLM that reasons about a task to curate, structure, and retrieve hierarchical knowledge in-process. This approach eliminates key failure modes associated with pipeline-based memory, including semantic drift, loss of coordination context, and fragile state recovery, and achieves state-of-the-art performance without relying on external infrastructure such as vector databases or embedding services (Nguyen et al., 2 Apr 2026).

1. Motivation and Theoretical Foundation

The primary motivation for ByteRover arises from intrinsic limitations in standard LLM architectures: fixed attention windows and rapid loss of long-term context, typified by the “lost-in-the-middle” phenomenon. Existing MAG approaches formalize memory as an external service $\chi$, accessed via:

$$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$

where $\psi$ retrieves memory elements from $\mathcal{M}_t$ using query $q_t$. This externalization leads to three principal failure modes:

• Semantic drift: divergence between intended and stored representations due to lack of agent-internal understanding.
• Lost coordination context: downstream agents lack access to rationale or intended future use of facts.
• Recovery fragility: state reconstruction from low-level indices is brittle after process interruptions.

ByteRover inverts this paradigm: the reasoning LLM is responsible for directly curating and managing memory, preserving end-to-end semantic integrity and full operational provenance.

2. System Architecture

ByteRover’s architecture consists of three logical layers:

• Agent Layer: The LLM executes a loop using in-process memory tools (curate, query, searchKnowledge).
• Execution Layer: Operations are serialized through a queue, invoking sandboxed CurateExecutor or QueryExecutor modules to avoid file conflicts.
• Knowledge Layer: Core knowledge is structured as a Context Tree—a file-based hierarchical knowledge graph $G = (N, E)$ implemented on disk with the directory structure: $q_t$5 Each node $n_i \in N$ is a standalone Markdown file encoding:

$$n_i = \langle \mathcal{R}_i, \mathcal{C}_i, \mathcal{V}_i, \mathcal{S}_i, \mathcal{L}_i \rangle$$

where: - $\mathcal{R}_i$: explicit @relation links (edges $E$), bidirectional with $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$0 traversal. - $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$1: provenance (task ID, source, timestamp, author). - $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$2: narrative details (rules, examples, dependencies). - $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$3: code, formulas, data snippets. - $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$4: adaptive lifecycle metadata (importance, maturity, recency).

A lightweight symbol tree of existing domains, topics, and subtopics is injected into the LLM system prompt, providing structural context without full content ingestion.

3. Adaptive Knowledge Lifecycle (AKL)

Each entry in ByteRover’s Context Tree is governed by an Adaptive Knowledge Lifecycle (AKL) across three axes:

• Importance Score $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$5: Assigned to $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$6, updated via:
  • Access: $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$7
  • Update: $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$8
  • Daily decay: $$y_t \sim f_\theta(\phi(o_t, s_t) \oplus \psi(\mathcal{M}_t; q_t)),$$9
• Maturity Tiers:
  • Transitions controlled by hysteresis:
  • draft $\psi$0 validated if $\psi$1, demote if $\psi$2
  • validated $\psi$3 core if $\psi$4, demote if $\psi$5
• Recency Decay:

$\psi$6

• Retrieval Ranking:

$\psi$7

where $\psi$8 is the normalized importance, and $\psi$9 are adjustable weights.

This lifecycle supports continuous refinement, controlled promotion of verified knowledge, and prioritization based on both operational recency and agent engagement.

4. Progressive Retrieval Mechanism

ByteRover utilizes a five-tier progressive retrieval cascade, designed to optimize both efficiency and accuracy. Queries typically resolve within the first three tiers without invoking an LLM, achieving sub-100 ms latencies:

Tier	Mechanism	Latency
0	Exact fingerprinted cache hit	$\mathcal{M}_t$00 ms
1	Fuzzy cache match (Jaccard similarity)	$\mathcal{M}_t$150 ms
2	MiniSearch BM25 + prefix (score threshold)	$\mathcal{M}_t$2100 ms
3	LLM call (optimized, prefetch, 1,024 tokens, $\mathcal{M}_t$3)	$\mathcal{M}_t$45 s
4	Multi-turn agentic reasoning (up to 50 iterations, 2,048 tokens, $\mathcal{M}_t$5)	8–15 s

Queries with no high-confidence BM25 match and low normalized scores ($\mathcal{M}_t$6) are flagged as “out of domain,” precluding hallucinations. The MiniSearch index supports BM25, fuzzy, and prefix matching, supplementing cache-based acceleration. The progressive architecture permits high-throughput retrieval for known facts, with automatic escalation to LLM-based reasoning for novel or complex queries.

5. Data Storage and Infrastructure

All knowledge is serialized as Markdown files in a Domain/Topic/Subtopic/Entry hierarchy on the local filesystem. There is no reliance on vector databases, graph databases, or external embedding services. Supporting infrastructure consists of:

• In-process MiniSearch index: Lightweight full-text search index.
• In-memory cache: Recent query-response mappings for Tier 0/1 hit acceleration.
• Write Atomicity: File updates utilize a write-to-temp-then-rename protocol for crash safety.
• Concurrency: A single agent process per project serializes writes through a task queue, obviating explicit file locks.

This zero-external-infrastructure design supports human-readability, robust crash recovery, and reproducibility.

6. Empirical Evaluation

ByteRover has been evaluated on two principal benchmarks:

• LoCoMo: 35 ultra-long dialogue sessions ($\mathcal{M}_t$720K tokens/session; 1,982 questions) spanning Single-Hop, Multi-Hop, Open-Domain, and Temporal reasoning.
• LongMemEval-S: 500 questions in six categories (Knowledge Update, Single-Session User/Assistant/Preference, Temporal Reasoning, Multi-Session) over $\mathcal{M}_t$848 sessions ($\mathcal{M}_t$9100K tokens/session).

Baselines include HonCho, Hindsight, Mem0, Memobase, Zep, and OpenAI Memory. Judgments are provided by Gemini 3 Flash and Gemini 3.1 Pro with apples-to-apples prompts.

Performance metrics on LoCoMo:

Method	Single-Hop	Multi-Hop	Open-Domain	Temporal	Overall
HonCho	93.2	84.0	77.1	88.2	89.9
Hindsight	86.2	70.8	95.1	83.8	89.6
ByteRover	97.5	93.3	85.9	97.8	96.1

ByteRover leads by $q_t$0 pp overall, $q_t$1 pp on multi-hop, and $q_t$2 pp on temporal.

Performance metrics on LongMemEval-S:

Method	KU	SSU	SSA	SSP	TR	MS	Overall
Chronos†	96.2	94.3	100.0	80.0	90.2	91.7	92.6
Hindsight	94.9	97.1	96.4	80.0	91.0	87.2	91.4
ByteRover	98.7	98.6	98.2	96.7	91.7	84.2	92.8

Cold query p50/p95/p99 latencies are 1.2/1.4/1.7 s for LoCoMo (272 docs) and 1.6/2.3/2.5 s for LongMemEval-S (23,867 docs).

Ablation studies demonstrate a critical reliance on tiered retrieval: omitting the progressive cascade reduces overall accuracy by 29.4 pp; disabling OOD detection or the relation graph yields marginal ($q_t$3 pp) reductions.

7. Discussion and Limitations

ByteRover’s agent-native curation eliminates major sources of semantic drift, as every memory manipulation is performed by the agent itself with explicit rationale. Coordination context—the rationale, usage intention, and provenance—remains accessible to all concurrently operating agents via shared Context Tree entries and lifecycle metadata. Crash recovery is robust: all operational state is maintained in the file hierarchy, enabling precise resumption.

Identified limitations include:

• Write-path cost: LLM-based curation is slower and less suited for high-throughput ingestion compared to pure embedding pipelines.
• Novel-query latency: Cache and MiniSearch misses escalate to LLM-based retrieval, which is slower than vector searches.
• LLM dependency: Quality of curation is sensitive to the backbone LLM’s reasoning and output compliance.
• Scalability: The in-memory MiniSearch and single task queue may require sharding or alternative backends for corpora exceeding $q_t$410,000 entries.
• Concurrency: Sequential serialization of curation operations limits concurrent throughput, though in practice curation frequency is typically low.

Planned extensions include integration with lightweight embedding-based retrieval for ultra-high-throughput scenarios, sharding strategies for file indices, and enhanced multi-agent synchronization atop the Context Tree (Nguyen et al., 2 Apr 2026).