Evolve: A Persistent Knowledge Lifecycle for Small Language Models

Abstract: Evolve pairs a small local LLM with a persistent, teacher-compiled knowledge store -- refined through sleep consolidation and usage-driven refresh -- to deliver substantial accuracy gains over the model's parametric baseline while amortizing teacher costs through cross-query knowledge reuse. Rather than retrieving document fragments at query time, Evolve constructs a store of semantically coherent sections compiled by teacher models at natural conceptual boundaries; new sections are staged on acquisition, consolidated offline through teacher-mediated merging, and refreshed inline when expired. A 2B-parameter local model handles classification and generation; large teacher models are invoked only for knowledge operations. Across 750 benchmark queries spanning custom specialist questions, NaturalQuestions, and TriviaQA, the 2B model augmented by Evolve improves from 20-33% baseline accuracy to 60-84% (+40-52pp) while reducing teacher invocations by over 50% through reuse. Post-consolidation compresses the knowledge store by 31-33.5% across three independent benchmarks while preserving accuracy; section-based retrieval outperforms chunk-based retrieval by 5-9pp across every lifecycle condition. The architecture supports two generation modes over the same lifecycle -- suppress (strict section-only grounding, auditable) and augment (section-supplemented responses).

• The paper introduces a persistent knowledge lifecycle architecture that separates reasoning from factual retrieval through a teacher-curated store, boosting accuracy by up to 52 percentage points.
• The paper implements a six-phase pipeline and dual memory system (staged and consolidated) to ensure coherent and semantically dense retrieval from externalized knowledge.
• The approach reduces teacher calls and cloud dependency, enabling cost-effective edge deployment and robust auditability in regulated environments.

Persistent Knowledge Lifecycle Architecture for Small LLMs

Introduction

"Evolve: A Persistent Knowledge Lifecycle for Small LLMs" (2604.23424) introduces a biologically inspired architecture for augmenting small LLMs by decoupling reasoning capability from factual knowledge. The core innovation is a persistent, teacher-curated external knowledge store, refined through staged acquisition, in-query refresh, and offline consolidation. This approach differs from conventional RAG, which retrieves ephemeral document fragments per query, by instead growing an internal knowledge base of semantically coherent sections and employing a two-phase memory architecture: newly acquired/staged (hippocampal analog) and consolidated/canonical (cortical analog).

Architectural Principles

Evolve implements a six-phase lifecycle pipeline: classify → retrieve → acquire → refresh → generate → consolidate. All factual knowledge is externalized; the small local model (2B parameters) contributes only linguistic competence and logical reasoning. Teacher LLMs (router-managed, category-specialized) are invoked exclusively for knowledge operations—never for live generation. Knowledge units are teacher-compiled sections: dense, fact-rich, semantically coherent, and reusable across queries. Each section is assigned a TTL reflecting knowledge volatility, with expired units triggering inline teacher refresh.

The architecture fundamentally supports two generation modes: suppress (strict section-only grounding; disables parametric knowledge) and augment (section-supplemented; allows parametric fallback). This separation ensures auditability when required, and flexible accuracy when permitted.

Figure 1: Evolve Architecture overview; demonstrates the separation of reasoning (small LLM) from persistent, teacher-mediated knowledge consolidation.

Knowledge Store and Lifecycle Management

Evolve's knowledge store distinguishes itself from static corpora and stateless retrieval. Sections are general-purpose, topically segmented units authored at conceptual boundaries by the teacher, not chunked via arbitrary splits. Each carries structured metadata—including topic, summary, content, category, and TTL—for governance and auditability.

Staging store admits all newly acquired or refreshed sections. Periodically, an offline "sleep" consolidation merges, deduplicates, and reorganizes sections via teacher compilation, compressing the knowledge footprint by 31–33.5% across benchmarks while preserving or modestly improving accuracy. Overlaps are detected within categories; teacher compilation may merge, split, or discard redundant information.

The design supports rapid organic growth: early queries create islands of knowledge, which gradually connect via embedding similarity, with cross-query reuse amortizing teacher cost and response latency as the store matures.

Query Pipeline and Retrieval

The orchestrator executes a conditional pipeline. The classifier (local LLM) routes queries by type (factual, coding, conversational), selects relevant categories, and emits domain-specific search texts. Category-search pairs maximize retrieval relevance through domain-sensitive embedding search (staging and canonical stores). Cache misses invoke teacher acquisition; cache hits validate TTL and trigger refresh as needed. Two-pool retrieval architecture ensures only the most relevant, coherent sections are passed to generation.

Section-based retrieval outperforms chunk-based approaches, consistently delivering higher accuracy from fewer, more semantically dense blocks. This design choice is empirically validated and becomes increasingly critical as context budgets shrink for small models.

Empirical Evaluation

Evolve was evaluated on 750 benchmark queries spanning specialist knowledge, NaturalQuestions, and TriviaQA. Key quantitative findings:

• Augmentation of 2B LLM accuracy from 20–33% baseline to 60–84% (+40–52pp), with strict grounding (suppress) and parametric supplementation (augment) both delivering substantial gains.
• Section retrieval yields 5–9pp higher accuracy than chunked retrieval across all lifecycle stages.
• Post-consolidation compresses store size by ~32% with strict accuracy preservation or increase (+1pp on custom benchmark).
• Warm-cache knowledge reuse reduces teacher calls per query by >50% and response latency by 29% compared to cold-cache (teacher-dominated).
• Lowering the retrieval similarity threshold from 0.80 to 0.75 doubles cache hit rate and halves teacher invocation with negligible accuracy impact (<0.4pp; within Wilson CI).
• On MMLU (parametric knowledge benchmark), suppress mode induces 3.6pp accuracy loss, confirming effective grounding and tradeoff.

Error analysis reveals that further accuracy gains are bottlenecked by reasoning precision and composition in the local model, not retrieval or external knowledge.

Practical and Theoretical Implications

Evolve offers auditability and cost governance critical for enterprise and regulated deployments. Suppress mode guarantees source traceability of all factual claims; augment mode balances coverage and efficiency. Category-level teacher routing enables integration of specialized, domain-trained models and aligns with flexible organizational requirements.

The deployment model supports edge-device self-sufficiency: as the store matures, queries are served locally, minimizing cloud teacher cost. Usage-driven TTL refresh ensures maintenance cost tracks utility—no stale content is refreshed unless actively queried.

Theoretically, the approach demonstrates that persistent externalized memory and retrieval unit quality (semantic sections) are more influential for small-model augmentation than retrieval frequency or stateless RAG. Biological principles of memory consolidation and experience-driven learning are explicitly mapped to computational primitives.

Limitations and Future Directions

• The architecture is tuned for factual QA; code generation tasks bypass retrieval as teacher sections are primarily descriptive.
• Reasoning and composition errors in small models limit ceiling—scaling local model size and specializing prompts could improve performance.
• Full edge deployment (teacher inference on-device) and scaling of sleep consolidation are pending future work.
• User-profile consolidation via fact extraction from conversation, and code-oriented retrieval/generation, represent natural extensions.

Conclusion

Evolve establishes persistent, teacher-compiled knowledge as a first-class augmentation for small LLMs, delivering significant accuracy gains over the frozen parametric baseline, robust auditability, and operational efficiency. Section-based retrieval and lifecycle management via sleep-inspired consolidation and TTL-driven refresh ensure the evolving knowledge base remains coherent, relevant, and cost-effective. The architecture generalizes across domains and benchmarks, with future developments poised to expand from factual QA into broader application spaces.