Context as Infrastructure: Knowledge Compilation for Retrieval in Agentic Systems with Harnesses

Abstract: Retrieval-augmented generation is becoming agentic. A search harness writes queries, reads results, reformulates, and decides when to stop. This largely solves accuracy: a harness with only BM25/grep and MiniMax M3 answers 85% of BrowseComp-Plus and 91% on multi-hop FRAMES in our runs, without compiled or vector indexes. It exposes a context problem: an undisciplined loop repeats queries, rereads documents, and fills the window with unused passages. The missing layer is knowledge compilation: a configurable ingestion pipeline where LLMs enrich documents with context no chunk contains alone--navigation hierarchies with claim-evidence leaves, entity graphs, timelines, mind maps, ontologies, wikis, and memory. Compilation is not a better retriever: under a competent harness, structural artifacts gave no detectable accuracy gain on FRAMES, and a PageIndex-style outline degraded at scale. It is a supply layer. A navigation tree whose leaves are claim-evidence pairs--the mechanism we measure--cuts query tokens by 91% when evidence suffices and adds 1.9% when it does not, a 38% average reduction; this implies a 2% break-even sufficiency rate and a measured rate near 43%. A decision gate cuts latency from 140 s to 30 s, though substitution's token saving is bought with retrieval rounds and carries its own latency cost. Other artifacts--ontology, graph, wiki, timeline, mind map, memory--pay off in deployment (governance, integration, onboarding, temporal orientation, continuity) where accuracy benchmarks cannot see them. We describe RAGFlow's pipeline and harness (open source at https://github.com/infiniflow/ragflow), which turns the loop into five disciplined stations and keeps the context entering the model the smallest sufficient set; together they form an agent-native database: a store that hands the agent compiled, governed context instead of a ranked list of passages.

• Knowledge compilation is an ingestion-time process that transforms raw documents into structured artifacts. It consists of parsing PDFs, chunking, enriching with LLMs, validating, assembling, and registering. Examples include navigation trees and entity-relation graphs.
• A claim–evidence representation method significantly reduces token usage by 91% without affecting accuracy but increases latency. The evidence substitution model can reduce end-to-end latency by implementing a decision gate.
• Retrieval harness functions control admission, fallbacks, budgets, citations, and termination accurately enhancing model robustness and efficiency.

Problem formulation and central thesis

“Context as Infrastructure: Knowledge Compilation for Retrieval in Agentic Systems with Harnesses” (2610.41172) argues that the principal bottleneck in contemporary retrieval-augmented generation is shifting from document discovery to context control. In the system model considered by the paper, retrieval is not a single ranking operation but an agentic loop: an LLM formulates lexical queries, inspects results, reformulates failed searches, invokes tools, and determines whether sufficient evidence has been collected. The component coordinating this process is termed the search harness.

The paper’s central claim is that, once a sufficiently capable harness can interact directly with a corpus through BM25, grep, and file-level operations, improvements to the retrieval index cease to be the primary source of gains. The authors report that a harness using only BM25 and grep, with MiniMax M3 as the answering model, achieves 85% on BrowseComp-Plus and 91% on FRAMES in their runs. These numbers are explicitly not presented as comparable benchmark results: BrowseComp-Plus involves approximately three million tokens per question, whereas FRAMES is a multi-hop benchmark with approximately 325,000 tokens per question. The results are instead used as saturation checks showing that high answer accuracy is attainable without dense retrieval or a compiled index.

The paper therefore separates two functions that are commonly conflated. Knowledge compilation is an ingestion-time supply layer that transforms source documents into typed, versioned, provenance-bearing artifacts. The retrieval harness is a query-time control layer that determines which artifacts or source passages enter the model context, how tools are exposed, how insufficient evidence is discarded, and when the process terminates. This separation motivates the paper’s broader architectural concept of an agent-native database: a system that supplies governed, structured, provenance-linked context rather than merely returning ranked passages.

Knowledge compilation as a supply layer

The authors define knowledge compilation as a query-independent transformation from a corpus to a collection of artifacts. Each artifact is derived from source documents, retains lineage to exact source spans, and is stored with versioning and validation metadata. The transformation may combine deterministic parsing with LLM-based semantic enrichment.

The proposed ingestion pipeline comprises six stages:

1. Parsing converts heterogeneous sources such as PDFs, Office documents, HTML, Markdown, tables, and logs into a canonical representation while preserving headings, lists, tables, and page anchors.
2. Chunking produces retrieval units at natural document boundaries. The paper treats chunking as a recall-oriented engineering operation rather than a semantic representation of meaning.
3. Enrichment extracts claims, evidence spans, entities, relations, events, dates, ontology labels, summaries, or synthesized articles.
4. Validation checks evidence-span exactness, entity types, dates, and schema conformance. Invalid units are dropped rather than repaired by another model call.
5. Assembly organizes extracted units into trees, graphs, timelines, ontologies, wikis, mind maps, or memory stores.
6. Registration records source versions, offsets, extractor versions, validation status, access-control labels, and other lineage metadata.

The architecture imposes a strict separation between raw and derived channels. Generated artifacts are read-only and are never written back into the source corpus. This restriction is important because recursively summarizing generated text can cause provenance drift and progressively detach derived statements from the underlying evidence. The design instead allows a model to consume a compact compiled claim and permits an auditor or fallback mechanism to resolve that claim to the exact source span.

The paper distinguishes several artifact types, each evaluated on a different operational ledger. Navigation trees support organization and incremental maintenance; entity-relation graphs support cross-source discovery; ontologies provide vocabulary control and validation; wikis provide synthesis and onboarding; timelines provide temporal orientation; mind maps support navigation; and session memory supports continuity across interactions. The authors explicitly reject treating all of these artifacts as retrieval improvements. Their value depends on the deployment objective, not necessarily on benchmark answer accuracy.

Claim–evidence compilation

The quantitatively evaluated mechanism is a navigation tree whose leaves are claim–evidence pairs. Each pair contains a normalized atomic claim, a verbatim evidence span, a source-chunk identifier with character offsets, and validity metadata such as document time. A claim without a locatable source span is not retained.

This representation is intentionally not a summary. The claim provides a compact semantic unit for retrieval and context construction, while the verbatim span preserves auditability, citation, and fallback. The authors characterize the representation as lossy in prose but lossless in provenance. Extraction is performed by the relatively small glm-4-flash model; safety is attributed primarily to exact span validation rather than to reliance on a frontier model.

The extracted claim–evidence units are clustered into a hierarchical navigation tree. Internal nodes provide organizational labels, while leaves contain the evidence-bearing units. The tree is therefore treated as a carrier and organization mechanism, not as the primary retrieval algorithm.

Three serving modes are evaluated:

• Evidence-first retrieval (E1) searches claim text directly with BM25 and anchors the results to source chunks.
• Evidence substitution (E2) performs ordinary raw-text retrieval, then replaces a localized chunk with its claim–evidence pairs before the chunk enters the model context. A sufficiency gate decides whether the compact evidence is adequate or whether the raw chunk must be opened.
• Decision-gated reading (E3) uses a lightweight decision model to determine whether a localized chunk warrants deep reading. The decision model controls reading depth rather than candidate ranking.

The paper reports that direct evidence retrieval outperformed a document-then-tree route in its serving ablations. The latter often located the correct document but failed to reliably select the correct internal chunk, while adding an extra interaction round. This result supports a narrow role for hierarchy: it can organize evidence, but it should not replace flat lexical recall or force the agent through a potentially incorrect routing path.

Evaluation of structural artifacts

The controlled benchmark is FRAMES, a multi-hop question-answering dataset whose questions generally require information from multiple Wikipedia articles. The baseline harness uses BM25 and grep, with the answering model held fixed across conditions. The evaluation includes RAPTOR-style summary trees, PageIndex-style outlines, an LLM wiki, an entity-relation graph, and the claim-leaf tree.

The paper’s most important negative result is that structural compilation did not produce a detectable accuracy improvement on FRAMES. The RAPTOR-style summary tree was neutral. The PageIndex-style outline was neutral at small scale but negative at FRAMES scale. The LLM wiki was neutral and expensive to rebuild. The entity-relation graph was neutral because FRAMES already supplies entity-linked evidence and most questions do not require a cross-source relational join.

These findings support several mechanisms. First, the harness already performs adaptive search and can reformulate queries after inspecting evidence. A static partition must therefore outperform the agent’s next lexical query to improve recall. Second, summaries and outlines do not add factual evidence; they reorganize or paraphrase information already present in the corpus. Third, structural errors compound across navigation decisions. The paper gives an analytic illustration: with a branching factor of eight, an assumed 95% per-decision routing accuracy would yield approximately 75% end-to-end routing success over a ten-level path. This calculation is not a measured scale experiment and is presented as a mechanism explaining why confident wrong routing can be worse than open search.

The PageIndex result is particularly consequential because it contradicts the assumption that a document hierarchy is automatically beneficial for long-context retrieval. At FRAMES scale, an incomplete or misaligned hierarchy can concentrate the harness on the wrong documents and impose additional interaction costs. The paper does not generalize this observation into a universal failure law; it treats it as a benchmark-specific negative result requiring validation on other corpora.

Token economics of evidence substitution

The strongest positive result concerns token consumption rather than answer accuracy. On the 200-question FRAMES evaluation, evidence substitution reduced total query tokens by 91% when the projected evidence was judged sufficient. When the evidence was insufficient, token usage increased by 1.9%, because the system incurred the cost of projection and then read the raw chunk. Averaged across the benchmark, total tokens decreased by 38%, with no reliable accuracy change.

The asymmetry is central. Let the sufficiency rate be the fraction of queries for which the projected evidence is adequate. With a 91% saving on successful projections and a 1.9% penalty on failed projections, the paper derives an asymptotic break-even sufficiency rate of approximately 2%. The measured 38% average reduction implies a sufficiency rate near 43% under the paper’s cost model. Thus, the mechanism remains economically favorable even when most queries still require fallback to raw text.

The result should not be interpreted as evidence that every corpus should be compiled. The 2% threshold is an infinite-horizon condition that excludes compilation cost. For a finite deployment, the required sufficiency rate increases with compilation expense and decreases with the number of future queries. The authors report that, at an implied sufficiency rate of approximately 43%, a compilation process costing ten query-equivalents would amortize after roughly 26 queries. Conversely, a short-lived corpus or a corpus with expensive extraction may not recover its compilation cost despite exceeding the asymptotic threshold.

The analysis also identifies the miss penalty as the parameter requiring the most careful measurement. Because the successful-projection saving is large, uncertainty in that quantity has limited effect on the break-even point. The comparatively small failure penalty, however, can move the threshold substantially. This gives a concrete deployment prescription: measure failed projections and their downstream cost rather than relying only on average token savings.

Latency and the cost of compact context

Token efficiency is not equivalent to end-to-end efficiency. The paper reports a significant latency trade-off: on a paired subset of seven questions, claim-based substitution approximately halved query tokens but roughly tripled end-to-end latency. The mechanism is additional retrieval rounds. Compact evidence reduces the amount of material read per interaction, but it can also make each individual context incomplete, requiring the harness to perform more searches before reaching sufficiency.

This result qualifies the paper’s token-centered benefit. If latency is a hard production constraint, reducing context size may be an unfavorable exchange unless paired with a mechanism that suppresses unnecessary deep reads. The authors therefore distinguish evidence projection from decision gating. Projection reduces the amount of context admitted when evidence is sufficient; gating reduces the number of cases in which deep reading occurs at all.

The decision gate reduced latency from 140 seconds to 30 seconds in the reported configuration. With evidence present, the gate-arm mean reduced tokens by 52% relative to the uncompiled baseline; without compiled evidence, it reduced tokens by approximately 30% by declining deep reads that did not alter the answer. The paper cautions that the 52% figure is an arm mean and should not be compared directly with the 91% conditional hit saving or the 38% full-benchmark average.

The comparison of decision models also illustrates the importance of infrastructure-level control. The Qwen decision model produced no failed groups and completed its evaluation in 30.5 seconds, whereas the StartLux model produced 18 failed groups and required 560.1 seconds. The paper consequently uses the Qwen-based model as the default gate and treats the decision component as a specialized operational model rather than as a general-purpose answer generator.

Retrieval-harness design

The harness is designed around explicit invariants rather than unconstrained tool selection. It owns deterministic properties that an LLM is unlikely to enforce reliably, including token and time budgets, citation assignment, evidence prefetch, tool gating, removal of insufficient context, retries under degraded conditions, and guaranteed termination.

The query path is organized into five stations:

1. Capability check determines whether compiled artifacts exist and suppresses unsupported tools.
2. Evidence prefetch performs an initial corpus-wide lexical recall before the model begins tool-mediated exploration.
3. Evidence gating evaluates sufficiency using criteria such as evidence count, source diversity, and question coverage. Insufficient evidence is discarded rather than accumulated.
4. Tool-loop execution permits the model to use remaining admissible tools under parallel execution, step limits, and wall-clock budgets.
5. Deterministic close produces an answer from available evidence if the loop terminates abnormally.

This design explicitly assigns semantic judgments to the model and procedural guarantees to the harness. The model selects query terms, determines reading order, evaluates relevance, and identifies missing evidence. The framework enforces budgets, controls tool availability, manages citations, and handles abnormal termination. The authors argue that confusing these responsibilities is a common source of unreliable agentic retrieval.

The zero-hit protocol is another example of this division. A zero result is treated as a property of the tool invocation rather than as evidence that the corpus lacks the requested information. The harness explains the failure and retries through a fallback ladder, such as relaxing filters, switching from compiled to raw lexical search, or widening the time range. Only after these diagnostics can the model interpret the corpus as silent. This prevents infrastructure failures from becoming false semantic conclusions.

Artifact-specific value beyond benchmark accuracy

The paper is careful not to infer deployment benefits from FRAMES. It proposes separate evaluation ledgers for artifacts whose primary purpose is governance, integration, navigation, or continuity.

An entity-relation graph can support related-source discovery and cross-document joins, but FRAMES offers entity-linked evidence and contains relatively few questions requiring such joins. An ontology can normalize aliases, constrain extraction, support typed retrieval, validate relations, and integrate vocabularies across systems. These benefits are unlikely to appear in a benchmark where terminology is relatively controlled and source structure is fixed.

Timelines, mind maps, and wikis similarly target context organization rather than raw retrieval recall. A timeline can reduce temporal reconstruction effort and expose stale-version errors. A wiki can improve onboarding and synthesis, but the paper notes that fluent synthesized prose is inadequate for audit unless every statement remains linked to source evidence. Session memory targets continuity, such as reducing repeated questions and improving recovery of prior decisions, rather than multi-hop factual QA.

The deployment ledger specifies metrics such as tokens per correct answer, joined-source success, canonical entity-mapping rate, onboarding time, temporal reconstruction time, repeat-question rate, and citation traceability. However, the authors state plainly that no deployment row had yet met its preregistered target. These artifact-level claims are therefore architectural proposals and measurement protocols, not established production effects.

Limitations and open questions

The empirical conclusions are constrained by a single benchmark, one extraction model, one answering configuration, and descriptive aggregate measurements. The authors report no confidence intervals or repeated-seed experiments. The 91% hit saving, 1.9% miss penalty, 38% average reduction, and 43% implied sufficiency rate are measured operating points rather than population estimates.

FRAMES is English, Wikipedia-based, and entity-linked. Enterprise collections containing scans, tables, domain-specific terminology, access-control boundaries, or rapidly changing records may produce lower extraction quality and higher projection failure rates. The PageIndex degradation is observed at FRAMES scale but is not established as a general property of hierarchical navigation. Likewise, the reported 75% routing figure is an analytic illustration rather than a measured scale sweep.

The gate’s acceptance rate is also not equivalent to true evidence sufficiency. The paper distinguishes acceptance, actual sufficiency, and final-answer correctness, and introduces a false-accept rate that cannot be identified from token savings alone. A labeled audit of 100–400 accepted queries is proposed, but the paper does not report a completed benchmark-scale calibration. Consequently, the gate’s measured acceptance behavior should not be interpreted as a guarantee that admitted evidence is adequate.

Several questions remain open within the paper’s scope: whether claim–evidence compilation retains its token and latency trade-offs on non-Wikipedia enterprise data; how extraction validation behaves under tables, OCR noise, and long-range discourse dependencies; whether dense retrieval can help in corpora where lexical reformulation is insufficient; and how artifact value changes when the benchmark explicitly requires cross-source joins, temporal reasoning, ontology-constrained inference, or session continuity.

Conclusion

The paper presents knowledge compilation as an upstream context-supply layer for agentic retrieval rather than as a replacement for search. Its controlled FRAMES results show no detectable accuracy benefit from summary trees, outlines, wikis, or graphs under a competent BM25/grep harness. The measurable gain comes from claim–evidence projection: a 91% token reduction on sufficient-evidence cases, a 1.9% failure penalty, and a 38% average reduction without reliable accuracy loss. That saving can increase latency through additional retrieval rounds, while decision gating reduces the reported latency from 140 to 30 seconds.

The resulting architecture assigns distinct responsibilities to compilation, tools, and the harness. Compilers produce typed and provenance-bound artifacts; tools expose them through constrained interfaces; and the harness controls admission, fallback, budgets, citations, and termination. The paper’s strongest contribution is therefore not a new retrieval index but an operational account of how compiled evidence and disciplined agent control can make context a reusable systems resource.