Facts as First Class Objects: Knowledge Objects for Persistent LLM Memory

Abstract: LLMs increasingly serve as persistent knowledge workers, with in-context memory - facts stored in the prompt - as the default strategy. We benchmark in-context memory against Knowledge Objects (KOs), discrete hash-addressed tuples with O(1) retrieval. Within the context window, Claude Sonnet 4.5 achieves 100% exact-match accuracy from 10 to 7,000 facts (97.5% of its 200K window). However, production deployment reveals three failure modes: capacity limits (prompts overflow at 8,000 facts), compaction loss (summarization destroys 60% of facts), and goal drift (cascading compaction erodes 54% of project constraints while the model continues with full confidence). KOs achieve 100% accuracy across all conditions at 252x lower cost. On multi-hop reasoning, KOs reach 78.9% versus 31.6% for in-context. Cross-model replication across four frontier models confirms compaction loss is architectural, not model-specific. We additionally show that embedding retrieval fails on adversarial facts (20% precision at 1) and that neural memory (Titans) stores facts but fails to retrieve them on demand. We introduce density-adaptive retrieval as a switching mechanism and release the benchmark suite.

• The paper demonstrates that discrete knowledge objects overcome inherent in-context LLM memory limitations by ensuring 100% lossless retrieval at scale.
• It introduces a density-adaptive retrieval mechanism that switches from embedding similarity to exact key matching, achieving 100% P@1 accuracy on adversarial corpora.
• The KO architecture dramatically reduces production costs with O(1) retrieval efficiency, enabling robust multi-hop reasoning and cross-domain synthesis.

Summary of "Facts as First Class Objects: Knowledge Objects for Persistent LLM Memory" (2603.17781)

Introduction and Motivation

This paper critically examines the architectural assumptions underpinning persistent memory in LLM deployments, distinguishing between in-context memory—where facts, constraints, and decisions are serialized into the prompt—and external discrete memory architectures referred to as Knowledge Objects (KOs). Empirical evaluation reveals that within finite context windows, top-performing transformer models such as Claude Sonnet 4.5 can maintain 100% exact-match accuracy for up to 7,000 structured facts. Nevertheless, production settings expose fundamental limitations: hard capacity boundaries, irreversible compaction loss, and insidious goal drift. The paper demonstrates that KO architectures eliminate these failure modes by storing facts as hash-addressed tuples for $O(1)$ retrieval and introduces density-adaptive retrieval as a principled switching mechanism to address embedding-based adversarial fact discrimination.

In-Context Memory: Scaling and Failure Modes

Experimental benchmarking with synthetic pharmacological corpora confirms that Claude Sonnet 4.5 achieves perfect fact retrieval across nearly its entire 200K-token window, even amidst high semantic density (pairwise cosine similarity > 0.95) and confusable fact sets. However, this success is strictly bounded by the context limit: at $N=8,000$ facts, prompt overflow induces an abrupt retrieval cliff. GPT-4o exhibits brittle performance, dropping to 0% accuracy at $N=3,000$ under similar testing conditions.

Figure 1: Scaling curve: exact-match accuracy versus corpus size ($N=10$ to $N=10,000$), with Claude Sonnet 4.5 maintaining 100% accuracy through $N=7,000$ and KO maintaining 100% regardless of $N$.

Beyond the context window, production use cases necessitate context compaction—summarization of outdated facts to reclaim prompt space—which causes catastrophic loss: 60% of facts become irrecoverable after a single 36.7x compaction pass.

Figure 2: Fact retrieval accuracy after 36.7x compaction. In-context memory loses 60% of facts; KO maintains 100% fidelity.

Of equal practical consequence, compaction erodes project-specific goals and behavioral constraints. Serial compaction in an 88-turn session eliminates over half of non-default constraints after three rounds, with the model silently reverting to reasonable defaults and no error signaling.

Figure 3: Goal drift under cascading compaction. KO maintains 100% constraint preservation regardless of compaction, while in-context memory rapidly decays.

Knowledge Object Architecture

KOs formalize memory as discrete $(subject, predicate, object, provenance)$ tuples, addressable via deterministic hash functions and stored externally. The LLM’s role is restricted to query parsing and answer generation rather than serving as the memory substrate. This separation ensures lossless storage and $O(1)$ retrieval regardless of corpus expansion, with per-query costs remaining flat as corpus size scales.

KO retrieval operates robustly across noisy contexts and supports ingestion both from structured databases and conversational extraction. Parsing into normalized keys achieves high accuracy except with adversarial phrasing, remediable via engineering synonymization.

Density-Adaptive Retrieval

Embedding-based retrieval in traditional RAG architectures fails on adversarial corpora—sets where near-identical embeddings correspond to factually distinct values—yielding only 20% P@1. The proposed density-adaptive mechanism measures average pairwise similarity in the candidate set, switching to exact structured key matching beyond a learned threshold ($\tau=0.85$). This hybrid approach attains 100% P@1 on adversarial benchmarks while matching embedding retrieval in conventional settings.

Multi-Hop Reasoning and Cross-Domain Synthesis

KO architectures facilitate multi-hop compositional reasoning, exceeding in-context baselines (78.9% vs. 31.6% accuracy on 2-hop queries). KO grounding substantively improves cross-domain generative synthesis, with groundedness scores increasing by +118%.

Figure 4: Multi-hop reasoning accuracy: KO achieves 78.9% accuracy on 2-hop queries, outperforming in-context memory.

Figure 5: Cross-domain synthesis quality scores. KO retrieval substantially increases groundedness and specificity.

Production Economics

KO architectures offer orders-of-magnitude reductions in compute and cost. For $N=7,000$ facts and 25,000 annual queries, in-context memory costs $14,201/year$ compared to KO’s $56/year$, a 252x reduction. The economic argument extends as corpus size increases, with in-context memory becoming infeasible beyond $N=100,000$.

Figure 6: Production economics. Per-query cost grows linearly with $N$ for in-context memory and remains flat for KO; annual cost disparities are substantial.

Theoretical Implications and Architectural Generalization

Empirical results provide evidence that context rot is a systemic issue, not a model-specific limitation. Compaction loss, goal drift, and adversarial vulnerability are intrinsic to prose-based memory. Cross-model replication confirms architectural, not parametric, causality. KO architectures circumvent the Orthogonality Constraint described by (Beton et al., 14 Jan 2026), avoiding interference by discrete, hash-addressed storage rather than dense parametric embedding.

Neural memory baselines such as Titans achieve perfect memorization but exhibit rapid retrieval degradation as corpus size increases, confirming the theoretical tradeoff predicted by the Orthogonality Constraint.

Figure 7: Titans neural memory vs. KO. KO maintains 100% retrieval accuracy across corpus sizes; Titans' accuracy degrades substantially on forced-choice and free-form completion tasks.

Limitations and Future Directions

The paper’s methodology relies on synthetic structured corpora, though robustness to noisy contexts and complex queries is demonstrated. Predicate normalization and query phrasing remain areas for engineering improvement. Density-adaptive thresholds may require tuning per embedding model.

Future work should focus on KO extraction from unstructured conversational data, scaling multi-hop reasoning, and integrating KO with existing RAG pipelines. The broader implication is a need to treat project-specific constraints and alignment decisions as first-class objects, especially given their criticality and vulnerability to compaction loss.

Conclusion

This work establishes that in-context memory in frontier LLMs is highly performant within the prompt boundary but fundamentally flawed for persistent, scalable, or precision-critical applications. KO architectures resolve all identified production failure modes—capacity, compaction, goal drift, adversarial retrieval, and economics—by separating storage and processing. The density-adaptive retrieval mechanism provides principled switching between embedding similarity and exact key matching, ensuring robustness even on adversarial corpora. Cross-domain synthesis and compositional reasoning are materially improved with KO grounding. The implication is that reliable LLM agents require persistent, structured, and discrete memory architectures—a requirement that scaling context windows alone cannot satisfy.