PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents
Abstract: LLM agents increasingly operate over long and recurring external contexts, like document corpora and code repositories. Across invocations, existing approaches preserve either the agent's trajectory, passive access to raw material, or task-level strategies. None of them preserves what we argue is most needed for repeated same-context workloads: reusable orientation knowledge (e.g., what the context contains, how it is organized, and which entities, constants, and schemas have historically been useful) about the recurring context itself. We introduce PEEK, a system that caches and maintains this orientation knowledge as a context map: a small, constant-sized artifact in the agent's prompt that gives it a persistent peek into the external context. The map is maintained by a programmable cache policy with three modules: a Distiller that extracts transferable knowledge from inference-time signals, a Cartographer that translates it into structured edits, and a priority-based Evictor that enforces a fixed token budget. On long-context reasoning and information aggregation, PEEK improves over strong baselines by 6.3-34.0% while using 93-145 fewer iterations and incurring 1.7-5.8x lower cost than the state-of-the-art prompt-learning framework, ACE. On context learning, PEEK improves solving rate and rubric accuracy by 6.0-14.0% and 7.8-12.1%, respectively, at 1.4x lower cost than ACE. These gains generalize across LMs and agent architectures, including OpenAI Codex, a production-grade coding agent. Together, these results show that a context map helps long-context LLM agents interact with recurring external contexts more accurately and efficiently.
Summary
- The paper introduces a context map as an orientation cache that persistently abstracts and organizes external context for LLM agents.
- The paper details a modular design comprising a Distiller, Cartographer, and Evictor to extract, structure, and manage contextual knowledge.
- The paper demonstrates significant improvements with up to 34% accuracy gains and 1.7–5.8× cost reduction across long-context benchmarks.
Peek: Context Map as an Orientation Cache for Long-Context LLM Agents
Motivation: The Limits of Existing Context Management
Modern LLM-based agents increasingly operate over large, persistent corpora such as document repositories or codebases, handling sequences of tasks that target the same external context. Current context management approaches—storing full chat histories, performing history compaction, retrieval-augmented generation (RAG), offloading, and prompt-learning—primarily address preservation of agent trajectory, access to raw or summarized material, or task-level policies. Critically, none of these techniques maintain a concise, transferable, and actively curated abstraction of orientation knowledge: persistent contextual understanding about what the external context contains, how it is organized, and which entities, constants, and schemas have proven instrumental across queries.
Peek addresses this missing piece by introducing an active, bounded, and agent-resident artifact: the context map. This map acts as a persistent orientation cache, explicitly storing knowledge that remains relevant across sessions and queries—knowledge akin to the navigation aids and structural memos a human analyst would maintain when repeatedly drilling into a large corpus.
Figure 1: The context map places agent context management into the active external-context quadrant, a previously vacant niche among agent design strategies.
System Design: Peek Architecture and Cache Management
Peek represents the context map as a constant-size text segment embedded within the prompt, persisted and evolved as the agent iterates upon the external context. Maintenance of the map is implemented through a modular policy composed of three core components:
- Distiller: Extracts transferable, context-level knowledge from inference-time diagnostic signals obtained by analyzing the full agent execution trajectory.
- Cartographer: Converts the Distiller’s abstracted findings into precise, structured modifications (additions, deletions, or edits) of the context map, maintaining stable identifiers for consistency across updates.
- Evictor: Enforces a strict token budget via priority-based eviction, ensuring bounded prompt overhead by pruning least-useful items based on value hierarchy and recency.
Figure 2: Peek’s modular architecture: knowledge is distilled, structured, and maintained within a context map, subject to size constraints.
The context map is initialized sparsely, with sections for Context Roadmap (abbreviated index), Context Understanding (entity/concept abstraction), optional Domain Constants, Reusable Results, and Parsing Schema. All map content is discovered organically from agent experience, not crafted in advance.
Figure 3: Example structure of a context map generated by Peek, demonstrating stable IDs and organization across sections.
The cache management procedure is deliberately separated into knowledge extraction (Distiller) and editing (Cartographer), a design found essential via ablation studies for preventing leakage of task-specific facts, redundancy, and noise accumulation into the map. The Evictor consistently prevents context window bloat.
Experimental Evaluation
Peek is evaluated on long-context reasoning and context-learning settings, specifically OOLONG (requiring distributed evidence aggregation) and CL-bench (context learning and application). Baselines include Recursive LLMs (RLM), RAG, compaction agents (MemAgent), and state-of-the-art prompt-learning via Agentic Context Engineering (ACE).
Key empirical findings:
- Accuracy gains: Peek exceeds all baselines on every task and metric. On OOLONG, it delivers 6.3–34.0% absolute improvement. On CL-bench, it improves solving rate and rubric accuracy by up to 14.0% and 12.1%, respectively, over strong prompt-learning methods.
- Efficiency: Peek achieves these gains with dramatically fewer agent iterations (93–145 fewer than ACE) and strong computational cost reduction (1.7–5.8× lower on OOLONG).
- Pareto efficiency: Peek consistently lies on the Pareto frontier for both quality vs. iterations and quality vs. monetary cost across all benchmarks.
Figure 4: Peek achieves the highest performance across multiple long-context benchmarks when compared to RLM, RAG, MemAgent, and ACE.
Figure 5: In both score-vs-iteration and score-vs-cost plots, Peek occupies the Pareto frontier, outperforming all baselines in efficiency and effectiveness.
Results generalize across LLM backbones (GPT-5-mini, GPT-5.5, Qwen3-Coder-Next-FP8) and agent architectures (RLM, Codex), demonstrating that the context map approach is robust to both model scale and differing agent paradigms.
Ablation and Analysis
Ablations reveal several design insights:
- No eviction (static map): Large gains are preserved even without eviction; dynamic maintenance adds an extra 10%+ improvement.
- Collapsed pipeline (no separation of Distiller/Cartographer): Drops performance by 7.7% on average, confirming that modular analysis and editing are necessary.
- Map size: While defaulting to tokens, significant gains persist across budgets from 512 to 2048, indicating the structure and curation matter more than absolute map size.
Discussion, Limitations, and Implications
Peek’s benefit is contingent on the agent’s capacity to surface transferable knowledge from context interactions; if the agent’s usage is myopic or the external context is structurally unsuited to abstraction, map utility is limited. The approach is fundamentally orthogonal to KV-cache optimization, which operates at the model-level for serving efficiency and does not encode semantic knowledge or orient future policy.
From a theoretical perspective, Peek advances the semantic abstraction layer in agent design, proposing an explicit, structured policy for knowledge caching that goes beyond passive retrieval or compaction. Practically, it delivers a substantial return in computational efficiency and output correctness, particularly as context size and reusability demands increase in persistent-task regimes typical of real-world deployments.
Figure 6: CL-bench leaderboard snapshot demonstrating Peek’s competitive performance, rivaling much larger models simply by adding agent-side context mapping.
Future work indicated by the authors includes trainable cache policies, adaptive budgeting, the development of new benchmarks to better target repeated-same-context workloads, and the exploration of richer reusable artifacts for hierarchical or federated agent systems.
Conclusion
Peek introduces an effective paradigm for long-context agentic interaction by persistently caching orientation knowledge in a context map continually refined through modular analysis and priority maintenance. This approach yields substantial improvements in accuracy, sample efficiency, and cost across diverse benchmarks and model architectures, filling an essential gap in current context management solutions for LLM agents. The work highlights the necessity of agent-level, semantically-structured caches—a direction likely to become fundamental as long-context deployments and persistent external knowledge scenarios proliferate.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
What is this paper about?
Imagine you’re exploring a giant library again and again to answer different questions. Instead of wandering from scratch every time, you’d keep a small, trusty map: where things are, what’s important, and shortcuts that worked before. This paper builds that kind of “map” for AI assistants that read and work with very large collections of text or code. The system is called Peek, and its “context map” is a tiny set of notes the AI keeps in its prompt to remember how a big, recurring collection is organized and what’s useful inside it.
What questions did the researchers ask?
The paper focuses on two simple questions:
- If an AI can only keep a small “map” of a huge collection, what should go into that map?
- Does keeping and updating this kind of map actually help the AI answer future questions faster, cheaper, and more accurately?
How did they approach it?
The authors built a system called Peek that maintains a small, always-present “context map” inside the AI’s prompt. Think of it like:
- A tiny table of contents for a huge book collection
- Notes about important names, data formats, and where certain facts are likely to be found
- A few reusable results or definitions that often come up
This map stays small on purpose so it’s quick to read and doesn’t crowd out the AI’s normal thinking space.
The “context map” (what’s inside)
The map can include:
- Context Roadmap: a quick guide to what the collection contains and where things live
- Context Understanding: key people, concepts, and how they relate
- Optional bits:
- Domain Constants: exact values that matter (like fixed category names)
- Reusable Results: answers or counts that are helpful again
- Parsing Schema: notes on how the text is formatted (e.g., “each entry starts with ‘Title:’”)
This is not a chat history or a how-to plan for one task. It’s more like a map of the world you keep returning to.
Three helpers that keep the map clean and useful
Peek uses three modules that you can think of as a small team:
- Distiller: like a detective who reviews how the AI worked on a problem and pulls out only the general, reusable knowledge about the collection (not one-time tricks).
- Cartographer: like a map maker who turns those findings into structured edits to the map—adding, updating, or removing entries with clear IDs so future changes stay organized.
- Evictor: like a backpack weight limit. If the map gets too big, the Evictor removes the least useful items first, keeping the most valuable notes within a fixed token budget.
This “cache policy” runs for the first few tasks to build up a good map, then can be frozen and reused for later tasks.
What did they find?
The team tested Peek on challenging tasks where an AI has to repeatedly work with long documents or rules and combine evidence spread across many parts. They compared Peek to other popular methods like:
- Keeping long chat histories
- RAG (retrieval) systems that pull in matching chunks on the fly
- Compaction systems that summarize large contexts
- Prompt-learning systems that store step-by-step strategies (like ACE)
Across several benchmarks, Peek:
- Improved accuracy:
- Long-context reasoning tasks: +6% to +34% better than baselines
- Context-learning tasks (learning rules from a shared context): +6% to +14% higher solving rate and +8% to +12% higher rubric accuracy
- Needed fewer steps: often 93–145 fewer iterations than a strong prompt-learning method
- Cut costs: typically 1.7× to 5.8× cheaper than that prompt-learning method on some tasks
- Worked broadly: gains showed up with different AI models and different agent frameworks, including a production coding agent
Why does this happen? Because the “context map” gives the AI a persistent understanding of the collection itself—what’s inside, how it’s structured, and what has been useful before—so it wastes less time rediscovering the same facts.
A few extra observations:
- Simply stacking past chats often adds noise and can hurt performance.
- Pulling relevant chunks (RAG) helps in organized settings but struggles when the collection is messy or tasks are complex.
- Summaries can help a little but don’t consistently boost detailed correctness.
- Separating the Distiller and Cartographer matters: when combined into one step, results got worse.
- The exact size of the map isn’t critical; even a small budget helped a lot. Freezing the map (no more edits) still beat no map at all, but active maintenance performed best.
Why does this matter?
- For companies: Analysts who repeatedly query the same customer feedback or logs can get faster, more accurate answers with lower costs.
- For coding agents: When working in the same codebase again and again, a map helps the agent remember file layouts, key functions, and constants without re-learning them.
- For students and researchers: It’s like keeping organized notes that make future study sessions much more efficient.
In short, Peek turns repeated work over the same large collection from “start over every time” into “reuse the best orientation notes.”
Final takeaway
Peek introduces a simple but powerful idea: give AI agents a small, well-managed “orientation cache”—a context map—that captures how a big, recurring collection is organized and what’s most useful inside it. This map is kept short, updated smartly, and pruned to stay within a fixed budget. The result is better answers, fewer steps, and lower costs across different tasks, models, and agent designs. It’s a practical way to make long-context AI work more like an experienced librarian than a first-time visitor.
Knowledge Gaps
Unresolved Knowledge Gaps, Limitations, and Open Questions
Below is a single, concrete list of what remains missing, uncertain, or unexplored in the paper, written to guide future research.
- Formalization of “orientation knowledge”: No precise definition, objective function, or measurable properties of what should optimally be stored in a context map; lacks theory for what constitutes valuable, transferable orientation vs. task-specific facts.
- Optimality of map content and structure: No information-theoretic or algorithmic analysis of what map sections (Roadmap, Understanding, Domain Constants, Reusable Results, Parsing Schema) are necessary/sufficient under varying context/task distributions.
- Adaptive budget selection: The token budget B is fixed and only lightly ablated; no method to adapt B to context complexity, task mix, or model/tokenizer differences, nor a policy for dynamic resizing over time.
- Automatic stopping criterion for evolution (m): The number of evolve steps m is user-set; no principled method to detect convergence or “sufficient knowledge” to freeze updates without hurting performance.
- Handling evolving or versioned external contexts: Limited treatment of corpus drift or refresh (new/deleted documents, code changes); lacks mechanisms to detect stale map entries, trigger re-crawls, or reconcile conflicting versions.
- Robustness to Distiller errors: Distiller relies on trajectory-only signals without ground truth, verification, or rollback; no safeguards against injecting spurious, adversarial, or biased knowledge into the persistent map.
- Security and prompt-injection defenses: No adversarial evaluation or mitigations for contexts that embed prompt injections or malicious patterns that could poison the map during Distiller/Cartographer updates.
- Multi-user isolation and privacy: Persisting maps across tasks raises leakage risks between users/tenants; there is no policy for access control, redaction, or scoped isolation when contexts are proprietary or sensitive.
- Concurrency and synchronization: No design for concurrent agent sessions updating the same map (locking, conflict resolution, CRDTs) or for merging maps from parallel runs on the same context.
- Portability across agents/models: Map utility is said to depend on how agents interact with context; no study of cross-agent transfer, map transformation between backbones, or model-agnostic distillers.
- Map reuse and transfer learning across similar contexts: Unexplored whether a learned map from one corpus can bootstrap orientation in a related corpus (e.g., “warm-start” maps, templates, or domain ontologies).
- Combining with RAG/offloading/compaction: Only individual comparisons are shown; no evaluation of synergies (e.g., using the map to steer retrieval, to condition compaction, or to learn index schemas).
- Scaling to extreme contexts and long task sequences: Benchmarks cover moderate-length contexts and ≤12 tasks per context; no evidence for stability, drift, or performance over hundreds of tasks or million-token corpora.
- Latency and wall-clock performance: Cost and iteration counts are reported, but not end-to-end latency or variance; no micro-benchmarks of Distiller/Cartographer/Evictor overhead under real-time constraints.
- Map stability and churn: No metrics or analysis for edit churn, eviction oscillations, duplicate/thrashing entries, or long-term stability of the cached knowledge.
- Eviction policy optimality: Section-priority and age-based tie-breaking are heuristic; lacks learned or theoretically grounded eviction strategies and sensitivity studies across task types.
- Distiller/Cartographer diagnostics: No measurement of extraction precision/recall, dedup effectiveness, false positives/negatives, or impact of diagnosis errors on downstream performance.
- Section-level ablations: While cache size and pipeline separation are ablated, there is no section-by-section ablation to quantify which map components drive gains or when sections should be omitted.
- Cold-start and low-signal runs: If early trajectories are weak (e.g., frequent failures), how to bootstrap reliable map entries is not addressed; no fallback strategies or curriculum for map seeding.
- Initialization from metadata/schemas: The map starts empty by design; unexplored trade-offs of initializing from known metadata (e.g., TOCs, schemas, repository structure) vs. pure online discovery.
- Tokenizer and model-dependence: Budgets are in tokens, but cross-model tokenization differences can impact capacity and sectioning; no normalization strategies for cross-LM portability.
- Evaluation breadth and realism: Main results rely on OOLONG and CL-bench; missing large, real-world deployments (e.g., enterprise corpora, evolving codebases) with human-in-the-loop assessments.
- Judge-model bias on CL-bench: Uses GPT-5.1 as judge without cross-judge validation; no robustness checks against evaluation bias or rubric misalignment.
- Frontier-model scalability and cost: Full-scale experiments with GPT-5.5 are limited by cost; uncertainty remains on total cost of ownership and viability for production with high-throughput workloads.
- Multimodal and structured contexts: Unexplored extension to images, tables, graphs, logs, or databases (e.g., storing schemas, keys, or graph summaries) and tool-integrated navigation (e.g., ASTs for code).
- Domain-specific variants: For code, data analytics, legal corpora, or scientific papers, the ideal map sections and edit policies may differ; no domain-tailored designs or evaluations.
- Safety and compliance: Persisting “Domain Constants” or “Reusable Results” may capture sensitive or regulated information; lacks compliance-aware caching, retention policies, or audit trails.
- Failure modes and error taxonomy: No systematic analysis of when Peek degrades (e.g., highly heterogeneous contexts, frequent task-switching, limited orientation signal), nor guidance for detection/mitigation.
- Automatic context identity and scoping: How maps are bound to a specific external context is unspecified (hashing, content signatures, URIs); no collision handling or cross-context contamination checks.
- Interaction with KV-cache and decoding optimizations: Claimed orthogonal but untested; no empirical study of joint gains or interference when combining with KV compression, speculative decoding, or caching.
- Human oversight and rollback: No provisions for human review of map edits, A/B testing, or safe rollback after degradations; lacks governance mechanisms for persistent artifacts.
- Parameter sensitivity: Limited sweeps for hyperparameters (e.g., top-k candidates, scoring thresholds, evolve cadence); no guidance on tuning for new contexts or models.
- Cross-lingual and multilingual contexts: Unexplored performance on non-English corpora or mixed-language contexts, where section design and Distiller prompts may require adaptation.
- Robustness to noisy/unstructured data: No targeted evaluation on messy, redundant, or contradictory corpora and the risk of caching incorrect or misleading patterns.
Practical Applications
Immediate Applications
Below are practical, deployable use cases that can be implemented with current LLM agents by adding Peek’s context map and cache policy (Distiller, Cartographer, Evictor) to their prompts and workflows.
- Enterprise analytics over recurring corpora (customer feedback, support tickets, product reviews)
- Sector: software/business intelligence
- What to deploy: A “Peek-Map” plugin for existing RLM/RAG pipelines that creates a Context Roadmap (sources, segments, topic clusters), Domain Constants (product names, SKUs, categories), and Reusable Results (common aggregates, sentiment baselines).
- Workflow: Run m=1–4 analysis tasks to populate the map; freeze updates; reuse the map across subsequent ad-hoc queries to cut iterations and cost.
- Assumptions/dependencies: A stable, recurringly queried corpus; agent execution logs available; data access and privacy controls; modest token budget (e.g., B≈512–1024) suffices.
- Coding assistants that “warm-start” on large codebases
- Sector: software engineering
- What to deploy: Integrate Peek with agents like Codex/Cursor to cache subsystem layout, key APIs, config files, test directories, schema/version constants, and parsing patterns (build files, log formats).
- Workflow: Trigger map refresh during onboarding or CI; add Cartographer edits after code search/refactor tasks; set Evictor to protect Roadmap/Understanding over low-value schema entries.
- Assumptions/dependencies: Stable repository structure; agent/tool-call traces; access control for private repos; maps stored per branch or release.
- Knowledge base Q&A for customer support and internal IT
- Sector: customer support/IT ops
- What to deploy: Map-driven retrieval where the Context Roadmap informs where answers live (FAQs, runbooks, incident postmortems), with Domain Constants for product tiers and SLAs.
- Workflow: Use Distiller to tag stale KB entries; Evictor removes outdated procedures; Reusable Results keep “known-good” steps and diagnostics.
- Assumptions/dependencies: Reasonably organized KB; basic embedding/RAG already in place; regular content updates.
- Compliance and policy assistants
- Sector: legal/compliance/governance
- What to deploy: Context maps over statutes, internal policies, audit controls; cache thresholds, definitions, enumerations, and frequent cross-references.
- Workflow: Use Distiller to separate general policy orientation from case-specific logic; Cartographer emits structured links to primary sources.
- Assumptions/dependencies: Up-to-date corpora; human-in-the-loop review; careful handling of legal interpretations and disclaimers.
- Financial document analysis (SEC filings, earnings calls, prospectuses)
- Sector: finance
- What to deploy: Orientation caches for issuer-specific filings across years; Domain Constants for fiscal calendars, segment names, KPI definitions; Parsing Schema for table formats.
- Workflow: Pre-build maps per ticker; freeze maps; reuse for quarterly analysis to lower cost and increase rubric accuracy.
- Assumptions/dependencies: Accurate ingestion; compliance/security; recurring queries across similar filings.
- Clinical guideline and EHR cohort analysis
- Sector: healthcare
- What to deploy: Context maps summarizing guideline sections, code sets (ICD/CPT), medication classes, and local formulary constants; Parsing Schema for EHR note templates.
- Workflow: Use Distiller on chart-review trajectories (with de-identified logs) to add reusable navigational cues and domain constants.
- Assumptions/dependencies: HIPAA-compliant environment; strictly de-identified execution traces; clinician oversight.
- Course and program assistants for universities and training providers
- Sector: education
- What to deploy: Maps for syllabi, schedules, grading rubrics, assignment templates; Reusable Results for common prerequisite checks and formula sheets.
- Workflow: Populate maps during course setup; reuse across student Q&A and grading support; Evict outdated modules each term.
- Assumptions/dependencies: Stable course materials; LMS integration; fairness and academic integrity review.
- Literature review accelerators for research labs
- Sector: academia/scientific research
- What to deploy: Context maps across a subfield: seminal papers, datasets, evaluation metrics, commonly used baselines; Reusable Results that cache computed comparisons or extracted tables.
- Workflow: Distiller tags helpful/harmful papers; Cartographer maintains minimal edit sets to avoid duplication.
- Assumptions/dependencies: Access to full texts; recurring queries within the same topic; periodic refresh as new publications arrive.
- Log and telemetry analysis for data/ML engineers
- Sector: data engineering/observability/MLOps
- What to deploy: Parsing Schema for log formats, Context Roadmap of services/components and their log sources, Domain Constants for status codes and operational thresholds.
- Workflow: Use the map to orient anomaly analysis and root-cause queries; freeze after initial layout discovery to speed future incidents.
- Assumptions/dependencies: Consistent logging conventions; access to observability data; secure environment.
- Enterprise knowledge management and internal search
- Sector: enterprise software/knowledge ops
- What to deploy: A lightweight “Orientation Cache” to guide search over Confluence/SharePoint/GitHub Wiki; cache key spaces, document types, and ownership.
- Workflow: Map built from a handful of navigation tasks; guide RAG chunking and retrieval with Roadmap entries.
- Assumptions/dependencies: Existing search/RAG indexers; governance over map sharing; recurring queries on the same content.
- Project documentation assistants
- Sector: product/PM
- What to deploy: Maps of RFCs, decision records, dependency graphs; Domain Constants for project codes, timelines, acceptance criteria.
- Workflow: Distiller derives orientation from planning and postmortem sessions; Evictor removes stale milestones.
- Assumptions/dependencies: Document hygiene; access to PM tools; recurring queries during the project lifecycle.
- Personal knowledge bases and productivity
- Sector: daily life/software productivity
- What to deploy: Personal “Peek-Map” for notes, recipes, finances, travel plans; Parsing Schema for spreadsheets and budget templates; Domain Constants for recurring bills.
- Workflow: Build once, reuse for queries like “Where did I record X?” or “Compare this year’s spend to last.”
- Assumptions/dependencies: Personal data kept local; privacy-by-default storage; small token budgets are sufficient.
Long-Term Applications
These use cases require further research, scaling, productization, or ecosystem development before broad deployment.
- Multimodal context maps (text, tables, code, diagrams, images)
- Sector: software, manufacturing, robotics, healthcare
- Potential product: “Map-of-Maps” supporting schema-aware cross-modal orientation (e.g., design drawings + spec documents).
- Dependencies: Multimodal LLMs/tools; unified identifiers; robust Cartographer for heterogeneous edit operations.
- Standardized context-map schemas and registries across teams/organizations
- Sector: enterprise software/governance
- Potential product: Map Registry/Exchange with access controls and lineage; shared orientation artifacts for onboarding and audits.
- Dependencies: Interoperable formats; permissioning; provenance tracking; adoption of an API standard.
- Map-driven retrieval and indexing (adaptive RAG guided by orientation caches)
- Sector: search/information retrieval
- Potential product: Retrievers that consult Roadmap entries to steer chunking, re-ranking, and diverse evidence aggregation.
- Dependencies: Connectors between maps and vector indexes; policy for updating retrievers when maps evolve.
- Continual learning with map feedback (optimize maps with RL or supervised signals)
- Sector: ML ops/research
- Potential product: Distiller reward models tuned to long-context efficiency; automated map-quality metrics.
- Dependencies: Reliable signals (rubric accuracy, iteration counts, tool success); labeled or weakly labeled feedback loops.
- Cross-agent memory OS (MapOps)
- Sector: agent platforms
- Potential product: A shared memory layer (MemGPT + Peek) where multiple agents consume/update the same context map safely.
- Dependencies: Concurrency control; conflict resolution; per-agent role-based permissions; standardized edit contracts.
- Auditable, policy-grade AI reasoning with map snapshots
- Sector: government/regulatory policy
- Potential product: Orientation maps as audit artifacts to explain which parts of a corpus informed decisions (FOIA, public records).
- Dependencies: Immutable snapshots; provenance and hash-based attestation; human oversight; legal frameworks.
- Privacy-preserving maps (e.g., differential privacy or redaction-aware Evictor)
- Sector: healthcare/finance/public sector
- Potential product: Map policies that auto-redact sensitive fields or add DP noise to Domain Constants/Reused Results.
- Dependencies: Formal privacy guarantees; data classification; secure execution traces.
- Autonomous map maintenance in dynamic contexts
- Sector: DevOps, customer support, news/media analytics
- Potential product: Scheduled agents that refresh maps when sources change (nightly builds, KB updates, new filings).
- Dependencies: Change detection; incremental map updates; budget-aware eviction; monitoring of drift.
- Hardware–software co-design: combining KV-cache optimizations with context maps
- Sector: inference systems/performance engineering
- Potential product: Serving stacks where model-level token caching and agent-level orientation caches jointly reduce cost.
- Dependencies: Runtime support for both; measurement harnesses; task-specific tuning.
- Orientation Cache SaaS and “Map Studio”
- Sector: enterprise platforms
- Potential product: Hosted map services with GUI editing, diagnostics, and policy configuration (Distiller scoring, Evictor thresholds).
- Dependencies: Customer data connectors; role-based access; billing; SLAs; integration SDKs for popular agents.
Notes on Feasibility and Dependencies
- Transferability assumption: Peek’s gains rely on tasks recurring over the same external context so orientation knowledge is reusable; benefits drop if every task shifts to a different corpus.
- Agent interaction dependency: Distiller quality depends on rich execution traces (tool calls, reasoning steps); silent or minimal traces reduce map signal.
- Token-budget constraint: Maps should be small (e.g., 512–1024 tokens); larger budgets can help but the presence of a curated map matters more than exact size.
- Privacy and compliance: Maps may accumulate sensitive constants; deploy with strict access control and redaction where needed.
- Model/agent variability: While gains generalize, usefulness can vary with agent architectures and domain (e.g., code-centric models underperform on context learning).
- Update policy: Freezing maps after early evolution (m≈1–4) yields most benefits at lower cost; frequent updates are unnecessary for stable corpora.
Glossary
- ACE (Agentic Context Engineering): A state-of-the-art prompt-learning framework that evolves and curates prompts via reflection and curation to improve task performance without weight updates. "ACE is the current state-of-the-art framework for prompt learning,"
- Agent skills: Reusable, task-facing abilities or tools maintained for agents to execute procedures or plans. "agent skills"
- Agentic plan caching: Storing and reusing previously successful task plans or procedures to accelerate future tasks. "agentic plan caching"
- Cartographer: Peek’s module that converts distilled insights into structured Add/Delete/Replace edits for the context map. "a Cartographer that translates it into structured edits,"
- CL-bench: A benchmark for context learning with multiple related tasks per context, evaluated by solving rate and rubric accuracy. "On CL-bench, Peek improves solving rate and rubric accuracy by 6.0--14.0\% and 7.8--12.1\%, respectively, at 1.4 lower cost than ACE."
- Compaction Agent: A system (e.g., MemAgent) that summarizes or condenses external contexts so they fit into the model’s context window. "Compaction Agent improves coarse success but barely improves CL-bench rubric accuracy."
- Context compaction: Methods that condense large external contexts into smaller summaries to fit the context window. "context-compaction methods"
- Context learning: Acquiring task-relevant knowledge from an external context and applying it across related tasks. "(2) context learning~\cite{dou2026cl}, which requires acquiring task-relevant knowledge from the external context and applying it across tasks."
- Context map: A small, persistent, prompt-resident cache that stores reusable orientation knowledge about a recurring external context. "a context map: a small, constant-sized artifact in the agent's prompt"
- Context offloading: Externalizing large contexts into an environment the agent can inspect and query during inference. "offloading it into an external environment that the agent inspects, slices, decomposes, and queries at inference time"
- Context Roadmap: A section of the context map that serves as an abbreviated index of what the external context contains and where. "the Context Roadmap, an abbreviated index of what the external context contains and where"
- Context Understanding: A context map section that captures higher-level descriptions, entities, and relationships within the external context. "the Context Understanding which consists of a higher-level description of the context including key entities, concepts, and their relationships."
- Context window: The maximum number of tokens a LLM can attend to in a single prompt. "live outside the LLM's context window"
- Distiller: Peek’s module that extracts transferable contextual knowledge from execution signals and labels map entries for update. "a Distiller that extracts transferable knowledge from inference-time signals"
- Domain Constants: A context map section reserved for exact values or enumerated sets that frequently appear in the context. "Domain Constants for exact values or enumerated sets"
- Evictor: Peek’s module that enforces the fixed token budget by evicting low-priority or stale map items. "a priority-based Evictor that enforces a fixed token budget"
- Execution trajectory: The sequence of reasoning steps, tool calls, and observations produced during an agent’s run. "Each agent run produces an execution trajectory:"
- Inference-time signals: Diagnostics and cues available during model execution used to distill transferable knowledge. "from inference-time signals"
- KV-cache: A model-level optimization that stores transformer key/value states to speed up or compress attention computations. "Unlike KV-cache management, which is a model-level optimization that reuses or compresses token states,"
- OOLONG: A long-context reasoning and aggregation benchmark used to test distributed evidence identification and combination. "On OOLONG, Peek beats the SOTA prompt-learning method ACE by --."
- Pareto frontier: The set of methods that are not dominated on two competing metrics (e.g., quality vs. iterations or cost). "Peek lies on the iteration--quality Pareto frontier across all benchmarks."
- Parsing Schema: A context map section that records format and delimiter conventions for parsing the context. "Parsing Schema for format and delimiter structure."
- Priority-based eviction: An eviction strategy that removes the lowest-scored or oldest items first to respect a fixed budget. "Evictor: Priority-based Eviction."
- Prompt learning: Treating the prompt as a learnable object to improve performance without modifying model weights. "prompt learning (also called context engineering) treats the prompt as a learnable object,"
- RAG (Retrieval-Augmented Generation): A method that retrieves relevant chunks from an external corpus and conditions generation on them. "RAG is a method in which an LM answers a query by first retrieving relevant pieces of information (chunks) from an external corpus"
- Read-Eval-Print Loop (REPL): An interactive environment pattern where code is read, evaluated, and results printed, used by some agents. "as variables in a Read-Eval-Print Loop (REPL) environment"
- Recursive LLMs (RLMs): Agents that externalize contexts and recursively invoke themselves to inspect and solve subproblems. "RLM is a recent, widely used agent that externalizes contexts as variables in a Read-Eval-Print Loop (REPL) environment"
- Reusable Results: A context map section that stores intermediate or derived computations useful across tasks. "Reusable Results for derived computations"
- Rubric accuracy: A fine-grained CL-bench metric scored by a judge model against a rubric. "report both solving rate (coarse) and rubric accuracy (fine-grained)."
- Semantic caching: Caching and reusing responses or prompts based on semantic similarity rather than exact match. "semantic caching, which maintain task-facing procedures, plans, or answers."
- Shared chat: Persisting prior conversation turns across tasks to enable in-context learning from previous trajectories. "Shared chat carries the agent's prior trajectory,"
- System prompt: The fixed system message that conditions the agent; Peek’s context map resides here. "sits in the agent's system prompt."
- Token budget: A hard limit on the number of tokens allocated to the context map. "The context map has a hard token budget , fixed across all queries on a context and across all experiments."
- Trajectory analysis: LM-driven analysis of execution traces to diagnose behavior and extract transferable insights. "Modern LMs, however, are very good at trajectory analysis."
Collections
Sign up for free to add this paper to one or more collections.
