Scoped Verification for Reliable Long-Horizon Agentic Context Evolution under Distribution Shift

Abstract: Deployed LLM agents rely on agentic context, the model-external textual control content assembled by an operational harness. In this work, the mutable component of that context is a persistent system-level instruction that is updated from operational experience while the model, tools, and harness remain fixed. Over long evolution horizons, flat-text maintenance makes verification increasingly difficult as accumulated instructions grow and interact. We propose Graph-Regularized Agentic Context Evolution (GRACE), which maintains the persistent instruction component as a typed semantic graph and validates proposed updates within the local typed neighborhoods of modified nodes. Accepted graph updates are reconstructed as incremental edits to the textual instruction checkpoint used at deployment. We evaluate GRACE within a fixed telecom agent harness derived from $τ<sup>2$-bench under a controlled distribution-shift protocol. Across five independent replications, GRACE improves strict reliability, measured by pass^3, from the Gemini 2.5 Flash zero-shot value of 0.091 to 0.673$\pm$0.136 at the final checkpoint. This exceeds a Gemini 3.1 Pro zero-shot reference of 0.242 on the same held-out set, while the flat-text HCE baseline finishes at 0.191$\pm$0.051. These results identify two requirements for reliable long-horizon context evolution, a structural substrate that makes verification local and a consolidation mechanism that keeps accumulated instruction content usable.

• The paper presents GRACE, a graph-regularized method that transforms flat-text context evolution into a typed semantic graph for robust, localized verification.
• It demonstrates significant reliability gains under distribution shift, improving the passˆ3 metric from 0.091 to 0.673 while outperforming flat-text baselines.
• The method’s consolidation and scoped validation reduce instruction contradictions and bloat, ensuring sustainable context evolution in dynamic deployments.

Scoped Verification for Reliable Long-Horizon Agentic Context Evolution under Distribution Shift

Introduction and Motivation

Long-horizon deployments of LLM-based agents require robust mechanisms for updating the system-level instruction components of agentic context as environments and task distributions change. Existing strategies, primarily based on flat-text instruction maintenance and ad hoc prompt-refinement methods, face significant verification challenges as instruction length and complexity increase. This paper introduces Graph-Regularized Agentic Context Evolution (GRACE), a method that lifts context evolution to a typed semantic graph substrate, enabling local, structurally informed validation and consolidation of persistent instruction content. The authors evaluate GRACE in a controlled continual distribution-shift setting and empirically demonstrate superior reliability and retention over standard flat-text and graph-based ablation baselines.

Agentic Context Evolution: The Setting

Deployed agentic LLMs operate under a context composed of static and mutable model-external instructions, tool outputs, and harness-supplied control content. The mutable component, a system-level instruction, encodes behavioral constraints, procedural guidance, and operational domain assumptions. During extended deployment intervals under shifting task distributions, this persistent component must evolve based on operational signals, without retraining the agent model or modifying tool/harness code.

As instruction artifacts accrue incremental updates, verification complexity grows superlinearly in flat-text settings. Prior empirical work documents failure modes such as context collapse, unbounded memory growth, and accumulated guideline conflicts, each arising from the implicitness of relational structure in sequential text [zhang2026ace, suzgun2026dc]. This motivates explicit structuring of accumulated guidance to facilitate scoped verification.

The GRACE Framework

GRACE formalizes persistent instruction context as a directed, typed semantic graph $G_t=(V_t, E_t)$, with nodes for atomic instruction units and edges for explicit relationships. Edges are typed as supports, refines, or sequence, capturing behavioral grounding, specialization, and procedural order, respectively. A fixed ontology schema defines permissible node and relation types, enforcing structural regularity (Figure 1).

Figure 1: Overview of the GRACE evolution pipeline. GRACE converts diagnoses into schema-constrained graph updates, performs local structural validation in the typed subgraph, and reconstructs updated instruction checkpoints for deployment.

Pipeline Structure

The GRACE evolution operator comprises three key stages:

1. Operation Planning: Diagnosis-driven proposals are translated into schema-preserving graph edits (add/remove/modify nodes/edges, merge). Edits are constrained by type signatures and constructed using a finite editing algebra.
2. Structural Validation: A local subgraph surrounding modified nodes is extracted using multi-hop typed neighborhoods. Leveraging meta-path-based analysis from heterogeneous information networks [sun2011pathsim], GRACE detects and repairs local contradictions and actionable redundancies, relying on the explicit relation and object types. This step enables local consolidation and specialization enforcement, which are infeasible in flat-text representations.
3. Delta Reconstruction: The difference between the new and previous graphs is deterministically mapped to corresponding textual edits, constructing the next deployed instruction checkpoint in text form for agent context assembly.

This design ensures that the persistent instruction artifact remains both semantically explicit and operationally usable, bridging structural validation and model-external context control.

Experimental Protocol and Setting

Evaluation is conducted in the telecom domain of $\tau^2$-bench under controlled distribution shift. The system-level instruction is evolved over 10 batches, with experience distributions alternating intent configuration to induce subpopulation shift. All methods—GRACE, GRACE without structural analysis (SA; ablation), and Holistic Context Evolution (HCE; flat-text baseline)—are evaluated with the same agent model (Gemini 2.5 Flash), user simulator (GPT-4.1), diagnosis signal, and held-out test set. Replicated experiments (five runs per method) quantify both within-phase improvement and cross-phase retention, using strict reliability metrics (pass$ˆ3$), best-of-k (pass@3), and transfer under phase change.

Main Results

The empirical analysis centers on the strict reliability metric (pass$ˆ3$; all three independent episodes of a task must succeed), a robust indicator of persistent context quality under shift. The main findings are:

• Substantial Reliability Gains Under GRACE: GRACE improves pass$ˆ3$ from 0.091 (zero-shot) to 0.673 ± 0.136 at the final checkpoint ($\ell_{10}$), outperforming both Gemini 3.1 Pro zero-shot reference (0.242) and the HCE text baseline (0.191 ± 0.051) by considerable margins (Figure 2).
• Failure of Flat-Text and Non-Consolidated Graph Baselines: HCE achieves transient improvement (up to 0.236 at $\ell_8$) before degrading, while GRACE without SA achieves intermediate peaks but fails to consolidate, ending at 0.248.
• Consolidation and Contradiction Suppression: Contradiction counts in instructions are minimized by both GRACE and its graph-only ablation; however, only the full GRACE pipeline, with local consolidation, achieves lasting performance without redundancy accumulation or excessive growth. The flat-text HCE accumulates more contradictions and swells in size, further impairing reliability.

  Figure 2: Replicated pass$ˆ3$ checkpoint trajectories for GRACE, GRACE without SA, and HCE. GRACE demonstrates sustained, monotonic reliability improvement across evolution, while baselines stagnate or regress.

Mechanistic Insights

Analysis of instruction artifacts (contradiction/redudancy audits, node/instruction size tracking) reveals the following:

• Flat-text Baselines Accumulate Contradictions: HCE exhibits increasing contradiction counts and instruction bloat, directly correlating with plateau or regression in downstream reliability.
• Graph Substrate Suppresses Contradiction but Needs Active Consolidation: GRACE's ablation retains the ability to suppress new contradictions but fails to consolidate redundant or divergent content, leading to persistent expansion and loss of previous gains.
• Consolidation via Scoped Validation is Critical: Active local merging and disambiguation in the affected subgraph neighborhood (enabled by SA) are necessary to prevent drift and instruction collapse under continual evolution and distribution shift.

Instruction size metrics further demonstrate the inefficiency of non-structured evolution strategies. At the final checkpoint, HCE's deployed instruction text has grown to 82,592 characters (mean), while GRACE remains at 51,413 characters.

Practical and Theoretical Implications

• GRACE as a General Mechanism for Context Verification: The explicit, typed-graph substrate not only structures the persistent instruction for efficient update and validation but also enables preservation under non-stationary task distributions. This permits deployment of more reliable LLM agents in long-horizon, shifting settings without continual retraining or rewriting.
• Extensibility to Other Domains and Artifacts: While evaluated in telecom-based user service dialog, the GRACE substrate is agnostic to domain-specific instruction semantics, suggesting adaptability to broader agentic interaction settings, memory management, and strategic playbooks.
• Future Research Directions: Open questions include scaling GRACE to larger, more heterogeneous task benchmarks, integrating richer ontologies, and automating the structural validation process. Matching against more sophisticated context-evolution baselines and optimizing LLM-call budgets in edit planning and validation are also vital next steps.

Conclusion

The GRACE framework sets a new operational standard for long-horizon agentic context evolution by uniting explicit structural representation and local validation with practical checkpoint text reconstruction. Empirical results demonstrate sustained, monotonic reliability under adversarial distributional shift, a property absent from current flat-text or insufficiently consolidated approaches. These findings underscore the necessity of explicit structure and scoped consolidation for verifiable and durable agentic instruction evolution. Future work should systematically extend this paradigm to more dynamic, cross-domain agent orchestration regimes.