PolicyBank: Adaptive Policy Memory

• PolicyBank is a structured memory mechanism that iteratively refines tool-level interpretations of organizational policies expressed in natural language.
• It uses a dynamic memory bank and a policy-evolution loop to update interpretations based on corrective feedback and structured revisions.
• Empirical benchmarks show that PolicyBank closes up to 82% of the gap toward a human oracle, significantly improving tool-calling accuracy in controlled scenarios.

PolicyBank is a structured memory mechanism for LLM agents that iteratively refines tool-level interpretations of organizational policies expressed in natural language, especially when the written specification is ambiguous, incomplete, or logically flawed relative to the true required policy. Its central premise is that organizational agents can fail even when they are fully compliant with the text they were given, because the text itself may not coincide with the actual requirement. PolicyBank therefore stores structured policy insights, updates them through interaction and corrective feedback, and reuses them during tool-calling execution rather than treating the policy as immutable ground truth. In the benchmark introduced with the method, PolicyBank closes up to 82% of the gap toward a human oracle on controlled policy-gap scenarios (Choi et al., 16 Apr 2026).

1. Formal setting: specified policy, required policy, and policy gaps

PolicyBank is formulated for stateful tool-calling agents. A tool-calling agent $\mathcal{A}$ has tools $\mathcal{F} = \{f_1, \ldots, f_m\}$, produces an action trace $\tau = (a_1, \ldots, a_k)$, and operates in environment state $\sigma \in \Sigma$. A policy is modeled as an authorization function

$$\psi: \Sigma \times \mathcal{A}_{\mathcal{F}} \to \{permit, deny\},$$

and a trace is accepted if every action is permitted in its execution state. This framing makes policy compliance a state-action authorization problem rather than a purely textual reasoning problem (Choi et al., 16 Apr 2026).

The paper distinguishes two policy objects. The Specified Policy is the written policy document, system prompt, or business-rule text exposed to the agent. The Required Policy is the true underlying organizational requirement. A policy gap exists when these disagree on at least one state-action pair. In the paper’s interpretation, this disagreement set is the essential source of alignment error: the agent may execute the specified policy correctly and still fail the required policy.

Three gap types are identified. Ambiguous Scope / Set Interpretation arises when vague quantifiers or examples are read too narrowly or too literally. Under-Specified Exceptions / Missing Boundary arises when a general rule omits valid exceptions and thereby over-restricts behavior. Logical Contradiction / False Dependency arises when the written policy ties one action to an unrelated condition. The airline compensation example in the paper illustrates the last case: a literal agent may deny compensation to a delayed customer who does not want to modify the booking, even when compensation was intended to be independent of modification intent.

This formalization supports the paper’s distinction between two failure classes. Type I: Execution failure denotes failure to follow even the written policy because of reasoning or tool-use limitations. Type II: Alignment failure denotes correct execution of the written policy when the written policy is itself wrong or incomplete. PolicyBank is aimed at Type II failure in particular, although its deployment occurs within a general tool-using agent stack.

2. Memory mechanism and policy-evolution loop

PolicyBank is a memory bank for evolving policy understanding, organized at the level of tool capabilities rather than at the level of generic task summaries. Each memory entry $e \in \mathcal{M}$ contains a unique capability identifier, a tool name, a capability name, a semi-structured natural-language specification $\text{Spec\_NL}$, and a learned key insight or delta. The recommended specification template contains five fields: TRIGGER, PRECONDITIONS, ELIGIBILITY, ACTION, and KEY INSIGHT (Choi et al., 16 Apr 2026).

This schema is meant to make policy memory operational. A tool such as cancel_reservation may support multiple distinct capability variants with different governing rules, and PolicyBank treats those as separate objects of memory. That differs from task-memory systems that compress an entire trajectory into a generic lesson. Here the unit of memory is the governable capability on which authorization depends.

The operational loop has five stages. Memory is first initialized from policy documents, database schema, and tool definitions. During execution, the agent may query memory through a callable retrieve_policy() tool. After a task completes, a dedicated Policy Agent reviews the trajectory and developer feedback. It then decides whether to Add a new memory entry, Revise an existing one, or Omit changes if nothing new was learned. The result is an updated memory $\mathcal{M}_{t+1}$.

Retrieval is agent-triggered rather than statically front-loaded. The agent first sees capability headers, selects relevant entries, and then receives the full structured policy for those entries. The paper argues that this is important for long-horizon conversations in which user intent and relevant authorization context evolve over time. Feedback can be a binary Reward signal or an Explanation in natural language. The latter is especially important because correcting a policy gap is a deductive problem: a pass/fail signal alone does not identify which clause in the specified policy was misinterpreted or should have been overridden.

PolicyBank therefore differs from systems that improve execution while assuming the policy text is correct. Its memory is explicitly revisionary: it treats failure feedback as evidence that the internal interpretation of policy may need repair.

3. Benchmark design and isolation of alignment failures

The empirical evaluation extends $\tau$-Bench with controlled policy gaps. The study uses the Airline domain with the original 50 tasks and the Retail domain with the original 114 tasks, excluding telecom because it was already highly saturated and less policy-intensive (Choi et al., 16 Apr 2026).

Policy gaps are identified by running baseline agents on original tasks, collecting failures, comparing the policy text against ground-truth intended behavior, classifying the gap type, and manually writing a Policy Clarification that resolves the gap. These clarifications serve as the gold-standard updates. This construction is central to the paper’s methodology because it separates mis-specified policy from generic model weakness.

For each affected parent task, the benchmark adds three sister tasks. The Simplified edit ($t$-1) removes confounding complexity to isolate the policy gap. The Different instance ($\mathcal{F} = \{f_1, \ldots, f_m\}$0-2) changes the user, product, or reservation while preserving the same policy issue. The Complex variant ($\mathcal{F} = \{f_1, \ldots, f_m\}$1-3) adds reasoning burden, irrelevant detail, or multi-requirement interactions. This design tests whether the agent learned the clarification itself rather than memorizing a single corrected example.

The benchmark is intended to isolate Type II alignment failure from Type I execution failure. The paper does this by analyzing tasks that frontier models consistently fail, identifying whether the failure arises from capability limitations or from mismatch between specified and required policy, and then validating the diagnosis through the sister-task construction. In this sense, PolicyBank is evaluated not merely as another memory module but as a mechanism for specification repair under controlled ambiguity.

4. Empirical performance and comparative findings

The main empirical claim is that PolicyBank closes up to 82% of the gap toward a human oracle. On policy-gap sister tasks, no-memory agents frequently collapse to near-zero performance, and baseline memory systems such as Synapse and AWM also remain near zero. ReasoningBank improves somewhat, but PolicyBank is substantially stronger while maintaining strong performance on original tasks (Choi et al., 16 Apr 2026).

The paper reports concrete examples of this contrast. In Airline/Gemini sister tasks, the no-memory baseline has pass@1 of 0.01, and pass@2 through pass@4 are 0.00. Under PolicyBank, Airline/Gemini sister-task pass@1 reaches 0.74. In Retail/Gemini sister tasks, PolicyBank pass@1 reaches 0.83. Claude-4.5-Opus also improves strongly, though generally less than Gemini in some settings. The paper further reports improved consistency under stricter pass$\mathcal{F} = \{f_1, \ldots, f_m\}$2 metrics, indicating that the gains are not confined to a single successful roll-out.

A separate ablation compares No memory, Reward only, Reward + Explanation, and Human Oracle. The findings are structurally important. Reward only yields only marginal gains. Reward + Explanation closes much of the gap. Human Oracle remains best, but PolicyBank approaches it by using explanations to infer the intended correction. The role of explanation is therefore not incidental; it is the informational channel through which latent policy defects become identifiable.

These findings support the paper’s claim that policy-gap resolution is not reducible to improved task rehearsal. A system that simply remembers successful trajectories can still reinforce “compliant but wrong” behavior if the underlying textual policy is defective. PolicyBank improves because it revises the internal representation of what the policy should mean at the tool level.

5. Relation to memory systems, guardrails, and adjacent policy infrastructures

PolicyBank is positioned against prior memory mechanisms such as Synapse, AWM, and ReasoningBank, which are described as optimizing task execution under an assumption that the written policy is correct and immutable. Its contribution is therefore not generic memory augmentation but memory-based policy repair. The paper also states that PolicyBank is complementary to hard enforcement layers such as guardrails or verifiers: PolicyBank refines interpretation, whereas verification and guardrail systems enforce the resulting policy (Choi et al., 16 Apr 2026).

This distinction matters because “policy” can denote different layers of control in adjacent research. In DPFedBank, the policy component is a governance layer for privacy-preserving federated learning among financial institutions: it organizes identity controls, monitoring, privacy accounting, incident response, and threat-specific mitigations around malicious clients, compromised servers, external adversaries, insider threats, inference attacks, and poisoning attacks (He et al., 2024). In TruChain, the relevant layer is not semantic interpretation but trust and provenance in open banking data, implemented through source validation with decentralized identity and verifiable presentation, data-level authentication with cryptographic signing, and tamper-proof storage through the Tangle (Rahman et al., 11 Jul 2025).

This suggests a layered decomposition of policy-aware systems. PolicyBank addresses evolving semantic interpretation of natural-language authorization constraints. DPFedBank addresses governance and privacy policy in collaborative training. TruChain addresses trustworthiness, authenticity, and immutability of the data substrate on which policy decisions may depend. These systems do not solve the same problem, but they clarify that “policy compliance” in deployed AI systems spans at least three separable strata: interpretation of rules, enforcement and governance, and trust in the underlying data.

A common misconception is therefore to treat PolicyBank as a complete compliance architecture. The paper does not make that claim. It presents PolicyBank as a memory mechanism for improving policy understanding, not as a substitute for cryptographic trust, access-control enforcement, privacy accounting, or runtime verification.

6. Limitations, misconceptions, and research directions

PolicyBank has several explicit limitations. It relies on corrective feedback, especially natural-language explanations, to resolve policy gaps effectively. Its benchmark is relatively small and concentrated in airline and retail domains. The Policy Agent and evaluation use frontier models, and weaker models may underutilize retrieved policy. The approach also depends on the quality of the policy clarification process and may be vulnerable to noisy or adversarial feedback. Finally, it is not itself a formal guarantee of compliance (Choi et al., 16 Apr 2026).

These limitations bear directly on interpretation. PolicyBank should not be understood as a proof-producing verifier, a formal methods layer, or an immutable policy store. Its strength is adaptive interpretation under imperfect specifications. Its weakness is that the adaptation process itself depends on the quality of feedback and on the model’s capacity to infer the correct semantic delta from that feedback. In other words, it improves understanding without removing epistemic dependence on post hoc clarification.

The paper’s future directions are correspondingly focused. It explicitly proposes larger-scale testbeds with more diverse policy topologies, robustness to noisy or adversarial feedback, extension to open-source models, and integration with formal verification layers for provable compliance. Those directions indicate that the present system is an intermediate layer in a broader alignment stack rather than an endpoint.

A second misconception is to equate PolicyBank with static retrieval over policy text. The paper’s design rejects that equivalence. Retrieval is agent-triggered; the stored objects are structured capability-level policies; and updates are driven by trajectory review plus feedback. A plausible implication is that PolicyBank is best viewed as an adaptive authorization memory for long-horizon tool use, where correctness depends not only on recalling rules but on revising them when the written specification is revealed to be incomplete.

In the current literature, PolicyBank’s distinctive contribution is therefore conceptual as well as empirical: it treats natural-language policy not as fixed truth but as an evolving hypothesis about organizational requirements, and it supplies a memory mechanism through which an agent can iteratively reduce the divergence between specified and required policy.