- The paper presents an LLM-driven pipeline that converts natural-language agent instructions into formally verified Cedar policies via a generator-critic loop.
- It utilizes a layered design with grounding, model, and safety layers to extract semantic primitives, synthesize candidate policies, and reduce hallucinations.
- The autoformalization process achieves high block rates (up to 85.7%-100%) against unsafe operations, surpassing traditional manual guardrails.
Motivation and Problem Context
Agentic LLMs increasingly operate in high-stakes domains requiring rigorous risk controls. Traditional approaches to agent safety—such as classifier-based guardrails or prompt engineering—lack formal guarantees and are vulnerable to adversarial prompt manipulation. Manual hand-coding of symbolic enforcement cannot scale across the breadth and heterogeneity of real-world policy requirements. The paper introduces an autoformalization pipeline that leverages LLMs and deterministic critics to synthesize formally verifiable security policies for autonomous agents, targeting enforcement via the Cedar policy language.
Architecture and Methodology
The pipeline is structured as a layered "Verification Sandwich":
- Grounding Layer: Extracts ontological primitives (principals, resources, actions) from agent cards and tool schemas, establishing the environment for policy synthesis. Entity extraction and schema generation ensure candidate policies reference valid system identifiers, reducing hallucination and type errors.
- Model Layer: Utilizes LLMs (Gemini 3 Pro in experiments) to interpret system prompts, tool schemas, and unstructured policy documents, generating candidate Cedar policies reflecting desired agent logic.
- Safety Layer: Implements a generator-critic loop with two complementary critics:
- Hard Critic: Invokes the Cedar reference tooling for syntactic, schema, and logical contradiction checks (e.g., vacuous policies, conflicting rules).
- Soft Critic: Employs an LLM as a semantic judge, assessing qualitative alignment via rubrics with the original natural language policy, ensuring generated policies accurately encode intent.
Iterative feedback refines the candidate policy set, terminating once both critics signal satisfactory alignment and correctness. The resulting policy suite is then deployed via Cedar's external deterministic policy engine, enabling runtime enforcement decoupled from the LLM's context window.
Evaluation and Numerical Results
The autoformalization pipeline is evaluated on MedAgentBench, a benchmark for EMR agentic workflows. The prior reference [hong2026symbolicguardrails] manually implemented symbolic guardrails for only 23 of 88 synthesized policy rules; the proposed pipeline automatically produces coverage across a substantially larger subset. Evaluation prioritizes enforcement coverage over agent utility—quantifying blocked policy violations rather than task success.
Key numerical findings:
- Cedar Block Rate: In adversarial scenarios, up to 85.7% of unsafe agent trajectories (49/57) are blocked by the autoformalized Cedar suite compared to 0% violation rate in Hong et al.'s restricted guardrail scope. For trajectories involving write operations (POST requests), Cedar provides near-total enforcement: up to 100% block rate across multiple test splits.
- Coverage Expansion: The pipeline achieves broader policy enforcement due to automatic synthesis of policies from the full natural-language specification rather than the restricted manual set, resulting in a consistently higher block rate and safety margin.
The results highlight the efficacy of deterministic external policy enforcement, robust to prompt injection and context manipulation, and emphasize the utility of formal verification over heuristic classifier-based guardrails.
Technical and Practical Implications
Decoupling policy enforcement from the LLM context removes vulnerability to semantic jailbreaks and indirect prompt exploits. The fail-closed principle ensures that agent actions default to denial upon enforcement failures, prioritizing safety. However, formal policy enforcement introduces operational friction—overly narrow policies can degrade agent functionality, requiring careful calibration. The generator-critic loop with semantic evaluation enables tuning between risk tolerance and operational flexibility.
Use of Cedar facilitates human auditing and formal verification, delivering type safety and allowing transparency for security practitioners. This marks a departure from purely neural approaches, addressing issues of confabulation and entity hallucination.
Limitations and Future Directions
Cedar's stateless model limits policy enforcement in multi-turn workflows that require temporal dependencies or persistent memory. Addressing these limitations involves:
- Temporal Logic Integration: Enabling authorization conditioned on action ordering or prerequisite completion.
- Memory-Aware Policies: Enhancing policies to reference agent historical trajectories—for example, preventing redundant or conflicting actions in persistent sessions.
Expanding the autoformalization pipeline toward temporal and memory-aware formal policies remains an open challenge, promising finer-grained and longitudinal agent governance.
Conclusion
This work demonstrates an LLM-driven autoformalization pipeline capable of translating agent instructions and policy documents into formally verified machine-enforceable security policies using Cedar. The approach scales policy coverage far beyond manual symbolic enforcement, delivers strong safety guarantees, and is robust against LLM non-determinism and adversarial manipulation. Ongoing research toward temporal and stateful policy formalism will further enhance agent safety in complex multi-turn scenarios.