Papers
Topics
Authors
Recent
Search
2000 character limit reached

Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes

Published 1 May 2026 in cs.CR, cs.AI, cs.MA, and cs.SE | (2605.00424v1)

Abstract: Agent skills -- structured packages of instructions, scripts, and references that augment a LLM without modifying the model itself -- have moved from convenience to first-class deployment artifact. The runtime that loads them inherits the same problem package managers and operating systems have always faced: a piece of content claims a behavior; the runtime must decide whether to believe it. We argue this paper's central thesis up front: a skill is \emph{untrusted code} until it is verified, and the runtime that loads it must enforce that default rather than infer trust from a signature, a clearance, or a registry of origin. Without skill verification, a human-in-the-loop (HITL) gate must fire on every irreversible call -- which is operationally untenable and degrades into rubber-stamping at any non-trivial scale. With skill verification treated as a separate, gated process, HITL fires only for what is unverified, and the system becomes sustainable. We give a trust schema (§\ref{sec:schema}) that includes an explicit verification level on every skill manifest; a capability gate (§\ref{sec:gate}) whose HITL policy is a function of that verification level; a \emph{biconditional} correctness criterion (§\ref{sec:biconditional}) that any candidate verification procedure must satisfy on an adversarial-ensemble exercise (§\ref{sec:eval}); and a portable runtime profile (§\ref{sec:guidelines}) with ten normative guidelines abstracted from a working open-source reference implementation \cite{metere2026enclawed}. The contribution is harness- and model-agnostic; nothing here requires retraining, fine-tuning, or proprietary infrastructure.

Authors (1)

Summary

  • The paper introduces a trust schema that treats skills as versioned, inspectable artifacts subject to rigorous verification.
  • It presents a capability-gate mechanism combined with HITL policies to control irreversible actions and ensure auditability.
  • Empirical adversarial ensemble evaluations demonstrate detection of failure modes and enforce runtime correctness via strict audit logs.

Verifiable Skill Artifacts and Trust Enforcement in HITL Agent Runtimes

Introduction and Context

The paper "Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes" (2605.00424) addresses critical trust and correctness issues in LLM agent ecosystems, particularly in runtime environments integrating modular “skills.” As LLM-based agents increasingly gain access to tools with real-world side effects, modular skills—embodied as versioned, inspectable SKILL.md artifacts—have become the standard mechanism for extensibility. However, this modularity introduces a complex software supply-chain surface, echoing long-standing issues in operating system and package management security.

The central thesis of the paper is that skills, regardless of signature or registry provenance, must be regarded as untrusted code until a distinct verification process is complete. The default to human-in-the-loop (HITL) gating on every irreversibly impactful call is shown to be both operationally unsustainable and ultimately susceptible to rubber-stamping in practice. The work proposes a discrete trust schema, an audit-based biconditional correctness criterion, and a portable runtime profile designed to enforce principled safety boundaries without requiring changes to the underlying LLM.

Skill Trust Schema and Verification Process

The proposed schema treats each skill as a tuple: manifest, content, and cryptographic signature. The manifest includes a formal Bell–LaPadula classification label, a declared capability set, a signer ID, monotonic versioning, and, crucially, a verification level field. Verification levels are strictly enumerated: unverified, declared, tested, formal. This discrete stratification, rather than a continuous trust score, ensures clear, auditable boundaries and prevents covert trust escalation via permission gradients.

Critically, the schema enforces immutability of skills during agent runtime. Any attempt by the agent to modify a skill triggers an irreversible operation gated by HITL and logged with a pre/post state record, necessitating full re-verification for any future use. This severely limits the attack surface for agent-driven or indirect skill mutation and aligns with capabilities-based security disciplines in established systems literature.

Verification is a bootstrap-only, offline process, never elevated or changed during runtime, and is explicitly bound to the authorized clearance of trusted signers. This mitigates time-of-use escalation attacks and unintentional trust propagation from benign but unverified artifacts.

Capability Gate and HITL Policy

A central contribution is a capability-gate architecture, coupling the permitted side effect set (capabilities) with the loaded skill's verification level. The runtime enforces a per-call, fixed policy:

  • unverified: All irreversible effects are universally HITL-gated, regardless of manifest declarations.
  • declared/tested: Only non-manifested or outside-capability irreversible calls are gated; verified irreversible calls proceed with logging.
  • formal: Reserved for machine-checked guarantees, currently aspirational.

The runtime's reversible/irreversible side-effect split underpins this gating logic. Reversible calls operate through a transaction buffer subject to roll-back, whereas irreversible calls undergo a four-stage HITL approval and logging lifecycle—including request, decision, execution, and audit.

Broker policies for HITL can be strictly deny-all, policy-driven, interactive human approval, or webhook-mediated. The architecture ensures that audit logs produced are sufficient for post hoc, deterministic verdicts on agent behavior.

Biconditional Correctness Criterion

A novel element is the biconditional correctness criterion, operationalized as follows: for a given agent run, the set of observed side-effects in the world (as a delta between initial and final corpus states) must exactly correspond to the set of approved-and-executed entries in the audit log. The biconditional is strictly enforced; any deviation constitutes a correctness violation, capturing four explicit classes of failure: gate bypass, audit forgery, silent host failures, and wrong-target execution.

This criterion is leveraged both as an operational correctness property for production deployments and as a formal benchmark in skill verification pipelines—specifically, for elevating a skill to the tested verification level through adversarial ensemble evaluation.

Notably, the authors acknowledge several residual limitations: read-only exfiltration, TOCTOU races, and overlapping adversarial actions remain partially unsolved at this layer, as they are outside the biconditional’s enforceable scope.

Adversarial Ensemble Evaluation

To stress-test the gate and correctness mechanisms, the paper introduces an adversarial-ensemble evaluation methodology: multi-agent ensembles, each given destructive or manipulative behavioral biases, are run against a fixed corpus with an integrity baseline. The verdict on agent actions is rendered mechanically, with after-the-run corpus deltas reconciled against the audit log under the biconditional criterion.

Strong empirical evidence demonstrates detection of all targeted failure modes when appropriate faults (e.g., tool bypass, log forgery, improper mutation) are deliberately injected. The deterministic nature of the evaluation supports reproducibility and transparency for operator policy and incident analysis.

Architectural Guidelines and Runtime Hardening

Twelve normative guidelines (G1–G12) are extracted from a working reference implementation (enclawed (Metere, 18 Apr 2026)), encapsulating the schema into actionable enforcement rules. Key elements include:

  • Bootstrap-time locking of all trust configuration (G1)
  • Hard deny-by-default posture at every trust boundary (G2)
  • Mandatory per-artifact classification (G3)
  • Hash-chained, append-only audit logs at gate-event granularity (G5)
  • Enforcement of skill immutability and mandatory HITL on all unverified irreversible calls (G11, G12)
  • No global bypass switches for core enforcement mechanisms (G10)

Collectively, these guidelines substantially reduce the practical attack surface for skill-based agent exploitation, mitigate both supply-chain and prompt-injection risks, and facilitate forensic post-incident review.

Implications and Future Directions

The work has significant practical implications for secure agent deployment in environments where LLM-driven agents are granted privileged tool access. The model/harness-agnostic design ensures broad applicability, particularly in supply-chain-sensitive or regulated contexts.

By operationalizing skill verification as a standalone, auditable process and reducing unconditional reliance on HITL, the approach offers a tractable path for agent operators to scale trusted skill libraries without succumbing to the failure modes of package manager ecosystems and indiscriminate prompt-injection vulnerabilities.

Theoretically, the strict biconditional criterion advances the rigor of runtime safety guarantees, providing a concrete property that is both necessary and sufficient for a wide class of irreversible action correctness. On the other hand, as with many classical information-flow and access control architectures, comprehensive defense against covert channels and subtle information leaks remains open, pointing to needed advances in egress-aware and content-flow controls.

Open questions remain around efficient skill composition, revocation protocols, and integration with hardware-rooted trust anchors. Scaling adversarial ensemble evaluations and pipeline integration for large, heterogeneous agent deployments is a mechanically feasible but practically significant challenge. These areas merit further work.

Conclusion

The paper establishes a model-agnostic, auditable framework for safe skill invocation in HITL agent runtimes through (1) an immutable, signed skill schema with formal verification levels, (2) a capability-gate architecture tuned to verification, (3) a biconditional correctness criterion enforceable at runtime and in verification pipelines, and (4) a suite of architectural guidelines rigorously abstracted from a production implementation (2605.00424). The approach enforces the principle that skills are untrusted until verified, replacing brittle reliance on signatures with reproducible, incident-resilient trust boundaries. This work lays a clear foundation for future research and deployment standards for agent safety in the rapidly evolving LLM agent ecosystem.

Whiteboard

There was an error generating the whiteboard.

Explain it Like I'm 14

Explaining “Skills as Verifiable Artifacts” in Simple Terms

What is this paper about?

This paper is about making AI “agents” (programs powered by LLMs) safer when they use add-on “skills.” A skill is like a small plug-in or instruction pack that tells the agent how to do something (for example, “send an email” or “clean up files”). The big idea: treat every skill as untrusted until it’s proven safe, and make the agent’s “runtime” (the system that runs the agent) enforce clear rules so harmful actions are blocked or carefully approved.

What questions does the paper try to answer?

In easy terms, the paper asks:

  • How can we stop dangerous or sneaky skills from making an AI agent do harmful things?
  • How can we avoid asking a human to approve every single action (which is slow and annoying), while still staying safe?
  • What does it mean to “verify” a skill, and how should a system behave differently when a skill is verified versus not verified?
  • How do we keep a trustworthy record of what the agent actually did in the real world?

How did the authors approach the problem?

Think of the agent’s world like a building with locked doors and a security guard. The paper designs the keys, rules, and logs for that building:

  • Skills as plug-ins: A skill is a package with a description (a “manifest”) and content (instructions, scripts). It may be signed by someone you trust, but a signature alone doesn’t prove the skill behaves safely.
  • Verification levels: Each skill has a clearly labeled safety status set before use:
    • unverified: we don’t trust it yet.
    • declared: a trusted person says the skill’s actions are limited to what it claims.
    • tested: the skill has been stress-tested in tough situations and behaved correctly.
    • formal: a rigorous (math-like) proof shows it can only do what it claims (future goal).
  • A capability gate: This is the security guard. When the agent tries to do something, the gate checks:
    • What action is it (for example, read a file, send a message, delete a file)?
    • Is the action reversible or irreversible?
    • What is the skill’s verification level?
    • Depending on the answers, the gate either:
    • executes safely,
    • asks a human for approval (Human-In-The-Loop, “HITL”), or
    • blocks it.
  • Reversible vs. irreversible actions:
    • Reversible: you can easily undo it (like saving a draft you can delete).
    • Irreversible: you can’t take it back (like sending an email or paying money).
    • Irreversible actions are the most carefully guarded.
  • No surprise changes during a session: Once a skill is loaded, it cannot be changed by the agent during that session. If the agent tries to change it, that counts as a serious, trackable action that must be approved and logged.
  • Audit log as a “receipt book”: Every important event is written into a chained, tamper-evident log (like linked receipts). This lets you review exactly what happened.
  • A simple “did the gate work?” test (biconditional correctness): After the agent runs, compare:
    • the real-world changes (what actually changed), and
    • the approved-and-executed records in the log.
    • They must match exactly—every change must have a matching approved record, and every approved action must have a matching change. If not, something went wrong (like a bypass or a fake log entry).
  • Adversarial testing: The authors describe a test where multiple “pushy” mini-agents try to cause destructive actions (like deleting files). The gate and the log test are used to make sure those actions only happen when properly approved, and any mismatch is caught.
  • Practical rules (guidelines): The paper extracts a small set of “do this by default” rules from a working open-source system. Examples include: deny by default, keep a locked list of trusted signers, separate reversible from irreversible actions, log everything important, and never provide a “turn off safety” switch.

What did they find, and why does it matter?

Main takeaways:

  • Treat skills as untrusted by default: Don’t assume a signed skill is safe; signatures say “who,” not “how it behaves.”
  • Use verification levels to reduce human approvals: Start with strict human approval for irreversible actions; as a skill is verified (tested), the system can safely ask the human less often—but only for actions the skill was verified for.
  • Use the biconditional check to catch serious problems: If real-world changes and approved actions don’t match, you can detect:
    • hidden actions that bypassed the gate,
    • fake “we executed this” records,
    • “approved” actions that silently failed,
    • wrong-target actions (approved for A but changed B).
  • Know the limits: The system doesn’t automatically catch data leaks from read-only actions (like reading a file and secretly sending it out another way), or certain race conditions where the world changes outside the agent’s control. Those need other defenses (like data loss prevention and stronger timing controls).
  • It’s model-agnostic: None of this requires retraining the AI model. It’s about runtime rules, checks, and logs.

Why it matters:

  • It makes AI agent operations safer and more manageable at scale. You’re not stuck approving every single action forever.
  • It gives clear “paper trails” so you can prove what happened.
  • It provides a portable blueprint that different agent systems can adopt.

What could this change in the real world?

  • Safer AI assistants in offices: Agents can help with real tasks—editing files, sending messages, updating records—without constantly bothering humans, and without risking silent harmful actions.
  • Clearer responsibility: If something goes wrong, the audit and the biconditional test help figure out exactly what happened.
  • Easier to share and reuse skills: Teams can build libraries of verified skills and move them between systems more safely.
  • Future growth: As formal verification tools improve (and possibly hardware security too), higher assurance (“formal” level) becomes possible.

In short, this paper proposes a practical safety plan: label every skill with a verification level, enforce permissions with a strict gate, keep an unbreakable receipt trail, and check that every real-world change matches an approved action. This lowers risk, reduces busywork for humans, and makes AI agents more trustworthy.

View Paper Prompt  View All Prompts 

Knowledge Gaps

Knowledge gaps, limitations, and open questions

The following list captures what remains missing, uncertain, or unexplored in the paper, phrased to be concrete and actionable for future work.

  • Specify a formal semantics for skills and SKILL.md that unambiguously maps manifests, content, and capability calls to runtime behavior, enabling tool support and proofs.
  • Define a rigorous, machine-checkable model of “side-effects” and targets (including identity, aliasing, and normalization rules) so capability claims can be verified uniformly across platforms.
  • Resolve an apparent inconsistency in gate behavior: sections describe irreversible calls executing via a “transaction buffer,” while earlier the buffer is only for reversible operations; clarify precise execution paths for declared/tested levels.
  • Provide an extensible, namespaced capability vocabulary with formal contracts (pre/post-conditions, idempotency, target schema) and guidance for adding new capabilities without degrading safety.
  • Standardize path and host canonicalization rules (symlinks, junctions, case sensitivity, UNC paths, URL parsing, IPv6 literals, IDN/Punycode, CNAME chains, redirects) to prevent capability-bypass via normalization ambiguities.
  • Develop concrete policies and checks for net.egress(host) that address DNS rebinding, shared hosting, CDNs, redirects, and SSRF-style pivots that could exfiltrate data despite an allow-list.
  • Formalize reversible vs. irreversible classification criteria per capability, including borderline cases (distributed writes, multi-step operations, eventual consistency) and how rollback is guaranteed.
  • Specify the transaction buffer’s semantics (atomicity, durability, failure modes, partial writes, compensating actions) and its integration with external systems that lack native rollback.
  • Define a two-phase or saga-like protocol for multi-resource operations so “reversible” guarantees remain valid under partial failures and cross-service dependencies.
  • Provide a precise definition and reference implementation of delta(s0, s1), including how to handle renames, metadata-only changes, partial updates, multi-target operations (e.g., recursive delete), and non-file corpora.
  • Expand corpus scope beyond files to include databases, message queues, remote stores, and blockchain state, with standardized delta extraction for each.
  • Address side-effects outside the designated corpus (e.g., network-visible state changes, payments) and how the biconditional should include or exclude them without being gameable by corpus selection.
  • Quantify detection limits of the biconditional under nondeterministic environments and concurrent actors; propose methods (e.g., instrumentation, isolation, or reference monitors) to reduce TOCTOU and overlapping-actor blind spots.
  • Design and evaluate data-flow and egress controls that complement the biconditional to detect read-only exfiltration and covert channels (e.g., content labeling, taint tracking, or DLP with robust evasion resistance).
  • Specify a formal threat model for the “formal” verification level and the exact statement to be proven (logic, model of the runtime, capability contracts, environment assumptions).
  • Propose a tractable proof framework (e.g., a DSL for skills with restricted expressiveness, or contracts for capability wrappers) and a toolchain (SMT/Coq/Isabelle) to achieve “formal” attestations.
  • Define the tested-level verification protocol: ensemble composition, prompts, seeds, sample sizes, rounds R and agents K, coverage metrics, statistical confidence thresholds, and reproducibility across harnesses.
  • Demonstrate external validity of adversarial-ensemble tests on realistic, large-scale workflows (web automation, code modification, data pipelines) with quantitative results and failure analyses.
  • Establish a standard benchmark suite and shared corpora for skill verification, with attack catalogs, reproducible policies, and interop tests across different agent runtimes.
  • Analyze performance overhead of the gate, transaction buffering, and audit logging: latency, throughput, backpressure, and operator experience under load; provide micro/macro-benchmarks and scalability guidance.
  • Address audit log security beyond hash-chaining: trusted time, secure log shipping/replication, remote append-only storage, encryption at rest, key rotation, integrity verification APIs, and recovery procedures.
  • Develop privacy-preserving audit techniques (redaction, minimization, structured logging, deterministic hashing, zero-knowledge proofs) that preserve the biconditional’s checkability while reducing sensitive content exposure.
  • Specify how broker decisions are authenticated, bound to audit entries, and protected against MITM/replay (e.g., signed decisions, nonce-binding, mTLS).
  • Study HITL broker ergonomics: decision fatigue, batching risks, escalation strategies, UI design to prevent rubber-stamping, and outcome metrics (false approvals/denials) under realistic workloads.
  • Define governance for verification-level assignment: who can attest “declared/tested/formal,” auditor qualification, conflict-of-interest controls, and revocation handling for erroneous or fraudulent attestations.
  • Provide a concrete revocation and eviction protocol: CRLs/OCSP or short-lived manifests, propagation guarantees, in-flight session handling, and maximum exposure windows.
  • Clarify boundaries between “skill code” (immutable in-session) and “data/configuration” a skill reads; prevent smuggling behavioral changes via mutable external dependencies, caches, or dynamic imports.
  • Address tool supply-chain security: how tool.invoke(name) is bound to signed, version-pinned, reproducible artifacts; prevent path hijacking and runtime substitution.
  • Explore composition safety: verifying that combinations of individually verified skills preserve properties (non-interference, capability bounds), including concurrency and cross-skill data flows.
  • Define label-propagation and declassification rules with formal guarantees; give practical mechanisms for safe declassification when outputs provably depend only on low-label inputs.
  • Provide a robust policy for in-session self-generated skills: discovery, sandboxed evaluation, temporary scopes, and post-session attestation workflows without opening escalation paths.
  • Specify defenses against denial-of-service vectors (gate request floods, log-volume exhaustion, broker overload) and provide rate-limiting and circuit-breakers without losing audit completeness.
  • Harmonize the schema with OS/hypervisor enforcement (SELinux/AppArmor, container isolation, TPM/TEE attestation) and show end-to-end attestations from hardware roots to runtime policy.
  • Propose interop standards for SKILL.md, manifests, capability vocabularies, and gate/audit APIs so multiple runtimes can share skills and verifications without bespoke adapters.
  • Provide rigorous proofs or model-checking that “no bypass switch” and “bootstrap discipline” hold under all configuration permutations, build flags, and deployment modes (including cloud-managed variants).
  • Clarify policy for denied irreversible calls: agent recovery strategies, avoiding dangerous retry loops, and observable signals to the model that minimize adversarial prompt adaptations.
  • Evaluate how different LLMs and decoding settings affect verification outcomes and HITL load; propose normalization or calibration methods to maintain safety invariants across model swaps.
  • Detail versioning and replay protections: manifest identity, monotonic version semantics across forks, multi-signer scenarios, and handling of concurrent updates without race-induced downgrades.
  • Map biconditional projections to many-to-one and one-to-many operations (bulk edits, batched transactions), ensuring the multiset equality remains well-defined and checkable.
View Paper Prompt  View All Prompts 

Practical Applications

Immediate Applications

The following applications can be deployed now by teams that operate LLM agents with tool access, provided the agent harness offers a typed tool-dispatch step the runtime can interpose on and skills are packaged with SKILL.md or equivalent manifests.

  • Enterprise IT and Security (Software)
    • Deploy a capability gate that enforces reversible/irreversible splits and HITL policies keyed to skill verification level (unverified, declared, tested).
    • Introduce deny-by-default egress, provider, and tool allow-lists with a locked trust root and signed skill manifests.
    • Assumptions/Dependencies: Typed tool dispatch; operator-defined trust root; ability to log and hash-chain audit records; broker for HITL decisions.
  • Regulated Workflows and Compliance (Healthcare, Finance, Government)
    • Enforce mandatory classification labels (e.g., Bell–LaPadula lattice) on skill artifacts, tool inputs/outputs, and side-effects; block undeclared capabilities.
    • Use hash-chained audit logs and the biconditional correctness check after each run to produce tamper-evident evidence for HIPAA/SOX/PCI audits and incident reviews.
    • Assumptions/Dependencies: DLP or egress guard for PHI/PII; policy broker configured for irreversible actions (e.g., pay(token, amount), publish(channel,…)).
  • DevOps and MLOps “Skill CI” (Software)
    • Build CI pipelines that: (1) run an adversarial-ensemble test suite over candidate skills, (2) elevate manifests to tested on pass, (3) sign and version manifests, (4) publish to a curated registry.
    • Gate promotion to production on biconditional pass/fail; pin deployments to tested manifest hashes.
    • Assumptions/Dependencies: Test corpus and agents; signing keys bound to clearance; manifest versioning; reproducible broker/policy.
  • SaaS/Cloud “Verified Skill Marketplace” (Software, Cloud)
    • Offer a marketplace where skills are labeled with verification levels and declared capabilities; provide hosted policy brokers (deny-all, policy-as-code, interactive, webhook).
    • Expose audit APIs and biconditional reconciliation endpoints for customers’ SIEMs.
    • Assumptions/Dependencies: Customer-controlled trust roots; capability vocabulary standardization; operational SLAs for broker latency.
  • Customer Support and Social Publishing Automation (Software, Media)
    • Treat publish(channel, …) as irreversible: route through HITL or policy broker unless the skill is tested; maintain a tamper-evident log for dispute resolution.
    • Adopt deny-by-default channel allow-lists per deployment profile (strict vs dev).
    • Assumptions/Dependencies: Channel adapters mapped to capability vocabulary; operator availability for approvals.
  • Data Engineering and Database Operations (Software, Data Platforms)
    • Gate mutate.schema(target) and destructive fs.write.irrev(path) operations via HITL; run reversible operations through a transaction buffer with rollback.
    • Reconcile schema/data changes with the audit log using the biconditional check to detect drift and gate-bypass.
    • Assumptions/Dependencies: Transaction buffer integration; deterministic corpus snapshots for post-run delta.
  • Robotics and IoT Gatekeeping (Robotics)
    • Map robot/PLC actions to capability vocabulary (e.g., spawn.proc, tool.invoke, publish); classify motion/actuator enables as irreversible and enforce HITL or policy.
    • Use reversible buffers for low-risk state changes (e.g., configuration staging) and immutable-in-session skill content for safety.
    • Assumptions/Dependencies: Typed action APIs; low-latency broker decisions; operator-defined allow-lists.
  • Finance and Payment Controls (Finance, Fintech)
    • Enforce HITL for pay(token, amount) unless the skill is tested and within declared caps; bind approvals to user identity in the audit chain.
    • Apply network allow-lists (net.egress(host)) to payment providers; deny unsigned/self-generated skills by default.
    • Assumptions/Dependencies: Identity-aware broker; PCI-aligned audit retention; capability mapping to payment APIs.
  • Government and Public Sector AI Assistants (Government)
    • Adopt standard profiles: strict (no egress, locked trust root) for production and open for development; no feature-flag “bypass” switches.
    • Require untrusted-by-default skill admission and per-call HITL for irreversible actions in citizen-facing assistants.
    • Assumptions/Dependencies: Policy governance; operator training; classification labeling policy.
  • Educational Labs and Benchmarks (Academia, Education)
    • Use the adversarial-ensemble evaluation and biconditional criterion as hands-on lab assignments to teach supply-chain and HITL design.
    • Publish open skill corpora with seeded faults and CI tests covering F1–F4 failure modes for coursework and replication studies.
    • Assumptions/Dependencies: Access to open-source reference (e.g., enclawed); reproducible test harness.
  • Personal AI with Safety Guardrails (Daily Life, Consumer Software)
    • Run personal assistants in strict profile: deny-all irreversible by default; interactive approvals for email sending, file deletes, and posts; immutable skills per session.
    • Provide a simple dashboard for approvals and post-run “what changed and why” via biconditional reconciliation.
    • Assumptions/Dependencies: Lightweight gate runtime; local audit storage; minimal setup for trust root.
  • Tooling and Framework Adapters (Software Vendors)
    • Ship SDKs that map LangChain/AutoGen/OpenAI tool calls to the capability vocabulary and integrate gate hooks, audit, and verification levels without model retraining.
    • Provide SKILL.md generators and manifest verifiers as developer tools.
    • Assumptions/Dependencies: Stable tool-call schemas; extension points in existing agent frameworks.

Long-Term Applications

These applications depend on further research, standardization, scaling, or integration with new infrastructure (e.g., formal methods, hardware roots of trust).

  • Formal Verification of Skills (Software, Safety-Critical Systems)
    • Develop static analyzers or model checkers that produce machine-checkable proofs that a skill’s behavior is a subset of its declared capabilities (manifest level: formal).
    • Integrate with CI to auto-elevate skills from tested to formal when proofs are available.
    • Assumptions/Dependencies: Restricted DSLs for skills; formal semantics for capability effects; proof tooling.
  • Hardware Roots of Trust and Remote Attestation (Cloud, Edge, Critical Infrastructure)
    • Bind signer keys and brokers to TPM/secure enclave; attest gate and broker posture; seal audit chains for non-repudiation.
    • Support “attested HITL devices” for approvals in high-assurance settings (e.g., energy control rooms).
    • Assumptions/Dependencies: Platform TEEs; remote attestation protocols; operational key management.
  • Industry Standards and Certification (Policy, Cross-Sector)
    • Standardize capability vocabularies, manifest fields (incl. verification levels), and biconditional audit requirements; create “Verified Skill” certification programs.
    • Embed these controls in regulatory frameworks (e.g., ISO/IEC AI management systems, sector-specific guidance).
    • Assumptions/Dependencies: Multi-stakeholder consortia; interoperability test suites; conformance tooling.
  • Skill Supply-Chain Ecosystem (Software, Governance/Risk/Compliance)
    • Establish third‑party auditors/insurers that attest tested/formal levels; maintain CRLs/OCSP for signer key revocations; operate public skill registries with provenance and SBOMs.
    • Create CVE-like advisories for skill vulnerabilities and automated revocation workflows that evict in-flight skills on the next gate event.
    • Assumptions/Dependencies: Governance models; incident reporting norms; scalable revocation distribution.
  • Cross-Agent Skill Composition and Information-Flow Control (Software, Research)
    • Formalize label-join, declassification rules, and end-to-end information-flow across composed skills and agents; prove noninterference conditions for composed workflows.
    • Tooling to synthesize composite manifests and propagate verification levels.
    • Assumptions/Dependencies: Formal IF frameworks; compositional reasoning; richer capability semantics.
  • Reduced-HITL Autonomy in Low-Risk Domains (Robotics, Manufacturing, Logistics)
    • For tested/formal skills with strictly reversible operations, shift from per-call HITL to batch approvals or fully automated execution with continuous biconditional monitoring.
    • Apply in factories/warehouses for routine tasks with mechanical rollback.
    • Assumptions/Dependencies: Reliable rollback mechanisms; well-bounded environments; SLA-backed monitoring.
  • Advanced Egress-Aware Content Flow Controls (Healthcare, Enterprise)
    • Combine the biconditional with content-flow/DLP gates to address read-only exfiltration and covert channels the criterion does not catch today.
    • Introduce semantic egress policies (e.g., PHI/PII/financial data classifiers) tied to classification labels.
    • Assumptions/Dependencies: High-precision DLP/semantic filters; low false-positive policies; performance budgets.
  • Large-Scale Monitoring and SIEM/UEBA Integration (Security Operations)
    • Build dashboards and analytics over hash-chained audit logs; anomaly detection on request/decision/execute patterns; automated reconcilers that run the biconditional continuously.
    • Correlate agent behavior with broader SOC telemetry for incident response.
    • Assumptions/Dependencies: Log pipelines; data models; alerting thresholds.
  • Critical Infrastructure Gatekeeping (Energy, Transportation, Utilities)
    • Enforce capability gates before agents can interact with SCADA/ICS systems; classify and audit all actuation; mandate tested/formal skills for any irreversible control action.
    • Assumptions/Dependencies: ICS-compatible adapters; human factors for approvals; regulatory alignment.
  • Research Testbeds and Benchmarks at Scale (Academia)
    • Expand adversarial-ensemble benchmarks to large corpora and diverse toolsets (network, payments, schema migration); host shared leaderboards on biconditional pass rates under fixed policies.
    • Study TOCTOU and overlapping-effect failure modes with refined corpora and instrumentation.
    • Assumptions/Dependencies: Community datasets; reproducibility infrastructure.
  • Legal and Insurance Frameworks for AI Operations (Policy, Finance)
    • Make biconditional-compliant audit and untrusted-by-default gates prerequisites for liability safe harbors or cyber insurance discounts.
    • Define evidentiary standards for AI-triggered incidents based on hash-chained logs and reconciliation proofs.
    • Assumptions/Dependencies: Regulatory buy-in; actuarial models; standard evidence formats.
  • Open-Source Agent Hardening Kits (Software, Community)
    • Mature portable runtime profiles (G1–G12) into drop-in modules for popular agent runtimes; one-command “harden my agent” installers with strict/dev profiles.
    • Community-maintained adversarial test suites that run in CI across frameworks.
    • Assumptions/Dependencies: Sustained maintainer support; cross-framework APIs; adoption incentives.
View Paper Prompt  View All Prompts 

Glossary

  • adversarial-ensemble: A stress-testing setup using multiple adversarial agents to provoke harmful actions and evaluate defenses. Example: "We propose an evaluation shape we call the adversarial-ensemble sketch."
  • allow-list: A security policy listing explicitly permitted entities or operations; anything not listed is denied by default. Example: "compare against an a-priori-pinned allow-list rather than a heuristic classifier."
  • append-only log: A log structure that only supports additions, preventing modification or deletion of past entries. Example: "Every gate event ... is recorded in an append-only log whose records carry a prevHash field;"
  • Bell--LaPadula: A classic mandatory access control model using a security lattice for information flow control. Example: "a Bell--LaPadula style lattice"
  • biconditional correctness criterion: A correctness condition requiring a 1-to-1 correspondence between audited approvals and actual side-effects. Example: "a biconditional correctness criterion"
  • broker: An external decision component that approves or denies irreversible actions requested by the agent. Example: "consults a broker --- an opaque oracle the operator chose at deploy time ---"
  • capability bootstrap discipline: An object-capability principle mandating that trust and capabilities are fixed before any external input is processed. Example: "This matches the ``capability bootstrap discipline'' of \cite{ocap-discipline} adapted to skills:"
  • capability gate: A runtime layer that mediates tool calls by checking declared capabilities and policy before execution. Example: "a capability gate (\S\ref{sec:gate})"
  • capability sandboxing: Restricting components to a minimal set of declared capabilities to limit potential damage. Example: "typed artifacts, signed manifests, capability sandboxing, hash-chained audit, and least-privilege gates"
  • clearance-bounded signing: Constraining signers so they may only sign artifacts up to a maximum classification level. Example: "Locked trust root with clearance-bounded signing."
  • covert channel: An unintended communication path that allows information exfiltration without leaving expected traces. Example: "covert channel that bypasses both the corpus and the audit log."
  • CRLs: Certificate Revocation Lists used in PKI to signal revoked certificates. Example: "CRLs, OCSP staples, short lifetimes"
  • dead-letter queue: A message queue that stores undeliverable messages for later inspection or retry. Example: "an SQS queue write to a dead-letter queue under operator control is reversible."
  • declassification: The controlled lowering of information classification when certain conditions are met. Example: "declassification when output is provably a function of low-label inputs only is a research direction."
  • decentralized information-flow control: A model where principals manage their own labels and policies for information flow across a system. Example: "extended decentralized information-flow control to whole systems."
  • detached signature: A digital signature stored separately from the signed content, covering its canonical bytes. Example: "σ\sigma is a detached signature over the canonical bytes of (M,content)(M, content)"
  • DLP: Data Loss Prevention; mechanisms to detect and prevent sensitive data exfiltration. Example: "a regex-based DLP scanner,"
  • Ed25519 signed-module loader: A loader that verifies modules using Ed25519 digital signatures before admission. Example: "an Ed25519 signed-module loader with a clearance-bounded trust root,"
  • egress guard: A control that restricts outbound network communication from the runtime. Example: "an egress guard,"
  • hash-chained audit log: An audit log where each record includes a hash of the previous entry, enabling tamper-evidence. Example: "a hash-chained audit log,"
  • human-in-the-loop (HITL): A mechanism requiring explicit human approval for certain actions, typically irreversible ones. Example: "human-in-the-loop (HITL) gate"
  • irreversible side-effect: An action whose effects cannot be rolled back by the runtime alone. Example: "An irreversible side-effect is one for which the runtime cannot, alone, restore the prior state of the affected world."
  • machine-checkable proof: A formal proof encoded so that verification tools can automatically validate it. Example: "a machine-checkable proof that the skill's behavior is a subset of its declared capability set"
  • no bypass switch: A design invariant that forbids any configuration or flag that disables core safety controls. Example: "No bypass switch."
  • object-capability systems: Security architectures where authority is conveyed by possessing unforgeable references (capabilities). Example: "adapted from object-capability systems \cite{ocap-discipline}."
  • OCSP staples: Short-lived status proofs for certificates stapled into TLS handshakes to indicate non-revocation. Example: "CRLs, OCSP staples, short lifetimes"
  • out-of-band policy: A policy resource unreachable by the agent/model, used to ensure impartial, reproducible decisions. Example: "The broker reads an out-of-band policy document"
  • PKI: Public Key Infrastructure; systems and protocols managing keys, certificates, and revocation. Example: "a familiar PKI problem (CRLs, OCSP staples, short lifetimes)"
  • prototype-pollution: A class of attacks that modify an object's prototype to influence inherited properties. Example: "reject unknown fields, prototype-pollution keys, and missing mandatory fields."
  • ReDoS: Regular Expression Denial of Service, where crafted input triggers catastrophic regex backtracking to exhaust resources. Example: "ReDoS bounds,"
  • reversible side-effect: An action whose effects can be rolled back by the runtime without external coordination. Example: "A reversible side-effect leaves a single object in a state from which the runtime, holding a recent snapshot of that object, can return it to its prior state"
  • Time-of-check / time-of-use: A race condition where a checked state changes before use, leading to inconsistencies or exploits. Example: "Time-of-check / time-of-use races."
  • tool-call envelope: A structured request object emitted by the agent describing an intended tool operation. Example: "the agent takes one turn, emitting a tool-call envelope."
  • TPM: Trusted Platform Module; a hardware root-of-trust for secure key storage and attestation. Example: "a TPM, secure element, or equivalent"
  • transaction buffer: A mechanism that stages reversible changes for commit or rollback depending on outcomes or approvals. Example: "a transaction buffer with rollback."
  • trust root: The set of authoritative keys and constraints that define who may sign and at what clearance. Example: "The trust root is a finite, append-only set of signer entries"
  • verification level: An explicit manifest field indicating the extent of assurance (e.g., unverified, declared, tested, formal) for a skill. Example: "Every skill manifest carries a verification level"
  • webhook: An HTTP callback endpoint used to receive or delegate decisions/events to external services. Example: "via terminal, message bus, or webhook,"
View Paper Prompt  View All Prompts 

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 7 tweets with 472 likes about this paper.

Sign Up for Free