Runtime Behavioral Monitoring for Agent Skills

Develop runtime monitoring approaches that can reliably distinguish malicious agent actions from legitimate ones in deployments of the Agent Skills framework without relying on a formal behavioral specification and while maintaining low false positive rates.

Background

Because Agent Skills specify behavior in natural language and operate over an unbounded action space, there is no formal behavioral specification to anchor runtime checks.

The authors note that approaches like anomaly detection, behavioral fingerprints, and LLM-based intent classification remain unvalidated at scale, while excessive false positives could render monitoring impractical.

References

Developing runtime monitoring approaches that can distinguish malicious agent actions from legitimate ones---without a formal behavioral specification and without generating prohibitive false positive rates---is an open challenge.

— Towards Secure Agent Skills: Architecture, Threat Taxonomy, and Security Analysis  (2604.02837 - Li et al., 3 Apr 2026) in Section 7.2, Open Challenges (C3: Runtime Behavioral Monitoring)

To what extent this secondary information can help the agents in evading the monitor is not clear.

— Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure  (2609.30217 - Schmotz et al., 24 Sep 2026) in Section “How Do Agents Bypass the Monitoring Guardrails?”, subsection “Common Patterns”

Even with these improvements, preventing multi-context attacks at an acceptable cost remains an open problem.

— Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents  (2609.19587 - Remedios et al., 17 Sep 2026) in Abstract, page 1

Because SkillShift uses no explicit injection and preserves valid task outputs, whether such violations of Skill Policy Integrity can be detected remains unclear.

— A Finger on the Scale: Covert Policy Steering through Agentic Skills  (2609.02564 - Li et al., 2 Sep 2026) in Section 2, Related Work, subsection “Skill Attacks and Detection Gap”

The open question is how to combine lightweight, agent-aware hooks with enforceable system-level isolation while avoiding false positives that block legitimate scientific workflows.

— Towards Efficient HPC Systems for Agents: Challenges and Opportunities  (2609.38723 - Zheng et al., 30 Sep 2026) in Section Safety, subsection “Opportunities: System-Level Guardrails, Provenance Tracking, and Safe Secrets”

Future work should examine whether monitoring tool use during ordinary agent sessions identifies consequential actions that a sandbox run misses, and whether such monitoring can support intervention without disrupting legitimate tasks.

— Beyond Maliciousness: Static and Runtime Attack-Surface Profiling of Agent Skills  (2610.19264 - Pandey et al., 13 Oct 2026) in Section Future Work, paragraph “Monitoring after installation.”

However, it is not yet clear how accurately and reliably honeypots catch misbehaviors and how honeypots compare and combine with other types of monitors (see“Technical Research Agenda” for discussion of these open research questions).

— Honeypots for AI Agent Insider Threats  (2610.26664 - Wehner et al., 19 Oct 2026) in Section 3, “Detection: Catching Unauthorized Actions”; Section 8, “Technical Research Agenda”