Inter-Agent Anomalies in MAS

Updated 3 November 2025

Inter-agent anomalies are defined as systemic irregularities emerging from agent interactions, impacting overall security, stability, and execution in multi-agent systems.
Detection methodologies leverage graph-based models and semantic labeling to trace communication deviations and pinpoint protocol or trust failures.
Analytical techniques like root cause attribution and intention mining offer actionable insights for mitigating emergent failures and enhancing system resilience.

Inter-agent anomalies are failures, irregularities, or unwanted behaviors that emerge from the interactions, configurations, or communications among multiple agents within an agent system, rather than arising from faults within any single agent. In the context of LLM-powered multi-agent systems (MAS), these anomalies present unique operational, security, and analytical challenges due to the complex interdependencies and emergent phenomena inherent in distributed, autonomous systems. Unlike intra-agent anomalies, which concern reasoning, planning, or memory faults at the single-agent level, inter-agent anomalies are systemic and manifest only through inter-agent dynamics, often requiring specialized frameworks for detection, diagnosis, and mitigation.

1. Taxonomy and Definition of Inter-Agent Anomalies

Inter-agent anomalies, as defined in (Wang et al., 4 Aug 2025), comprise anomalous events that disrupt system-level behaviors, security, or stability through agent-to-agent interactions, network configuration, communication protocols, or emergent group dynamics. The principal categories are:

Task Specification Anomalies: Arise from ambiguous, underspecified, or conflicting task definitions or role assignments, leading to breakdowns in group execution or coordination.
Security Anomalies: Encompass attacks on the multi-agent substrate, including prompt injections, tool poisoning, communication protocol exploits, and adversarial collusion.
Communication Anomalies: Include message storms, excessive redundancy, and protocol-level failures that cause bottlenecks or resource exhaustion.
Trust Anomalies: Stem from failures in agent-level trust assessments, including acceptance of misleading information or the breakdown of cooperative trust structures.
Emergent Behavioral Anomalies: System-level behaviors not predictable from agents in isolation, e.g., feedback loops, deadlock, or self-reinforcing failure propagation.
Termination Anomalies: Manifest as improper task completion, infinite recursion, or premature/failed task exit.

Intra-Agent Anomalies	Inter-Agent Anomalies
Reasoning, Planning, Memory, Action	Task Specification, Security, Communication, Trust, Emergent Behavioral, Termination

A salient distinction is that inter-agent anomalies “are examined from a global perspective, moving beyond individual agents to focus on the interactions between different agents, as well as the overall system's security, stability, and task execution” (Wang et al., 4 Aug 2025).

2. Detection Methodologies and Systemic Modeling

Modern MAS anomaly detection frameworks leverage the system’s intrinsic network-like structure. A prototypical approach, as introduced by SentinelAgent (He et al., 30 May 2025), frames the entire MAS execution as a dynamic, labeled, directed interaction graph:

$G = (V, E), \quad V = \{\text{agents, tools}\}, \quad E = \{\text{interactions}\}$

Nodes encode agent roles, permissions, and tool capabilities.
Edges encode communication, tool invocation, or delegation events, annotated with semantic and status information.
Node/Edge Labeling: Semantic labels identify anomalies such as role violations, hallucinations (node), unauthorized transitions, or parameter covert manipulations (edge).

SentinelAgent realizes multi-tiered detection:

Global system output validation (violation of end-user policy/intent),
Single-point agent/tool anomaly localization,
Multi-point attack path detection through subgraph and semantic pattern matching.

Complementary frameworks, such as GUARDIAN and AgentPrune (Wang et al., 4 Aug 2025), further apply graph encoder-decoder models to reconstruct expected agent-system states, using high reconstruction loss to flag anomalous regions of the interaction graph (e.g., communication redundancy or structural drift).

For trust anomalies, attention-based trust scoring mechanisms (e.g., ATrust) evaluate the reliability of inter-agent communication by leveraging model attention statistics, factual consistency, and historical interaction logs.

3. Analytical and Root Cause Attribution Techniques

Root cause analysis in MAS contexts requires disentangling multi-agent, multi-layered causality. SentinelAgent performs explainable root-cause attribution by traversing the execution graph, extracting:

Entry points of exploits (e.g., where a prompt injection entered the system),
Propagation chains (which agents/tools relayed the exploit),
Node/edge vulnerabilities (which checks, policies, or communications failed).

The AgentOps operational stack (Wang et al., 4 Aug 2025) further organizes RCA into:

System-centric (infrastructure/API failures)
Model-centric (LLM failures and prompt misinterpretations)
Orchestration-centric (protocol, role, or task specification flaws)

Counterfactual replay and checkpointing allow for comparative analysis between failed and successful runs, supporting empirical causal inference and facilitating iterative refinement of detection policies.

4. Empirical Manifestations and Benchmarking

Systematic anomalies have been validated in case studies and benchmarks:

Email Assistant System (He et al., 30 May 2025): Multi-step collusion allowed a malicious prompt injection to bypass standard workflow, causing unauthorized mail replies and sensitive data leak—detected as an attack path via graph deviation and content constraints.
Cooperative Vehicle Perception (Bastola et al., 28 Jan 2025): Inter-agent anomalies manifest as anomalous driving behaviors (zigzag, tailgating) detectable only with fused perspectives from multiple agents. The CPAD framework’s graph transformer demonstrates robust anomaly recognition under agent dropout.
GUI-Robust Benchmark (Yang et al., 17 Jun 2025): In GUI agent collectives, cross-agent anomalies arise when one agent’s abnormal action (stuck on login/captcha, ad pop-ups) blocks the global task flow, exposing brittleness to real-world interface anomalies and calling for explicit anomaly escalation/recovery mechanisms.

Framework	Anomaly Modality	Detection Basis
SentinelAgent	Graph-structured attacks, collusion, delegation	Labeled execution graphs, subgraph matching
GUI-Robust	GUI interaction/system failures	Annotated natural user logs, response evaluation
CPAD	Spatiotemporal vehicular behavior	Graph-transformers, trajectory fusion
AgentOps	Systemic protocol, trust, security	Multi-dimensional root cause analysis, operational tracing

5. Abnormal Emergence and Collective System Effects

A distinct subclass of inter-agent anomalies involves abnormal emergence, where micro-level intention drift cascades into macro-level system failure modes. The EAMI framework (Shen et al., 21 Jul 2025) models this through LLM-powered intention mining, extracting and clustering parallel agent reasoning streams to identify phase transitions and collective intention shifts. The emergence of “involution” in synthetic O2O food delivery, marked by escalating competition and welfare loss, illustrates the detectability of such system-level anomalies.

Temporal emergence diagrams reveal how new intentions spread through agent populations, enabling causal linkage between microscopic anomalies and macroscopic system failures.

6. Governance and Resilience Mechanisms

Formal assurance of inter-agent integrity extends to foundational provenance and governance frameworks. For non-human identities in MAS, (Malkapuram et al., 22 Sep 2025) proposes cryptographically anchored lineage verification via append-only Merkle logs, enabling third-party auditors to validate entire agent action chains. Each agent action is signed, time-stamped, and anchored as a Merkle leaf, with federated proof servers supporting multi-hop, cross-organizational attestation. Any deviation (e.g., action omission, impersonation, forked logs) triggers validation failure, supporting anomaly detection at the provenance level.

Operationally, AgentOps (Wang et al., 4 Aug 2025) structures response as a continuous, empirical pipeline: monitoring, anomaly detection, RCA, and iterative resolution—emphasizing system-level observability, checkpointing, redundancy, and policy adaptation.

7. Research Directions and Open Challenges

Despite substantial progress, several classes of inter-agent anomalies remain unsolved:

Emergent behavioral and termination anomalies lack standard detection/mitigation methods (Wang et al., 4 Aug 2025).
Trade-offs exist between generic, protocol-based anomaly detection and domain-specific, context-aware approaches.
Graph-based and transformer-based frameworks show promise for large-scale automated anomaly attribution but may face scalability or real-time constraints in dense MAS deployments.

Future work necessitates unifying symbolic graph methods, LLM-driven semantic analysis, and cryptographic provenance, supported by public anomaly-prone benchmarks to drive robust, interpretable, and scalable solutions.

Inter-agent anomalies are fundamentally systemic, demanding global, topology- and protocol-aware detection, explainable attribution, and resilient, adaptive resolution strategies. Advances in graph-based modeling (He et al., 30 May 2025), intention mining (Shen et al., 21 Jul 2025), trust quantification, and cryptographic lineage assurance (Malkapuram et al., 22 Sep 2025) are converging to make MAS safety, security, and interpretability tractable at operational scale, but open questions on emergent failure, fault recovery, and cooperative robustness remain active research frontiers.