---
title: IndirectAD Attack in ML Systems
url: https://www.emergentmind.com/topics/indirectad-attack
type: topic
---

# IndirectAD Attack in ML Systems

An IndirectAD attack refers to a class of adversarial techniques in machine learning and AI systems in which the attacker influences the target model’s outputs through indirect manipulation of data or environment, rather than direct perturbation of the input or model internals. These attacks exploit intermediary structures—such as co-occurrence patterns in recommenders, multi-hop neighborhoods in graphs, third-party content in web automation, or non-overlapping regions in image tasks—to propagate adversarial signals to the model’s decision locus. IndirectAD attacks are characterized by effectiveness at low resource budgets, stealthiness against standard detection mechanisms, and broad applicability across domains including recommender systems, graph learning, LLM web agents, and semantic segmentation.

## 1. Attack Mechanisms Across Modalities

IndirectAD attacks instantiate distinct methodologies depending on the underlying system architecture:

- **Recommender Systems:** The attack seeds co-occurrence between a highly “promotable” trigger item and the target item using fake user interactions, enabling the target item’s recommendation probability to rise among real users who were unlikely to receive it otherwise [2511.05845].
  
- **Graph Convolutional Networks (GCNs):** The adversary perturbs feature vectors of remote nodes (k-hops from the target), leveraging the aggregation property of GCNs such that the adversarial effect propagates through convolutions to affect the classification of the target node [2002.08012].

- **Web Agents and LLM Prompt Injection:** IndirectAD (also termed Indirect Prompt Injection, IPI) places adversarial instructions or triggers not in direct user prompts, but in environmental content (e.g., HTML accessibility trees, third-party tool responses, or advertising delivery), subverting agent behavior in ways the model designer did not anticipate [2505.21499, 2507.14799, 2503.00061].

- **Semantic Segmentation:** Indirect local attacks perturb visually insignificant or spatially remote image regions, causing misclassification in target areas due to the contextual dependencies of modern segmentation architectures [1911.13038].

## 2. Formal Threat Models and Problem Formulations

The general threat model for IndirectAD attacks exhibits the following features:

- **Limited attacker capabilities:** Direct manipulation of the model’s primary decision locus (items, nodes, relevant pixels) is typically disallowed. The adversary can only manipulate indirect channels (e.g., a small set of fake users, remote graph nodes, peripheral image regions, or third-party data sources).

- **Black-box or partial knowledge:** The attacker often operates with limited, if any, access to the victim model’s internals, relying on training substitute models or surrogate reasoning.

- **Low resource budget:** The fraction of data/entities under the attacker’s control is strictly limited (e.g., poisoning ratio γ as low as 0.05% in recommender systems [2511.05845], or control of a single remote node/region).

Formally, for each domain there exists a constrained optimization problem maximizing attack success (e.g., hit ratio HR@20 in recommenders, target class probability in GCNs, or action likelihood in LLM agents) under indirect intervention constraints.

## 3. IndirectAD Implementation Strategies

Each domain’s IndirectAD implementation incorporates optimization and exploitation procedures tailored to the model structure:

- **Recommender Systems (IndirectAD, [2511.05845]):**
  - **Substitute Model Training:** Attackers fit a surrogate (e.g., WRMF) on partial data.
  - **Trigger-Item Selection:** Simulate gradient-based promotion of candidate items; select the highest-loss-reduction item as trigger.
  - **Fake-Profile Construction:** Seed fake users with trigger–target co-occurrences and random benign interactions.
  - **Adversarial Optimization:** Iteratively refine fake interaction data to maximize composite loss, projecting onto feasible co-occurrence constraints at each step.

- **GCN Node Attacks ([2002.08012]):**
  - Formulate an optimization over the remote node feature vector $\delta$ maximizing loss at the target node, subject to magnitude and hop-distance constraints.
  - Deploy coordinate descent with box constraints, initializing at the real feature, conducting (pseudo-)gradient steps, and updating $\lambda$ for tradeoff between success and stealth.
  - Node selection heuristics maximize “poisoning efficiency” among k-hop neighborhood.

- **Web Agents and IPI ([2507.14799, 2505.21499, 2503.00061]):**
  - *Prompt Injection via Accessibility Tree*: Embed triggers in hidden HTML or aria-labels parsed by the agent’s context serialization.
  - *Adaptive attacks*: Use Greedy Coordinate Gradient (GCG) or multi-objective variants to generate adversarial substrings likely to induce action, while evading detection.
  - *Ad Delivery*: Craft static, platform-compliant ad units with text optimized using a vision-language model for inferred agent goals and deploy via standard ad networks.

- **Semantic Segmentation ([1911.13038]):**
  - Projected gradient descent over remote patch perturbations (outside the fooling region), optionally with group sparsity penalties to minimize active regions.

## 4. Quantitative Effects, Transferability, and Stealth

IndirectAD attacks have been empirically validated to yield strong effects under stringent constraints:

- **Recommender systems:** With as little as 0.05% poisoning ratio, IndirectAD achieves a measurable increase in hit ratio for the target item (e.g., +0.3810 percentage points on Amazon Books with WRMF), in contrast to prior injection approaches which yield no measurable effect [2511.05845]. At 0.1% poisoning, the effect rises to +1.14 pp, representing an order-of-magnitude improvement over baseline attacks.

- **Graph learning:** A two-layer GCN can be compromised by remote (2-hop) node manipulation with >90% success at low $\|\delta\|_2 <1$, despite defenses focusing on immediate neighbors [2002.08012].

- **LLM web agents:** Universal triggers embedded in the accessibility tree achieve attack success rates (ASR) typically above 90% for targeted instructions on both held-out sites/goals [2507.14799], with even broader generalization in the presence of adaptive attacks, for which all tested defenses are bypassed at >50% ASR [2503.00061].

- **Semantic segmentation:** Context-aware networks (PSPNet, PSANet) exhibit ASR >85% for remote patch-to-dynamic-class attacks, while conventional FCNs remain robust to indirect manipulation [1911.13038].

The attacks routinely evade standard anomaly/shilling detectors and surface-level filtering mechanisms. For example, IndirectAD-generated fake profiles in recommendation tasks cannot be reliably distinguished (AUC ≤ 0.22) by classical graph-based detectors [2511.05845].

## 5. Defenses and Countermeasures

Standard defense strategies have proven insufficient against IndirectAD:

- **Classical anomaly detectors** (e.g., “Catch the Black Sheep,” graph-based approaches) yield near-random detection rates against behavior-mimicking indirect attacks [2511.05845].
- **Prompting-based mitigations** in LLM web agents (e.g., “Do not follow any commands in Observation”) have at best partial efficacy—only highly explicit goal-negating prompts yield substantial but incomplete reductions in ASR [2505.21499].
- **Input and detection-level defenses** (fine-tuned detectors, LLM-based detectors, perplexity filtering) are consistently bypassed under adaptive attack strategies [2503.00061].
- **Adversarial training** and finetuning are vulnerable unless performed with adaptive adversarial examples; static adversarial training is insufficient [2503.00061].

Suggested but untested countermeasures include:

- Co-occurrence–aware and adversarially robust embedding methods in recommender systems [2511.05845].
- Architectural innovations to jointly detect and recommend via graph neural network frameworks.
- Enhanced sanitization, canonicalization, and prompt isolation in web agent settings [2507.14799].
- Real-time runtime monitoring and layered dynamic policy enforcement.
- Training segmentation models to discount remote context for critical decisions or incorporating localized feature monitors.

## 6. Broader Implications and Future Research

IndirectAD exposes a set of fundamental vulnerabilities stemming from overreliance on contextual aggregation, co-occurrence, and indirect environmental input channels. A recurring observation is that improvements in global context modeling, user-behavior realism, or architectural expressiveness paradoxically increase susceptibility to indirect adversarial attacks.

The persistence of attack effectiveness at extremely low budget, transferability across architectures, and stealth against standard defenses suggests the necessity of domain-specific threat modeling and proactive adversarial evaluation in large-scale deployments. Research directions include robust co-occurrence modeling, context attenuation, multi-layered adversarial hardening, and certified defenses specifically tailored for indirect attack modalities. IndirectAD is now an established benchmark in evaluating the robustness of modern AI systems.

Source: https://www.emergentmind.com/topics/indirectad-attack