Goal Hijacking in ML Systems

• Goal hijacking is an adversarial technique where attackers redirect a system’s high-level objective away from its intended task.
• Experiments demonstrate high attack success rates—up to 90% in some LLM prompt injection and training-time scenarios—highlighting the practical threat.
• Defense strategies like goal prioritization and structured prompting can significantly mitigate these attacks while preserving system utility.

Goal hijacking is a class of adversarial manipulation in which a machine learning system is induced to pursue an attacker-chosen objective instead of its intended user, system, or training objective. In the recent literature, the term covers several related phenomena: prompt injection that forces a LLM to emit a predetermined response; visual or environmental interventions that redirect multimodal or embodied agents toward alternative actions; and training-time manipulations that repurpose a model or a client-side learner for a different task while preserving apparent utility on the nominal one (Zhang et al., 2024, Huang et al., 2024, Yang et al., 2024, Salem et al., 2021). The unifying property is objective redirection rather than generic degradation: the attacker does not merely cause error, but attempts to substitute, override, or parasitize the system’s operative goal.

1. Conceptual definition and scope

Across the cited work, goal hijacking denotes a change in the system’s high-level objective. In LLM security, this often means overriding the original task associated with a user prompt or system instruction and replacing it with a target malicious response or a new task. In embodied settings, it means causing an agent or robot to ignore an instruction and execute an attacker-defined action or trajectory. In training-time settings, it means altering what the learned model effectively does, sometimes without reducing its apparent performance on the original task (Huang et al., 2024, Zhou et al., 10 Oct 2025, Salem et al., 2021).

A recurrent explanation for prompt-based variants is Instruction-Data Ambiguity: trusted instructions and untrusted external data are concatenated into a single token sequence, so malicious directives embedded in the data can be interpreted on equal footing with the intended instructions. In a formal classification setting, one formulation writes the perturbed input as $\tilde{x} = x \Vert s$, where $x$ is untrusted input and $s$ is an injected suffix; attack success is measured over originally correct samples by

$$\mathrm{ASR} = \frac{|\{ i \in S : f_V(I, x_i \Vert s_i) \neq y_i \}|}{|S|}.$$

This formalization captures the central property of goal hijacking: the output reflects the attacker’s task rather than the intended label or response (Liu et al., 15 Jan 2026).

Setting	Mechanism	Representative papers
LLM inference	Prompt injection, universal suffixes, pseudo-conversations, adversarial demos	(Zhang et al., 2024, Huang et al., 2024, Chen et al., 2024, Zhou et al., 2023)
Multimodal and embodied systems	Visual prompt injection, adversarial 3D objects, physical triggers	(Kimura et al., 2024, Yang et al., 2024, Zhou et al., 10 Oct 2025)
Training-time systems	Model hijacking, cloaks, training-hijacking, poisoned demonstrations	(Salem et al., 2021, Li et al., 2024, Erdogan et al., 2021)

This scope also clarifies what goal hijacking is not. It is distinct from untargeted degradation, where utility simply collapses, and from attacks that preserve the task but alter the decision rule. That latter distinction is explicit in later work on reasoning hijacking, which treats goal hijacking as changing the high-level task itself (Liu et al., 15 Jan 2026).

2. Prompt-based goal hijacking in LLMs

The most developed literature concerns LLM prompt injection. One line of work defines a Goal-Guided Generative Prompt Injection Attack (G2PIA) that replaces heuristic prompt crafting with an optimization target: maximize the KL divergence between the conditional output distributions under clean and adversarial prompts,

$\max_{x'} \mathrm{KL}(p(y|x), p(y|x')).$

Under the assumption that the conditional distributions are Gaussian with shared covariance, this becomes equivalent to maximizing the Mahalanobis distance between the clean and adversarial embeddings. The attack then approximately solves this objective subject to cosine-similarity and semantic-core-word constraints, using POS tagging and noun-verb-object parsing, synonym replacement, and an auxiliary LLM to generate injection text. The reported experiments on seven LLM models and four datasets show attack success rates up to 80% on SQuAD2.0, about 45% for math tasks, and about 40–50% for QA tasks, while remaining query-free, black-box, and low in computational cost (Zhang et al., 2024).

A more specialized formulation is universal goal hijacking, in which a fixed suffix $S$ is sought such that, for arbitrary prompts $I$ from a prompt distribution $\mathcal{K}$,

$$\mathbf{LM}(I \oplus S) = R^T \quad \forall I \sim \mathcal{K}.$$

The POUGH method argues that prompt organization, not only optimization, is decisive. It greedily samples semantically diverse prompts, ranks them by semantic similarity to the target response, and then applies iterative optimization to build a universal suffix. In the reported experiments on four open LLMs and ten target-response types, POUGH achieves 90.9% average ASR versus 83.8% for M-GCG, while using about 13.7% of the computational time; suffixes from naive single-prompt or manual methods show less than 6.3% ASR on the test set, whereas POUGH’s suffix shows over 80% average ASR on unseen prompts (Huang et al., 2024).

Another variant exploits conversational role parsing rather than direct imperative overrides. Pseudo-Conversation Injection appends a fabricated assistant reply to the initial prompt, followed by a malicious new user turn, so that the model treats the original task as completed and answers the falsified follow-up. The paper distinguishes targeted, universal, and robust pseudo-conversation strategies. In experiments on ChatGPT-4o, ChatGPT-4o mini, and Qwen 2.5, the targeted variant reaches 92.0%, 90.6%, and 84.5% success respectively, outperforming the classic “ignore the previous” attack, which reaches 78.7%, 77.5%, and 69.3% on the same models (Chen et al., 2024).

A related but structurally different attack surface arises in in-context learning (ICL). There, adversarial suffixes are attached not to the user query but to demonstration examples. The proposed Greedy Gradient-guided Injection (GGI) learns short suffixes that force the model to output an attacker-chosen label or harmful response while leaving the instruction and query untouched. The paper reports that on SST-2 and Rotten Tomatoes, the attack forces the target sentiment with 100% ASR across settings, and on AG News only GGI achieves near-100% ASR in the more complex multi-class case (Zhou et al., 2023).

Taken together, these studies show that prompt-based goal hijacking is not exhausted by explicit “ignore previous instructions” strings. It includes semantic-level divergence maximization, universal suffix learning, role-structure forgery, and demo poisoning within ICL. A plausible implication is that the operative vulnerability lies in how LLMs compose instructions, demonstrations, and untrusted context into a single generative state.

3. Multimodal and embodied forms of goal hijacking

In large vision-LLMs (LVLMs), goal hijacking can be induced visually rather than textually. Goal hijacking via visual prompt injection (GHVPI) draws a malicious instruction directly onto the input image, consisting of a goal-hijacking prompt and a target-task prompt. Evaluation on 500 cases sampled from the LRV Instruction dataset shows that GPT-4V reaches a 15.8% attack success rate, Gemini 6.6%, and several open-source models much lower rates. The paper further reports that successful GHVPI is strongly correlated with long-text OCR performance, with Pearson’s $r = 0.861$, and that a system prompt instructing GPT-4V to ignore instructions in the image reduces success from 15.8% to 1.8% without eliminating the attack (Kimura et al., 2024).

For embodied navigation agents, the hijacking medium can be the environment itself. A white-box attack on Vision-and-Language Navigation (VLN) optimizes the texture of a 3D object so that, once observed, a pretrained agent ignores its instruction and either terminates early or follows an attacker-defined multi-step path. The attack is trained on subsets of train data and evaluated on novel instruction-trajectory pairs from val-seen data. On R2R test episodes, 75.98% of agents terminate early at the attack location versus 0.98% in the unaltered environment; for trajectory-level hijacking, attack success rises to 21.6% versus 3.66%, while success on the original navigation objective drops from 82.4% to 53.9%. On RxR, the same study reports attack success of 20.5% versus 5.5% and an oracle success rate of 78.7% versus 35.4% (Yang et al., 2024).

In vision-language-action (VLA) systems, the attack can be inserted during training through physical objects used as triggers. Goal-oriented backdoor attacks (GoBA) poison demonstrations so that the model behaves normally without the trigger but executes attacker-defined, goal-oriented actions when the trigger is present. The paper introduces BadLIBERO and a three-level evaluation—“nothing to do,” “try to do,” and “success to do.” It reports that GoBA enables the victim VLA to successfully achieve the backdoor goal in 97 percentage of inputs when the physical trigger is present, while causing zero performance degradation on clean inputs. The study also finds that action trajectory and trigger color significantly influence attack performance, while trigger size has surprisingly little effect (Zhou et al., 10 Oct 2025).

These multimodal and embodied results show that goal hijacking does not depend on textual token streams alone. OCR, differentiable rendering, physical triggers, and action execution loops all create distinct pathways for redirecting model goals. This suggests that the notion of “prompt” in prompt injection is too narrow for the broader class.

4. Agentic and long-horizon action hijacking

In LLM-based agents, goal hijacking often takes the form of action hijacking: the attacker does not merely alter a textual response, but induces the planner to generate and execute a harmful action plan. One recent method, AI$x$0, targets agents with a “Brain,” “Memory,” and “External Tools,” especially retrieval-augmented systems. Its core idea is to extract action-aware knowledge from the agent’s memory, then construct semantically harmless Trojan prompts that cause the retriever to surface operation-relevant knowledge and the planner to assemble harmful plans from it. On three real-world applications, the reported average attack success rate is 84.30%, with the best case at 99.70%; the attack also achieves an average bypass rate of 92.7% against common safety filters and 59.45% against a dedicated defense (Zhang et al., 2024).

A distinct long-horizon variant is mid-task hijacking in browser and file agents. WebTrap divides the attack into a lure trap, an inertia trap, and a payload trap. Rather than opposing the user goal directly, it fuses the attacker goal with the user workflow so that the agent executes the malicious subgoal and then resumes the original task. The paper formalizes success by requiring entry into a restricted area during execution while still completing the user goal. In experiments on extended WASP and InjecAgent environments, WebTrap reaches ASR-end-to-end of 67%–92%, ASR-intermediate of 76%–96%, utility under attack of 75%–92%, and dual-goal success of 24%–48%; the best baseline reaches up to 59% ASR-end-to-end and at most 18% dual-goal success (Liu et al., 8 May 2026).

These agentic results expand the semantics of goal hijacking in two ways. First, they show that the adversarial objective may be formulated over plans and tool invocations rather than single responses. Second, they show that stealthiness can come from preserving overall usability rather than visibly replacing the original task. A plausible implication is that long-horizon interaction increases the attack surface because the agent continuously re-grounds its planning on newly observed content and accumulated history.

5. Training-time and model-level hijacking

A broader literature treats goal hijacking as a training-time repurposing problem. In model hijacking attacks against computer-vision models, the attacker poisons training data so that the resulting model supports a second, attacker-defined task while preserving utility on the original task. The central mechanism is the Camouflager, an encoder-decoder model that produces samples visually similar to the original training distribution but semantically aligned with the hijacking task. Two variants are introduced: Chameleon, using visual and semantic loss, and Adverse Chameleon, which adds an adverse semantic loss for more complex or overlapping domains. When MNIST is used to hijack CIFAR-10 or CelebA models, the reported ASR exceeds 99% with utility drop at or below 0.5%; in the more difficult CelebA $x$1 CIFAR-10 setting, Adverse Chameleon reaches 58.6%–73.7% ASR with utility drop at or below 3.8% (Salem et al., 2021).

The same general idea has been extended to federated learning. HijackFL avoids direct poisoning of model parameters and instead learns class-specific pixel-level perturbations, or cloaks, that align hijacking samples with original-task samples in feature space. The attack is executed by a single malicious client, which computes cloaks on its local model and applies them at inference time. The optimization objective minimizes feature distance between a cloaked hijacking sample and a class anchor of the original task, with an added negative-anchor term to reduce overlearning. The paper reports attack success rates above 90% for some tasks with virtually no degradation in utility on the original task, outperforming data- and model-poisoning baselines in the federated setting (Li et al., 2024).

Split learning exposes a related but distinct training-time threat: training-hijacking by the server. Because the server controls the backward signal, it can direct the client model toward a different task, such as outputting easily invertible values. SplitGuard detects this by inserting fake batches with randomized labels and comparing the resulting client-side gradients to those from regular batches. Under honest training, fake and regular gradients differ much more than two random subsets of regular batches; under hijacking, this contrast collapses. The paper reports 100% true positive rate for all policies and datasets, with detection within the first epoch, while maintaining negligible accuracy loss (Erdogan et al., 2021).

These studies show that goal hijacking need not be an inference-time override. It can also be a latent functional repurposing of the trained system itself. This makes the concept overlap with backdoors and poisoning, but not collapse into them: several of the cited works explicitly distinguish model hijacking from ordinary backdoors because the goal is a different task rather than a trigger-conditioned misclassification.

6. Defenses, boundaries, and relation to reasoning hijacking

The defense literature emphasizes that many successful goal hijacks exploit unresolved conflicts among model objectives. A prominent example is goal prioritization for jailbreaking defense. The key claim is that LLMs are trained to be both helpful and safe, but conventional SFT and RLHF do not explicitly specify which objective should dominate under conflict. Prompt-based goal prioritization at inference instructs the model to prioritize safety over helpfulness and asks it to produce internal thoughts plus a final response; training-based goal prioritization exposes the model to both helpfulness-first and safety-first instructions during fine-tuning and uses only safety-first instructions at deployment. Reported results include a reduction in ASR from 66.4% to 3.6% for ChatGPT at inference time and from 71.0% to 6.6% for Llama2-13B during training; even without jailbreak samples in training, ASR drops from 71.0% to 34.0% (Zhang et al., 2023).

Other defenses target the instruction/data boundary more directly. In the classification-oriented evaluation of goal hijacking attacks, prompt-based defenses such as warnings, reminders, and sandwiching reduce ASR for some methods, while structured prompting and safety alignment are more effective. The cited results indicate that StruQ and SecAlign bring ASR for goal hijacking attacks down to around 1–20% for most methods, although Topic Attack remains difficult and can still achieve 84.8% on SecAlign in one reported setting (Liu et al., 15 Jan 2026). In multimodal settings, the GPT-4V system prompt telling the model to ignore instructions in the image reduces GHVPI success from 15.8% to 1.8% but does not eliminate the threat (Kimura et al., 2024).

The boundary of the concept is clarified by the introduction of Reasoning Hijacking. That work argues that current safety research predominantly focuses on goal hijacking, defined as changing the high-level objective, while reasoning hijacking preserves the task and instead injects spurious decision criteria. The distinction is operational: goal hijacking changes “summarize emails” into “phish users,” whereas reasoning hijacking leaves the classification or moderation task intact but corrupts the model’s decision logic through injected heuristics (Liu et al., 15 Jan 2026). This suggests that defenses tuned to detect intent deviation may succeed against many goal hijacks yet miss attacks that preserve the visible task specification.

Several recurrent misconceptions are therefore corrected by the literature. Goal hijacking is not limited to explicit override strings; not limited to text-only systems; not limited to inference-time prompting; and not identical to generic unreliability. It is a technically diverse class unified by attacker control over the operative objective. The empirical record across LLMs, LVLMs, navigation agents, VLA models, federated learning, and split learning shows that objective redirection is a cross-paradigm vulnerability rather than a narrow prompt-engineering artifact.