Trajectory-Based Backdoor Attacks

• Trajectory-based backdoor attacks are a class of threats that use dynamic, multi-step triggers to manipulate model outputs by altering entire trajectories rather than single-step decisions.
• They exploit vulnerabilities in systems like autonomous driving, reinforcement learning, and robotic control by selectively perturbing decision-making processes such as trajectory ranking and motion planning.
• Evaluation metrics such as Attack Success Rate and mean return drop highlight how these attacks maintain clean behavior under normal conditions while causing controlled deviations when triggered.

Trajectory-based backdoor attacks are backdoor mechanisms in which the trigger, the attack target, or the vulnerable internal variable is formulated as a trajectory rather than a static label or one-step output. Recent work uses this idea across motion forecasting, world-model planning, vision-language-action control, offline reinforcement learning, and prototype-based federated learning: the trigger may be an observed motion pattern of a nearby vehicle, a coordinated multi-vehicle behavior, a universal visual patch that reranks imagined futures, a brief visual cue that biases an action chunk, or a perturbation that steers prototype evolution over communication rounds (Pourkeshavarz et al., 2023, Messaoud et al., 2023, Chen et al., 21 Sep 2025, Duan et al., 3 May 2026, Xu et al., 20 Jan 2026, Dai et al., 15 Jun 2025, Zeng et al., 16 Sep 2025). The common property is selective activation: clean behavior is largely preserved, while trigger conditions induce a trajectory-level deviation that propagates into prediction, planning, or control.

1. Conceptual scope

The literature uses the term “trajectory-based” in several distinct but related senses. In autonomous-driving prediction, trajectories are observed and future vehicle motions, so the backdoor couples a motion-pattern trigger to a malicious future trajectory (Pourkeshavarz et al., 2023, Messaoud et al., 2023). In world-model agents, trajectories are imagined futures scored and ranked by a planner, making the attack surface the internal ordering of candidate rollouts rather than a single action output (Duan et al., 3 May 2026). In vision-language-action systems, trajectories are chunked delta-pose action sequences whose integration determines the executed end-effector path (Xu et al., 20 Jan 2026). In trajectory-optimization models for offline RL, trajectories are state-action-return segments used for autoregressive action reconstruction (Dai et al., 15 Jun 2025). In prototype-based federated learning, trajectory can even denote the round-wise evolution of global prototypes, because the attack manipulates prototype motion over communication rounds rather than directly overwriting global model parameters (Zeng et al., 16 Sep 2025).

This breadth distinguishes trajectory-based backdoors from conventional classifier-style or one-step policy backdoors. TRAP explicitly argues that attacks on local features, latent activations, next-step predictions, or immediate policy logits are poorly aligned with world-model planning, because planning depends on comparative evaluation over many imagined futures (Duan et al., 3 May 2026). The trajectory-prediction attacks likewise replace pixel-patch intuition with motion-pattern triggers performed by other agents in the scene (Pourkeshavarz et al., 2023, Messaoud et al., 2023). SilentDrift and TrojanTO make the same shift on the action side: the malicious effect is distributed over a trajectory, not concentrated in a single abrupt output (Xu et al., 20 Jan 2026, Dai et al., 15 Jun 2025).

2. Trigger modalities and threat models

Threat models vary sharply across the literature. In autonomous-driving trajectory prediction, one line of work studies training-time poisoning in a black-box, model-agnostic setting, where a small number of poisoned samples modify an attacking vehicle’s observed trajectory, replace its future with a malicious trajectory, and transform lane geometry so the poisoned scene remains realistic (Pourkeshavarz et al., 2023). A related offline-RL driving attack also uses training-time poisoning, but the trigger is environmental at deployment: one or more surrounding vehicles execute a prescribed trajectory pattern, and the offline dataset has already been poisoned with corresponding ego actions and rewards (Chen et al., 21 Sep 2025).

World-model planning introduces a different threat model. TRAP assumes a white-box attacker with access to the target world model and planner internals, but without the ability to tamper with parameters, retrain the model, alter environment dynamics, or poison the original training pipeline. The attack is a deployment-time inference attack using a universal visual patch trigger, spatially localized and bounded by an $\ell_\infty$ budget, with patched observations defined by $\tilde{o}_t = T(o_t,\delta)$ and $\delta = \epsilon \cdot \tanh(\rho)$ (Duan et al., 3 May 2026).

Robotic control work spans still other assumptions. SilentDrift is a black-box, training-time poisoning attack on VLA systems that inserts a red circular patch of radius $r=5$ px and transparency $\alpha=1.0$ only during a short key attack window; the default poisoning regime is under $2\%$, concretely one poisoned episode per task (Xu et al., 20 Jan 2026). TrojanTO attacks pretrained trajectory-optimization models by fine-tuning on only $10$ trajectories, approximately $0.3\%$ of the average dataset size, with a trigger inserted into selected dimensions of the current state in a context window (Dai et al., 15 Jun 2025).

The realism of triggers also spans a spectrum. Some attacks use visible patches because the key vulnerability lies in downstream sequential processing rather than in trigger naturalism itself (Duan et al., 3 May 2026, Xu et al., 20 Jan 2026). Others make the trigger semantic and physically plausible: a nearby vehicle’s rare but feasible motion pattern, a coordinated multi-vehicle maneuver, or a non-causal rear vehicle performing a braking or composite behavior that nonetheless manipulates the predictor (Messaoud et al., 2023, Chen et al., 21 Sep 2025). This suggests that trajectory-based backdoors are not tied to one trigger modality; they are unified more by where the malicious dependency is learned than by how the trigger looks.

3. Attack mechanisms and vulnerable decision structures

In world-model planning, the vulnerable structure is the planner’s ranking over imagined futures. The planner is abstracted as

$$a_{t:t+H-1}^{*} = \arg\max_{a_{t:t+H-1}} J\bigl(o_t, a_{t:t+H-1}\bigr),$$

and TRAP argues that imagined trajectory scores have a long-tailed distribution in which only a few decision-critical trajectories dominate choice. Its tail-aware ranking loss suppresses the clean-condition top tail of each trajectory via clean-versus-triggered score deviations $\Delta J_{b,k}$, aggregates per-trajectory tail scores with a softmin, and stabilizes optimization with a sign gate and a magnitude gate (Duan et al., 3 May 2026). The attack therefore changes planning by reranking candidate futures, not by uniformly lowering predicted return.

In VLA models, SilentDrift attacks the temporal structure of action execution. The core architectural vulnerability is the combination of action chunking and delta-pose control:

$\tilde{o}_t = T(o_t,\delta)$0

with the policy emitting a $\tilde{o}_t = T(o_t,\delta)$1-step chunk and executing it in an intra-chunk visual open-loop. The accumulated deviation after a poisoned chunk is

$\tilde{o}_t = T(o_t,\delta)$2

SilentDrift exploits this with a keyframe attack strategy that poisons only the critical final-approach phase and shapes the perturbation with the Smootherstep function

$\tilde{o}_t = T(o_t,\delta)$3

chosen because it gives $\tilde{o}_t = T(o_t,\delta)$4 continuity and zero velocity and acceleration at the attack boundaries (Xu et al., 20 Jan 2026). The result is a near-miss manipulation failure that appears kinematically natural.

In trajectory-optimization models, the mechanism is direct action-level fitting inside an autoregressive sequence model. TrojanTO argues that reward manipulation, effective in Bellman-style RL backdoors, is largely ineffective here because TO models reconstruct actions rather than optimize rewards directly. The attack instead poisons a single transition inside a context window, supervises the model toward a target action $\tilde{o}_t = T(o_t,\delta)$5, and combines trajectory filtering, batch poisoning, and alternating trigger-model optimization. Its poisoned batch has the form

$\tilde{o}_t = T(o_t,\delta)$6

and the trigger is optimized with MI-FGSM while the model is updated on mixed clean and poisoned supervision (Dai et al., 15 Jun 2025).

In autonomous-driving trajectory prediction, two distinct mechanisms appear. One is naturalistic poisoning: the attacker perturbs the observed trajectory of the attacking vehicle under kinematic bicycle-model constraints, then disguises the backdoor by transforming lane geometry through a bilevel optimization and gradient-alignment procedure (Pourkeshavarz et al., 2023). The other is direct trigger taxonomy over traffic motion: spatial triggers, temporal triggers such as braking or DAD, behavioral triggers such as synchronous motion, and composite triggers that combine spatial and temporal structure. In this line, the trigger may be produced by a non-causal vehicle behind the target, yet still force a trigger-activated response such as brake or curve (Messaoud et al., 2023).

In end-to-end driving and federated representation learning, the attacked trajectory can be multi-agent or latent. The temporal-logic driving attack specifies trigger trajectories with Signal Temporal Logic predicates such as

$\tilde{o}_t = T(o_t,\delta)$7

$\tilde{o}_t = T(o_t,\delta)$8

and

$\tilde{o}_t = T(o_t,\delta)$9

then refines them using a quantitative robustness score $\delta = \epsilon \cdot \tanh(\rho)$0 and adds negative training with patch trajectories that are similar to triggers but explicitly non-triggering (Chen et al., 21 Sep 2025). BAPFL, by contrast, attacks the trajectories of global prototypes in federated learning, reflecting trigger prototypes around a projection point so that benign clients’ local prototypes are gradually pulled away from trigger embeddings while target-specific triggers align with target-label global prototypes (Zeng et al., 16 Sep 2025).

4. Representative frameworks and design patterns

The recent literature does not present a single canonical architecture for trajectory-based backdoors. Instead, it exposes multiple design patterns tied to the sequential object being attacked.

Framework	Trajectory object	Main backdoor mechanism
TRAP (Duan et al., 3 May 2026)	Imagined trajectories in world-model planning	Tail-aware reranking of decision-critical trajectories
Naturalistic poisoning (Pourkeshavarz et al., 2023)	Observed and future vehicle trajectories	Trajectory trigger plus lane-transformation disguising
AutoBot backdoors (Messaoud et al., 2023)	Multi-agent traffic motion histories and futures	Spatial, temporal, behavioral, and composite triggers
SilentDrift (Xu et al., 20 Jan 2026)	Chunked delta-pose action sequences	Smooth trigger-conditioned drift during key approach phase
TrojanTO (Dai et al., 15 Jun 2025)	State-action-return segments in offline RL	Action-level target forcing in sequence reconstruction
TL multi-vehicle attack (Chen et al., 21 Sep 2025)	Coordinated attacker-vehicle trajectories	STL-specified trigger generation plus negative training
BAPFL (Zeng et al., 16 Sep 2025)	Round-wise global prototype trajectories	Prototype poisoning and target-specific trigger optimization

Across these frameworks, a recurring pattern is the selection of a high-leverage temporal locus. TRAP focuses on the high-score tail of imagined rollouts. SilentDrift attacks the final approach phase, described as the point of no return. TrojanTO poisons one random transition per batch so train-time and test-time trigger context match. The driving papers rely on rare but semantically coherent traffic interactions, and BAPFL steers cumulative prototype drift across rounds. Another recurring pattern is clean-behavior preservation: smooth perturbation profiles, naturalistic scene transformation, single-transition poisoning, negative training on near-miss triggers, or objectives defined through clean-versus-triggered deviations all reduce overt degradation outside trigger conditions.

5. Empirical behavior and evaluation

In world-model planning, TRAP is evaluated with Mean Return Drop and Attack Success Rate rather than classification accuracy. On DreamerV3 / Crafter, the random patch baseline gives $\delta = \epsilon \cdot \tanh(\rho)$1 mean drop and $\delta = \epsilon \cdot \tanh(\rho)$2 ASR, while TRAP achieves $\delta = \epsilon \cdot \tanh(\rho)$3 mean drop and $\delta = \epsilon \cdot \tanh(\rho)$4 ASR. On DreamerV3 / DMControl humanoid-walk, TRAP reaches $\delta = \epsilon \cdot \tanh(\rho)$5 mean drop and $\delta = \epsilon \cdot \tanh(\rho)$6 ASR; on TD-MPC2 cheetah-run it reaches $\delta = \epsilon \cdot \tanh(\rho)$7 mean drop. Under the constrained setting $\delta = \epsilon \cdot \tanh(\rho)$8, full TRAP achieves $\delta = \epsilon \cdot \tanh(\rho)$9 ASR and $r=5$0 Mean Return Drop, with lower performance when the sign gate or magnitude gate is removed. Mechanistic analyses show persistent latent divergence $r=5$1 across rollout depth and substantial off-diagonal mass in clean-to-triggered rank-transition visualizations, consistent with reranking rather than uniform score suppression (Duan et al., 3 May 2026).

In robot control and offline RL, the dominant metrics are Clean Task Success Rate, ASR, Benign Task Performance, and the harmonic-style combined score CP. SilentDrift reports, for VLA-Adapter averaged over LIBERO suites, Baseline SR $r=5$2, CTSR $r=5$3, and ASR $r=5$4; for $r=5$5, the corresponding averages are $r=5$6, $r=5$7, and $r=5$8. Its ablations show that moving from $r=5$9 to chunked control substantially increases ASR, and that keyframe-only poisoning is nearly as effective as earlier or full-trajectory triggering unless activation is too late. TrojanTO reports overall ASR $\alpha=1.0$0, BTP $\alpha=1.0$1, and CP $\alpha=1.0$2 across tasks, architectures, and target-action types, using only $\alpha=1.0$3 trajectories, about $\alpha=1.0$4 of the average dataset size. Its ablations show that removing batch poisoning lowers average ASR from $\alpha=1.0$5 to $\alpha=1.0$6 and CP from $\alpha=1.0$7 to $\alpha=1.0$8, while reward manipulation ablations indicate that changing reward values does not materially affect ASR or BTP (Xu et al., 20 Jan 2026, Dai et al., 15 Jun 2025).

In trajectory prediction and driving, evaluation shifts to trajectory-specific metrics such as tCA, tASR, ADE, FDE, cumulative reward, and mission violation rate. The naturalistic poisoning attack reports on nuScenes clean correctness around $\alpha=1.0$9–$2\%$0 with tASR around $2\%$1–$2\%$2, and on Argoverse tASR is often around $2\%$3–$2\%$4 while tCA often remains above $2\%$5–$2\%$6 at low poison rates; it already works at $2\%$7 poisoning (Pourkeshavarz et al., 2023). The AutoBot study shows that composite triggers can be learned with only $2\%$8 backdoored training data while clean-data degradation remains much less than $2\%$9 m; for the composite brake trigger, error against the unchanged original ground truth rises from ADE $10$0 m to $10$1 m and FDE $10$2 m to $10$3 m, showing a real redirection of the predicted future (Messaoud et al., 2023). The temporal-logic multi-vehicle driving attack often drives poisoned MVR to near $10$4, and its negative training substantially improves specificity: F-MVR drops from $10$5 to $10$6 on Easy / Sync-bypass, from $10$7 to $10$8 on Easy / Overtake, and from $10$9 to $0.3\%$0 on Medium / Overtake, while non-trigger reward rises sharply (Chen et al., 21 Sep 2025).

6. Defenses, limitations, and open problems

Existing defenses are mostly partial. In world-model planning, median filtering, JPEG compression, Gaussian smoothing, and bit-depth reduction all reduce TRAP somewhat, but none eliminates it; Gaussian smoothing gives the largest reduction in mean drop, while bit-depth reduction has limited effect (Duan et al., 3 May 2026). In naturalistic poisoning for trajectory prediction, gradient shaping yields only marginal improvements, roughly $0.3\%$1–$0.3\%$2 in tCA and $0.3\%$3–$0.3\%$4 reduction in tASR (Pourkeshavarz et al., 2023). In the AutoBot study, simple off-road checks detect some curve-like TARs but fail on brake TARs, whereas clustering future trajectories can reduce manual inspection on the $0.3\%$5-sample validation set to fewer than $0.3\%$6 samples while keeping at least $0.3\%$7 chance of finding a TAR (Messaoud et al., 2023). TrojanTO notes that reward-based defenses such as PolicyCleanse are mismatched because the attack channel is not reward-based (Dai et al., 15 Jun 2025). In end-to-end driving, trajectory smoothing helps more for the brake target than for sudden left turn, but DP-SGD remains largely ineffective, with MVR staying at $0.3\%$8 in the reported cases (Chen et al., 21 Sep 2025).

A separate line of work develops trajectory-aware defenses and detectors. ReAgent treats an LLM agent’s behavior as a trajectory of instructions, thoughts, actions, and observations, and detects backdoors by checking consistency between planning and execution; in database tasks it can reduce ASR to $0.3\%$9 or $$a_{t:t+H-1}^{*} = \arg\max_{a_{t:t+H-1}} J\bigl(o_t, a_{t:t+H-1}\bigr),$$0 in representative settings and by up to $$a_{t:t+H-1}^{*} = \arg\max_{a_{t:t+H-1}} J\bigl(o_t, a_{t:t+H-1}\bigr),$$1 overall (Changjiang et al., 10 Jun 2025). UniGuard treats the propagation trajectory of a sample through network layers as a time series, processes it with UMAP, a bidirectional LSTM autoencoder, FFT, and Deep SVDD, and reports about $$a_{t:t+H-1}^{*} = \arg\max_{a_{t:t+H-1}} J\bigl(o_t, a_{t:t+H-1}\bigr),$$2-level detection accuracy against BadNet, WaNet, ISSBA, Blend, and SSDT, including dynamic and partial backdoors (Fu et al., 28 Jun 2025). These works do not attack trajectories in the same sense as TRAP or SilentDrift, but they indicate that trajectory signatures are now central on the defense side as well.

Several open problems recur across the attack literature. Dedicated defenses for trajectory-ranking manipulation in world models remain underdeveloped. Black-box or partial-knowledge variants are still largely open in planning settings. Naturalistic, physically grounded triggers are more realistic but harder to isolate cleanly, which is why negative training becomes important in multi-agent driving. Automated selection of trigger dimensions and target actions remains unresolved in TO models, and subtle semantically aligned backdoors remain difficult to distinguish from ordinary preferences or control variability. A plausible implication is that these attacks should be studied not only as end-state failures but also as dynamic learning processes: backdoor learning curves parameterized by a poisoning interpolation variable $$a_{t:t+H-1}^{*} = \arg\max_{a_{t:t+H-1}} J\bigl(o_t, a_{t:t+H-1}\bigr),$$3 show that backdoor uptake depends on model complexity, poison fraction, and trigger size, and that there can be regions of hyperparameter space where clean accuracy remains high while backdoor effectiveness stays low (Cinà et al., 2021). This suggests that trajectory-based backdoor evaluation should measure when and how a backdoor is learned, not only final ASR or return drop.