Trajectory Alignment via Adversarial Perturbation (TAFAP)

• The paper introduces a novel method that uses constrained adversarial perturbations to continuously align deep model trajectories with expert paths, yielding robust and verifiable redirection.
• TAFAP leverages optimization techniques such as projected gradient descent and Adam to ensure perturbations remain imperceptible while targeting persistent outcomes across various domains.
• Empirical benchmarks confirm TAFAP’s superiority over traditional snapshot-based methods, demonstrating low prediction errors and high alignment fidelity in diffusion models, trajectory forecasting, and LiDAR perception.

Trajectory Alignment via Adversarial Perturbation (TAFAP) encompasses a family of methodologies aimed at algorithmically redirecting the output (predictions or model updates) of deep learning systems through the design of carefully constrained, imperceptible perturbations to inputs or training data. Unlike traditional adversarial attacks that merely degrade performance, TAFAP methods optimize perturbations to achieve targeted, persistent outcomes, often by aligning an attacker’s induced trajectory with a pre-selected expert path in parameter or prediction space. Recent advances have instantiated TAFAP in settings ranging from diffusion-model personalization to trajectory prediction and 3D sensory systems, with applications in privacy, anti-surveillance, and safety-critical autonomy (Lee et al., 11 Dec 2025, Tan et al., 2022, Li et al., 2021).

1. Core Methodological Framework

At its core, TAFAP formulates the problem as an optimization over a perturbation, denoted $\delta$, such that a model’s behavior—when presented with the perturbed input—closely matches a user-chosen target trajectory over a sequence of iterations or states. The design distinguishes itself from “snapshot” approaches by requiring persistent, stepwise alignment throughout training or inference.

In diffusion model personalization (Lee et al., 11 Dec 2025), TAFAP is framed bi-level:

• An expert trajectory $(\theta_1^t, \ldots, \theta_T^t)$ is precomputed by fine-tuning a model on target data.
• Given protected images $x_0^p$, a constrained $\delta$ is optimized so that subsequent fine-tuning of the model on $x_0^p + \delta$ yields parameter trajectories $(\theta_1^p(\delta), \ldots, \theta_T^p(\delta))$ satisfying $\theta_t^p(\delta) \approx \theta_t^t$ for all $t$.

In the context of trajectory prediction for autonomous agents (Tan et al., 2022), TAFAP seeks an adversarial modification $\Delta$ of historic agent trajectories $x$ such that the forecast $f(x+\Delta)$ matches a target trajectory $t$, under kinematic and physical safety constraints.

In LiDAR-based perception systems (Li et al., 2021), the adversary perturbs the ego-trajectory $R(t)$ through low-dimensional polynomials, so that the motion-compensated point cloud yields perception outcomes (e.g., object detections) that significantly deviate from ground truth, while remaining nearly indistinguishable at the physical-sensor level.

2. Mathematical Objectives and Optimization

TAFAP approaches formalize their objectives using variants of trajectory-matching losses subject to imperceptibility constraints.

Diffusion Model Case (Lee et al., 11 Dec 2025)

For images $x_0^p$ and precomputed target weight trajectories, the problem is

$$\delta^* = \arg\min_{\|\delta\|_\infty \leq \epsilon} \sum_{i=0}^{T-1} w_i \|\theta_{i+1}^p(\delta) - \theta_{i+1}^t\|_2^2$$

where $w_i$ normalizes per-step distances to ensure scale invariance across the trajectory.

Trajectory Prediction (Tan et al., 2022)

For nominal past input $x$, target $t$, and DNN predictor $f$, the goal is

$$\min_{\Delta} J(\Delta) = \sum_{m=1}^F w_m \|f(x + \Delta)_{t+m} - t_{t+m}\|_2^2$$

subject to $x_n + \Delta_n \in \mathcal{C}_n$, where $\mathcal{C}_n$ enforces physical plausibility—e.g., position, velocity, acceleration, and jerk bounds.

LiDAR Perception (Li et al., 2021)

The optimization runs over polynomial coefficients $a$ parameterizing perturbations $\delta(t)$:

$$\min_{a} L_{\text{detector}}(P^{\text{corr}}(R(t) + \delta(t)), Y)$$

s.t. $\|\delta\|_\infty \leq \epsilon$ and a trajectory smoothness constraint.

In all cases, variants of projected gradient descent or Adam optimizers are used, often with penalty or regularization terms tailored to the domain (e.g., smoothness of vehicle trajectories, image norm bounds).

3. Algorithmic Realization

TAFAP approaches feature a multi-stage optimization process that jointly unrolls system dynamics or training updates and backpropagates trajectory-matching losses through these sequences.

For diffusion models (Lee et al., 11 Dec 2025), the algorithm involves:

• Precomputing and storing the expert fine-tuning trajectory on target data.
• Iteratively unrolling blocks of $k$ fine-tuning steps for the student trajectory (on protected images plus perturbation), computing the loss relative to the expert trajectory segment, and updating $\delta$ via a sign-based (PGD-style) step with $\ell_\infty$-clipping.

In trajectory prediction (Tan et al., 2022), each iteration:

• Updates the adversarial trajectory $\Delta$ by Adam to reduce the target loss.
• Projects infeasible perturbations back onto the allowable set via per-step line search.

In LiDAR attacks (Li et al., 2021), the adversary optimizes polynomial parameters, balancing detection losses, trajectory smoothness, and explicit norm-bound projection at each step.

4. Comparison to Alternative and Snapshot-Based Approaches

Conventional snapshot-matching methods (e.g., Mist, Anti-DreamBooth) craft attacks that affect model behavior at a handful of discrete training checkpoints. However, these modifications can be “washed out” by continuing updates, resulting in transient or partial protection. TAFAP’s trajectory-based matching achieves persistence by explicitly shaping the full evolution of parameters or prediction state, yielding robust and verifiable redirection (Lee et al., 11 Dec 2025).

Similarly, in trajectory-prediction and LiDAR contexts, matching only the output at a single time step or sweep fails to guarantee that the full sequence (or aggregate detection outcome) reflects the target specification, especially under realistic system or sensor feedback (Tan et al., 2022, Li et al., 2021).

5. Empirical Evaluation and Benchmarks

Diffusion Model Personalization (Lee et al., 11 Dec 2025)

Experiments on CelebA-HQ and VGGFace2, using 12 images per identity and Stable Diffusion 1.4 (DreamBooth+LoRA), reveal that TAFAP markedly exceeds baselines (T-ASPL, Mist, ACE) in both redirection and image quality. Specifically:

Method	ISM_protect ↓	ISM_target ↑	BRISQUE ↓	SER-FIQ ↑
No defense	0.536	0.042	1.40	0.78
T-ASPL	0.226	0.147	23.7	0.48
Mist	0.368	0.108	26.7	0.65
ACE	0.405	0.177	32.7	0.67
TAFAP	0.202	0.393	11.5	0.80

TAFAP maintains low similarity to the protected identity and high similarity to the target, with robust performance under image manipulations (Gaussian blur, JPEG compression, resizing) and cross-model transferability.

Trajectory Forecasting (Tan et al., 2022)

Across multiple predictors (Grip++, Trajectron++) and datasets (Apolloscape, Nuscenes), TAFAP yields weighted post-attack errors $\bar J$ orders of magnitude lower than the target-trajectory separation, confirming precise alignment. For example:

Model/Dataset	$\bar J_{\text{acc}}^{\text{nom}}$	$\bar J_{G-Y}$	$\bar J$	Time (s)
Grip++ / Apolloscape	0.013	2.363	0.140	19.4
Trajectron++ / Nusc.	0.450	0.937	0.031	144.6

LiDAR Perception (Li et al., 2021)

On nuScenes with PointRCNN, TAFAP (polynomial and full-trajectory perturbations) reduces automobile AP at IoU 0.7 (Easy scenario) from 47.44 to as low as 0.19, outperforming coordinate or random perturbations under equal imperceptibility bounds.

Ablation studies clarify that degree-3 polynomial parameterizations balance attack strength and stealth, with increased smoothness constraint yielding minimal loss in efficacy.

6. Theoretical Underpinnings and Practical Considerations

TAFAP adapts the insight from dataset distillation—particularly Matching Training Trajectories (MTT)—whereby a small synthetic set can enforce a desired sequence of training states (Lee et al., 11 Dec 2025). Normalized losses and sign-based updates are crucial to avoid vanishing/exploding gradients over long unrolls.

In trajectory domains, constraint-based formalisms ensure that adversarial modifications remain undetectable and safe under physical models. Notably, TAFAP is more expressive than untargeted attacks, as it enables direct specification of desired output evolution or final state.

No formal global convergence guarantees are presently available in these nonconvex, high-dimensional settings, though empirical results and connections to distillation theory suggest practical robustness in trajectory alignment. Scalability to longer sequences is limited by storage needs for intermediary states (“expert trajectories”) and computational cost.

7. Limitations, Extensions, and Open Problems

Limitations of TAFAP include:

• Storage overhead for saving entire expert parameter or state trajectories, which scales with training/fine-tuning steps (Lee et al., 11 Dec 2025).
• The requirement for white-box access in many adversarial applications (Tan et al., 2022).
• Manifestation of partial or “blended” concept interpolation in certain image domains, potentially exploitable for controllable interpolation.
• Nonconvex optimization landscapes, with no guarantees of global optima.

Potential extensions include:

• Compression of expert trajectories via model-difference storage or lossy encoding.
• Generalization to arbitrary target attributes beyond identities (e.g., visual styles, objects).
• Formalization of convergence bounds for adversarially shaped trajectories and improved robust defenses (e.g., adaptive regularization or multi-sensor cross-checking) (Li et al., 2021).
• Explorations into automatic detection of adversarially shaped trajectories by flagging “unlikely” or inconsistent temporal signatures.

TAFAP establishes a unifying framework for trajectory-centric adversarial alignment, marking significant advances in verifiable control, protection, and manipulation of deep model behaviors in diverse domains (Lee et al., 11 Dec 2025, Tan et al., 2022, Li et al., 2021).