Fine-Tuning Trajectory Simulation (FTS) Methods

• Fine-Tuning Trajectory Simulation (FTS) is a trajectory-centered approach that models the entire fine-tuning process as evolving paths rather than single endpoints.
• FTS spans domains like vision-language, diffusion models, agent interactions, and traffic simulation, enabling robust adversarial attacks, improved convergence, and enhanced recovery mechanisms.
• By leveraging operational mechanisms such as PGD perturbations, ODE/SDE rollouts, and closed-loop simulations, FTS methods provide actionable insights for optimizing fine-tuning and ensuring model safety.

to=arxiv_search.query വിവരം 全民彩票天天送json_string": "{\"search_query\": \"all:\"Fine-tuning Trajectory Simulation\\" OR all:\"trajectory simulation\\" fine-tuning\", \"start\": 0, \"max_results\": 10, \"sort_by\": \"submittedDate\", \"sort_order\": \"descending\"}"} to=arxiv_search.query വിവരം 菲律宾申博json_string": "{\"search_query\": \"ti:\"Simulated Ensemble Attack\\" OR id:(Wang et al., 3 Aug 2025) OR ti:\"Efficient Adjoint Matching\\" OR ti:\"AgentBank\\" OR ti:\"Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models\\"\", \"start\": 0, \"max_results\": 10, \"sort_by\": \"submittedDate\", \"sort_order\": \"descending\"}"} Fine-tuning Trajectory Simulation (FTS) denotes a trajectory-centered family of methods in which fine-tuning is analyzed or optimized through explicit trajectories rather than only through endpoint parameters or endpoint outputs. In the most specific recent usage, the Simulated Ensemble Attack (SEA) defines FTS as a mechanism that simulates how the vision encoder of a base vision-LLM might change under downstream fine-tuning in order to generate parameter-robust adversarial images (Wang et al., 3 Aug 2025). Closely related arXiv work uses the same trajectory-centered logic for denoising paths in diffusion fine-tuning, multi-step agent-environment traces, closed-loop traffic rollouts, information-maximizing experiment design, and model-lineage verification (Shin et al., 12 May 2026, Song et al., 2024, Chen et al., 6 May 2025, Wilson et al., 2017, Shang et al., 16 Jan 2026). Taken together, these works suggest that FTS is best understood as an umbrella label for methods that explicitly model, optimize, replay, or verify the trajectories induced by fine-tuning.

1. Scope and operational meanings

Across recent work, FTS refers to several distinct but structurally related objects. In SEA, the simulated object is a neighborhood of fine-tuned vision-encoder parameters; in diffusion work, it is the denoising path; in agent tuning, it is the interaction trajectory; in traffic simulation, it is the closed-loop rollout; and in lineage attestation, it is the joint evolution of parameters and knowledge (Wang et al., 3 Aug 2025, Shin et al., 12 May 2026, Song et al., 2024, Chen et al., 6 May 2025, Shang et al., 16 Jan 2026).

Domain	Trajectory object	Operational mechanism
Vision-language jailbreaks	Fine-tuning neighborhood of the vision encoder	Random Gaussian perturbation during PGD
Diffusion and flow models	Denoising or generative trajectory	ODE/SDE rollout, path reward, or adjoint-based optimization
Agents and traffic	Interaction or rollout trajectory	Supervised replay, RL fine-tuning, or stepwise forward simulation

This variety matters because concrete claims about FTS are domain-dependent. In some papers, FTS is an optimization device; in others, it is a data representation, a simulator, or a forensic construct. A plausible implication is that the term names a methodological stance—treating fine-tuning as a trajectory-bearing process—more than a single canonical algorithm.

2. Parameter-space FTS in transferable vision-language jailbreaks

In SEA, FTS is the mechanism by which the attack simulates how the vision encoder of a base VLM might change under downstream fine-tuning in order to generate parameter-robust adversarial images that transfer to unknown fine-tuned variants (Wang et al., 3 Aug 2025). The threat model is grey-box: the attacker has full white-box access to the public base VLM $\mathcal{M}_\theta$, but no access to the fine-tuned target models’ weights, fine-tuning data or procedures, training configuration, or any queries to the fine-tuned models during attack generation. The goal is to craft a single adversarial image $\mathcal{I}_{adv}$ that, when combined with harmful text queries, jailbreaks many fine-tuned variants in one shot.

The core FTS mechanism perturbs only the vision tower plus projector while keeping the LLM intact:

$$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$

Here $\Theta_0$ is a stored copy of the original base vision encoder parameters, the perturbation is re-sampled at every optimization step, and the noise scale is tied to per-layer weight statistics. Over PGD steps, the adversarial image is therefore optimized against a sequence of virtual models rather than a single parameter point. The two-stage jailbreak objective first injects toxic semantics by maximizing harmful sentences given only the image, then induces affirmative responses for harmful goal–target pairs; in the default SEA pipeline, this is combined with Targeted Prompt Guidance (TPG), which modifies the text input to

$$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$

Algorithmically, SEA is FTS + TPG + two-stage PGD. The hyperparameters specified for FTS are $\sigma=0.3$, step size $\alpha=1/255$, $T_1=500$, $T_2=500$, batch size $b=4$, and unconstrained optimization with pixel clipping only. Perturbations are applied for all 1000 PGD steps. The design rationale is explicit: standard PGD attacks overfit to a single parameter setting, whereas FTS approximates optimization over a local parameter neighborhood that is meant to represent potential fine-tuned variants.

The empirical results identify FTS as the primary driver of transferability. Standard PGD-based adversarial images can achieve very high ASR on the base model, but on fine-tuned variants—especially MIS safety-tuned models—they collapse to ASR $\mathcal{I}_{adv}$0 for Freeze Proj. (LLM+Vision) and Full FT across all budgets. By contrast, SEA attains ASR 99.68% on the base Qwen-2-VL-7B, 86.54% on the fully fine-tuned MIS Qwen-2-VL-7B, and 96.41% on the fully fine-tuned MIS Qwen-2-VL-2B. In the key ablation, “SEA w/o Encoder Perturbation” falls to 1.92% ASR on MIS and 26.35% on Omni, whereas full SEA reaches 86.54% and 91.15%, respectively (Wang et al., 3 Aug 2025).

The same pattern appears in toxicity evaluation. On a fully fine-tuned MIS Qwen-2-VL-7B model, no-attack toxicity on RealToxicityPrompts is low; transferred PGD images do not increase toxicity and can even reduce it; SEA images raise Perspective “Any” toxicity to 58.18%, Perspective “Toxicity” to 52.25%, and Detoxify “Any” to 49.46%. The paper therefore argues for inheritance-aware defenses and notes an important limitation: FTS is validated only on models fine-tuned from the same base, with poor transferability to models with different architectures such as GPT-4o or even next-generation variants within the same family such as Qwen2.5-VL (Wang et al., 3 Aug 2025).

3. Denoising-path FTS in diffusion and flow fine-tuning

In diffusion and flow models, FTS appears as explicit simulation and optimization of generative trajectories. “Efficient Adjoint Matching” reformulates reward fine-tuning as a stochastic optimal control problem and targets the major bottlenecks of original Adjoint Matching: stochastic simulation of full generative trajectories under a non-trivial base drift and backward ODE simulation of the adjoint state along each sampled trajectory (Shin et al., 12 May 2026). EAM chooses a linear base drift

$\mathcal{I}_{adv}$1

subject to memorylessness and kernel matching, which yields both cheap trajectory construction and a closed-form adjoint. Training-time trajectory simulation then reduces to sampling the endpoint $\mathcal{I}_{adv}$2 with a few-step deterministic ODE and sampling intermediate states via

$\mathcal{I}_{adv}$3

while the adjoint becomes

$\mathcal{I}_{adv}$4

The practical consequence is that EAM uses 10 ODE NFEs per training iteration versus 40 SDE NFEs for AM, eliminates backward adjoint simulation, and converges up to 4x faster while matching or surpassing AM on PickScore, ImageReward, HPSv2.1, CLIPScore, and Aesthetics (Shin et al., 12 May 2026).

“Diffusion-Sharpening” pushes the trajectory perspective further by optimizing denoising trajectories directly during training rather than only weighting a single random timestep (Tian et al., 17 Feb 2025). For a short partial trajectory $\mathcal{I}_{adv}$5, it defines a trajectory-level reward

$\mathcal{I}_{adv}$6

selects the best trajectory for supervised fine-tuning or the best and worst trajectories for a DPO-style RLHF update, and amortizes the search cost into the model parameters. The paper reports superior training efficiency with faster convergence and best inference efficiency without requiring additional NFEs. In the reported comparisons, RLHF Diffusion-Sharpening exceeds Diffusion-DPO and IterPO on CLIP Score, ImageReward, and MLLM Score, while preserving standard inference cost (Tian et al., 17 Feb 2025).

A related but domain-shifted instance appears in low-thrust astrodynamics. “Gradient-Informed Monte Carlo Fine-Tuning of Diffusion Models for Low-Thrust Trajectory Design” treats indirect low-thrust transfers as samples from an unnormalized density over initial costates, uses MALA or HMC initialized from a diffusion prior, and then fine-tunes the diffusion model with reward-weighted likelihood maximization,

$\mathcal{I}_{adv}$7

Here MALA raises the feasibility rate from 17.34% to 63.01% relative to random-walk Metropolis and yields denser Pareto-front coverage; the fine-tuned diffusion model then learns the global solution structure of the new mission and removes the need for a separate data-generation phase (Graebner et al., 9 Dec 2025).

These diffusion and flow formulations share a common principle: the object being fine-tuned is not only a score network or denoiser at isolated timesteps, but an entire trajectory distribution.

4. Interaction-trajectory FTS in agents and vision-language-action models

For agentic LLMs, FTS appears as supervised replay or online RL over full interaction trajectories. “AgentBank” defines trajectory tuning as supervised fine-tuning over full agent–environment interaction trajectories

$\mathcal{I}_{adv}$8

where $\mathcal{I}_{adv}$9 is the instruction, $$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$0 is the agent step in ReAct form, and $$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$1 is the environment observation (Song et al., 2024). Training uses masked autoregressive loss only on agent-side tokens:

$$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$2

with $$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$3. AgentBank contains 51,287 trajectories across 16 tasks and five skill dimensions, with an average of 3.9 interaction turns per trajectory. The final Samoyed models are trained on a mixture

$$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$4

and attain held-in/held-out averages of 41.6/29.5 for Samoyed-7B and 50.1/35.5 for Samoyed-13B, substantially above their Llama-2-Chat bases. The paper also shows that step-level CoT is especially important for held-out generalization, and that scaling the number of tasks and trajectories improves performance with diminishing returns (Song et al., 2024).

“TGRPO” moves from offline trajectory imitation to online RL fine-tuning of Vision-Language-Action models in LIBERO-Object (Chen et al., 10 Jun 2025). A trajectory is

$$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$5

and the central innovation is fused group-relative advantage:

$$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$6

where $$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$7 is the step-wise group-relative advantage and $$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$8 is the trajectory-wise group-relative advantage. The TGRPO objective uses these advantages without a learned critic and includes a KL term to a reference policy. On ten LIBERO-Object tasks, TGRPO reaches an average success rate of 91.0%, compared with 86.4% for SFT and 86.6% for PPO. The ablations show that trajectory-wise advantages are the main source of improvement, while the fusion of step-wise and trajectory-wise signals performs best overall (Chen et al., 10 Jun 2025).

These two strands make a clear distinction within FTS. AgentBank treats trajectories as supervision targets for behavioral cloning. TGRPO treats trajectories as on-policy rollouts whose rewards are normalized at both the step and episode levels. The common object is the full interaction trace rather than the single next action.

5. Closed-loop rollout FTS in traffic simulation

Traffic simulation has developed one of the most explicit FTS literatures because open-loop imitation learning suffers directly from covariate shift in closed-loop deployment. “Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models” proposes CAT-K, which keeps training fully supervised but changes the rollout distribution used during fine-tuning (Zhang et al., 2024). For each agent, at each step, the model considers the top-$$\mathcal{M}_\theta^{\text{vision}}=\Theta_0+\delta,\quad \delta\sim\mathcal{N}\Big(0,\big(\sigma\cdot\operatorname{std}(\Theta_0)\big)^2\cdot\mathbf{I}\Big).$$9 predicted motion tokens and selects the one whose next state is closest to the ground-truth next state:

$\Theta_0$0

The rollout then uses these CAT-K states, and recovery targets are retokenized from those rollout states. On WOSAC, a 7M-parameter SMART-tiny model with CAT-K fine-tuning reaches RMM 0.7635 and minADE 1.3206, outperforming the 102M SMART-large baseline at RMM 0.7614. The paper presents this as a purely supervised closed-loop fine-tuning strategy that mitigates covariate shift without RL or adversarial imitation (Zhang et al., 2024).

“RIFT” replaces supervised recovery with closed-loop RL fine-tuning in CARLA while preserving trajectory-level multimodality learned by a Pluto-style open-loop model (Chen et al., 6 May 2025). Its core mechanism is GRPO-style group-relative advantage over candidate trajectories, combined with a dual-clip objective that replaces KL regularization. Fine-tuning applies only to the trajectory scoring head, leaving the trajectory manifold learned by imitation intact. On the reported CBV realism and controllability metrics, RIFT-Pluto improves route progress, two-dimensional TTC, and distributional realism relative to IL-only Pluto and several RL or hybrid baselines, while the reward weights can be changed to induce “Normal” or “Aggressive” driving styles (Chen et al., 6 May 2025).

“RLFTSim” formalizes the same closed-loop alignment problem around the Waymo realism meta-metric (RMM) and introduces Meta-metric Leave-One-Out (MLOO) to obtain a dense, low-variance per-rollout reward from a population meta-metric (Ahmadi et al., 18 May 2026). The key reward is

$\Theta_0$1

and the goal-conditioned variant mixes realism and controllability via

$\Theta_0$2

On the WOSAC test leaderboard, RLFTSim achieves RMM 0.7867, with the best Interactive score among the listed methods and strong map-based realism, while also reducing passing miss rate under goal conditioning (Ahmadi et al., 18 May 2026).

“ForSim” modifies the forward simulation component inside the RIFT framework rather than only the policy loss (Chen et al., 2 Feb 2026). For each virtual step,

$\Theta_0$3

and the selected candidate trajectory is the one with minimum aligned ADE to a branch-specific reference modality. Other agents are updated through stepwise predictions rather than non-reactive replay. Relative to RIFT-Pluto, ForSim-Pluto reduces collisions per kilometer from 6.83 to 3.95, lowers uncomfortable rate from 76.90 to 57.12, and preserves strong realism and efficiency metrics (Chen et al., 2 Feb 2026).

A common misconception in this literature is that trajectory fine-tuning is equivalent to adding an RL objective on top of open-loop prediction. These results indicate that the rollout simulator itself—whether CAT-K recovery, dual-clip multimodal rollout, MLOO-based realism alignment, or stepwise forward simulation—is a first-class component of FTS.

6. Trajectory critics, experiment design, and optimization surrogates

Not all FTS formulations generate trajectories; some evaluate, select, or design them. “Motion Instruction Fine-tuning” treats the trajectory as an object to be semantically judged by a VLM (Hwang et al., 2024). Instead of feeding a full video, it overlays a single keypoint trajectory on the final frame, producing an image $\Theta_0$4, and asks the model to predict

$\Theta_0$5

where $\Theta_0$6 is the task instruction and $\Theta_0$7 is the motion description. MotIF-1K contains 653 human and 369 robot demonstrations across 13 task categories. The fine-tuned model improves precision by at least twice and recall by 56.1% relative to state-of-the-art VLM baselines, and the paper explicitly uses it to refine and terminate planning and to rank trajectories (Hwang et al., 2024). In FTS terms, this is a trajectory-level critic or reward model.

“Trajectory Synthesis for Fisher Information Maximization” formulates experimental-trajectory design as continuous-time optimization of a Fisher information criterion (Wilson et al., 2017). For nonlinear dynamics with measurement noise, it defines the continuous-time Fisher information analogue

$\Theta_0$8

and optimizes trajectories to maximize its minimum eigenvalue. In the double-pendulum cart experiments, the optimized trajectory increases the minimum eigenvalue of the Fisher information matrix by three orders of magnitude in simulation and yields an order of magnitude improvement in parameter estimate error experimentally. This is trajectory fine-tuning for system identification rather than for policy alignment (Wilson et al., 2017).

“Surrogate Neural Networks for Efficient Simulation-based Trajectory Planning Optimization” addresses a related setting in which only a high-fidelity black-box simulator is available (Ruff et al., 2023). A neural surrogate maps 13 trajectory design parameters to three touchdown metrics, is trained on 400 Sobol samples, and is then used inside a gradient-based optimization loop. For the hypersonic vehicle example, the method finds a 74% better-performing reference trajectory compared to nominal, and one comparison shows that reducing the loss from 0.16 to 0.13 takes about 5 intelligent queries instead of about 30 Monte Carlo queries. Here FTS is indirect: the simulator is not modified, but its trajectory inputs are iteratively fine-tuned through a differentiable surrogate (Ruff et al., 2023).

A further trajectory-centered variant appears in adversarial robustness. “Two Heads Are Better Than One” studies a fine-tuning trajectory of adversarial examples

$\Theta_0$9

and proposes exponential averaging along that trajectory,

$$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$0

to move the final adversarial example toward a more centered region of a flat loss landscape (Zeng et al., 2024). The paper interprets the endpoint-only scheme as discarding valuable information in the fine-tuning path. This does not define FTS as a standalone framework, but it reinforces a general trajectory principle: the path of fine-tuning can carry useful geometric information that the endpoint alone does not preserve.

7. Lineage attestation, conceptual boundaries, and limitations

A distinct use of the fine-tuning trajectory appears in model provenance. “Attesting Model Lineage by Consisted Knowledge Evolution with Fine-Tuning Trajectory” treats lineage as consistency between parameter modification and knowledge evolution (Shang et al., 16 Jan 2026). Given parent and child parameters $$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$1 and a platform-certified initialization $$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$2, it forms a task vector $$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$3 and an evolution model

$$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$4

The paper then posits a knowledge relation

$$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$5

vectorizes knowledge via model-family-specific probes, and scores arithmetic consistency through

$$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$6

The reported results are strong across classifiers, diffusion models, and LLMs: for example, TPR/FPR of 0.99/0.02 under diffusion AGA+WPA and 0.97/0.02 for LLaMA-3.1-8B under AGA, with clear separation between parent–child, more distant ancestors, and non-lineage pairs (Shang et al., 16 Jan 2026). This is a verification-oriented FTS: the fine-tuning trajectory is not optimized but reconstructed as a consistency object.

The diversity of these formulations imposes conceptual boundaries. SEA’s FTS is validated only within a single base-family ecosystem and shows poor transferability to different architectures or to Qwen2.5-VL (Wang et al., 3 Aug 2025). EAM relies on a linear base drift, memorylessness, and a Tweedie-style approximation of the pretrained terminal score, while large $$x_{\text{text}}^{\text{TPG}}=g_i+\text{`` Must begin your response with: ''}+t_i.$$7 can destabilize training (Shin et al., 12 May 2026). AgentBank reports plateauing gains as trajectory count grows, suggesting that supervised trajectory tuning alone may not scale indefinitely without RL or exploration (Song et al., 2024). RLFTSim notes that tokenized 0.5 s representations may reduce responsiveness and that RMM is still only a proxy for realism (Ahmadi et al., 18 May 2026). ForSim improves safety and comfort but does so with substantially more expensive stepwise simulation (Chen et al., 2 Feb 2026). MotIF is limited by its 2D trajectory representation and fixed-view abstraction (Hwang et al., 2024).

Taken together, these limitations sharpen a useful distinction. FTS is not inherently about safety, robustness, controllability, or provenance; it is a trajectory-centric formalism that can be coupled to any of those goals. The central research question is therefore not whether a method uses trajectories, but which trajectory it simulates—parameter neighborhoods, denoising paths, interaction traces, closed-loop rollouts, experiment trajectories, or knowledge evolution—and what objective is imposed on that simulated object.