Chain-of-Trajectories: A Generative Planning Paradigm

• Chain-of-Trajectories (CoTj) is a modeling paradigm that treats entire predicted trajectories as structured sequences of states, actions, or latent representations.
• It applies across diffusion, language modeling, recommendation, and robotics by employing techniques like graph planning and autoregressive decoding.
• CoTj improves performance through explicit multi-step planning and adaptive computation allocation, yielding enhanced quality and global consistency.

Searching arXiv for the cited Chain-of-Trajectories-related papers to ground the article. Chain-of-Trajectories (CoTj) denotes a modeling perspective in which prediction or generation is organized around an explicit trajectory—a sequence of states, actions, latent representations, or diffusion timesteps—and the trajectory itself becomes the primary object of reasoning. In the narrow sense, CoTj is the inference-time diffusion framework introduced in "Chain-of-Trajectories: Unlocking the Intrinsic Generative Optimality of Diffusion Models via Graph-Theoretic Planning," where sampling is reformulated as shortest-path planning over timesteps using a low-dimensional difficulty signature called Diffusion DNA (Chen et al., 16 Mar 2026). In a broader, cross-domain sense suggested by related work, CoTj also describes architectures that generate future user behaviors in semantic-ID space, allocate variable-length latent reasoning trajectories before each token, or synthesize backward action chains anchored on a goal keyframe (Huang et al., 26 Jan 2026, Zeng et al., 9 Feb 2026, Zhang et al., 11 Jun 2025).

1. CoTj as a trajectory-centered modeling paradigm

The defining move in CoTj is to replace isolated next-step prediction with chained trajectory construction. Standard diffusion samplers, for example, operate on a fixed, content-agnostic time grid and therefore do not plan over possible denoising paths; CoTj instead predicts a per-input difficulty profile, plans a path over timesteps, and executes the base model only along that path (Chen et al., 16 Mar 2026). In recommendation, Generative Chain of Behavior (GCB) maps items into semantic IDs and autoregressively generates the next $k$ behaviors as a future trajectory rather than optimizing only a next-item objective (Huang et al., 26 Jan 2026). In language modeling, token-level adaptive latent CoT inserts a variable-length latent trajectory $z_t^{(1)},\dots,z_t^{(K_t)}$ before emitting each token, with $K_t$ learned through halting (Zeng et al., 9 Feb 2026). In robotic manipulation, Chain-of-Action (CoA) factorizes a trajectory in reverse temporal order so that the first generated token is a goal keyframe action and the remaining tokens form a backward action chain (Zhang et al., 11 Jun 2025).

Across these formulations, the trajectory may be discrete, continuous, or fully latent. The chain may be produced by shortest-path planning on a directed acyclic graph, by autoregressive decoding in a semantic code space, by repeated reuse of a Transformer stack under a 2D causal mask, or by reverse-time autoregression in action space. This suggests that CoTj is best understood not as a single architecture but as a family of mechanisms that impose multi-step structure, global consistency constraints, and explicit allocation of computation across a trajectory.

2. Diffusion CoTj: Diffusion DNA, graph planning, and Predict–Plan–Execute

In the diffusion formulation, CoTj addresses two stated problems of standard sampling: the curse of state dimensionality and computational misallocation (Chen et al., 16 Mar 2026). The state $\mathbf{x}_t$ is high-dimensional, so direct search over denoising trajectories in the noise manifold is intractable. CoTj avoids that search by abstracting each timestep into a scalar difficulty measure and planning in a low-dimensional time domain.

The central quantity is the Reconstruction Error Reference

$$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$

where $\mathbf{x}_t^*$ is the canonical state at time $t$ and $\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)$ is the model’s single-step estimate of clean data from that canonical state. The Diffusion DNA for an input is the vector

$$\mathcal{D} = (\mathcal{C}(t_0), \mathcal{C}(t_1), \dots, \mathcal{C}(t_T)).$$

In the paper’s formulation, $\mathcal{C}(t)$ is treated as an intrinsic error upper bound for denoising from time $z_t^{(1)},\dots,z_t^{(K_t)}$0, and the canonical trajectory $z_t^{(1)},\dots,z_t^{(K_t)}$1 is treated as the minimum-error reference relative to off-manifold realized states (Chen et al., 16 Mar 2026).

Trajectory planning is then expressed through a Trajectory Correction Cost,

$z_t^{(1)},\dots,z_t^{(K_t)}$2

with temporal lever

$z_t^{(1)},\dots,z_t^{(K_t)}$3

under the linear-flow derivation. CoTj constructs a Super-Node DAG whose nodes are timesteps plus a Super-Source $z_t^{(1)},\dots,z_t^{(K_t)}$4 and Super-End $z_t^{(1)},\dots,z_t^{(K_t)}$5. Edge weights are defined by

$z_t^{(1)},\dots,z_t^{(K_t)}$6

A trajectory is a path $z_t^{(1)},\dots,z_t^{(K_t)}$7 from $z_t^{(1)},\dots,z_t^{(K_t)}$8 to $z_t^{(1)},\dots,z_t^{(K_t)}$9, and planning reduces to

$K_t$0

This produces the paper’s Predict–Plan–Execute paradigm. First, a small predictor estimates $K_t$1 from the condition embedding. Second, a shortest-path solver selects an optimal timestep path, either with a fixed step budget or adaptively. Third, the planned path becomes the sampling schedule for an unchanged solver such as Euler, DDIM, DPM-Solver++, UniPC, or UCGM. The base diffusion model is not retrained or modified; “train-free” refers to the base diffusion model, with only a lightweight MLP DNA predictor trained once (Chen et al., 16 Mar 2026).

For adaptive-length planning, the stopping rule is expressed through the explained gain ratio

$K_t$2

and execution stops when $K_t$3, with $K_t$4 in the experiments. In the paper’s interpretation, this marks the point where further steps yield diminishing returns (Chen et al., 16 Mar 2026).

3. Empirical behavior of diffusion CoTj

The diffusion CoTj results are reported across Qwen-Image, Z-Image-Turbo, and Wan2.2, with evaluations on PickScore prompts, GenEval, FlowEdit, and VBench (Chen et al., 16 Mar 2026). Under fixed step budgets, CoTj improves quality without changing the underlying model. On GenEval with Qwen-Image, the baseline at 10 steps has overall 0.70, whereas CoTj at 10 steps reaches 0.85; at 50 steps, CoTj reaches 0.88 versus baseline 0.85. On Z-Image-Turbo, baseline 2-step overall is 0.58, while CoTj 2-step overall is 0.77; at 4 steps, baseline is 0.70 and CoTj is 0.78.

The solver ablation in the few-step regime is particularly explicit about trajectory planning. At 5 steps, baseline + Euler yields GenEval 0.428, baseline + high-order UCGM yields 0.528, CoTj + Euler yields 0.626, and CoTj + UCGM yields 0.775 (Chen et al., 16 Mar 2026). This supports the paper’s claim that changing the trajectory matters more than increasing solver order in the few-step regime, while the best performance comes from combining planned trajectories with higher-order solvers.

FlowEdit reconstruction results are used to characterize trajectory reachability. At 10 steps, baseline yields MSE 39.67 and PSNR 13.78, while CoTj yields MSE 15.36 and PSNR 20.36. At 25 steps, baseline MSE is 12.95 and CoTj MSE is 5.19, with PSNR 27.93 for CoTj (Chen et al., 16 Mar 2026). The paper interprets this as evidence that planned schedules move substantially closer to the 50-step reference latent optimum under the same budget.

For video generation with Wan2.2, the reported pattern is more nuanced. At 40 steps, baseline and CoTj are very close on VBench metrics, with CoTj slightly improving Motion Smoothness and Dynamic Degree while preserving subject/background consistency and aesthetics. At 10 steps, CoTj yields better Imaging Quality and similar subject/background consistency, while the baseline can show higher Dynamic Degree that qualitative analysis attributes to motion instability rather than true dynamics (Chen et al., 16 Mar 2026). In the paper’s reading, CoTj under tight budgets stabilizes the spatial manifold first and then builds motion.

The Diffusion DNA itself is also treated as a structural diagnostic. For Qwen-Image, DNA magnitude decays smoothly and step-wise reconstruction gain is strictly positive and monotone decaying. For SD3.5, late gains oscillate around zero. For Z-Image-Turbo, gain at $K_t$5 is negative and late gains remain high, which the paper associates with non-convergence and over-cooking when too many steps are used (Chen et al., 16 Mar 2026). CoTj uses these profiles to avoid regressive regions and truncate non-convergent tails.

4. Discrete semantic trajectories in sequential recommendation

In sequential recommendation, GCB is described as a concrete realization of CoTj over semantic behavior tokens (Huang et al., 26 Jan 2026). Each item is first encoded into a semantic ID using RQ-VAE with k-means refinement, producing a discrete latent space intended to preserve semantic proximity. The user history is then represented not as raw item IDs but as a sequence of semantic IDs, and a transformer-based autoregressive generator predicts a multi-step future trajectory in that same token space.

The model’s future factorization is

$K_t$6

where $K_t$7 is the tokenized history and $K_t$8 is the tokenized future. The implementation details specify code length $K_t$9, with the first 3 positions from RQ-VAE + k-means and a 4th position reserved for collision avoidance. The transformer uses 4 encoder layers, 4 decoder layers, hidden size 128, FFN dimension 1024, and 6 attention heads; optimization uses Adam, learning rate $\mathbf{x}_t$0, batch size 256, max epochs 300, a 50-epoch warm-up, and early stopping based on validation metrics (Huang et al., 26 Jan 2026).

The task is explicitly multi-step. Experiments use $\mathbf{x}_t$1, and inference performs beam search in semantic-ID space with beam size $\mathbf{x}_t$2. Evaluation extends next-item metrics to trajectory-level metrics: $\mathbf{x}_t$3_HR@K and $\mathbf{x}_t$4_NDCG@K for each horizon, MHR@K and MNDCG@K as means across horizons, SHR@K and SNDCG@K as sequence metrics, and SeqHR and SeqNDCG as geometric means of per-step metrics (Huang et al., 26 Jan 2026). The geometric-mean definition is presented as a way to penalize trajectories that are strong at early steps but weak later, thereby emphasizing trajectory consistency.

The reported results show that the horizon matters. On Beauty Seq-1, GCB achieves MHR@5 0.0454 versus FPMC 0.0446, SASRec 0.0327, STOSA 0.0354, and DCRec 0.0393; on Seq-2, MNDCG@10 is 0.0227 for GCB versus a best baseline around 0.0192; on Seq-3, MNDCG@5 is 0.0154 versus baselines around 0.0145 or lower. On Cell Phones & Accessories, Seq-1 MHR@5 is 0.0637 for GCB versus best baseline 0.0535, Seq-2 MHR@5 is 0.0261 versus baselines below 0.0171, and Seq-3 MHR@5 is 0.0252 versus baselines below 0.0185 (Huang et al., 26 Jan 2026). The stated pattern is that all models degrade as the horizon lengthens, but GCB degrades more slowly, which is presented as the benefit of jointly modeling future trajectories in a structured latent space.

5. Per-token latent trajectories in LLM pretraining

"Pretraining with Token-Level Adaptive Latent Chain-of-Thought" instantiates CoTj as a latent, per-token internal trajectory in hidden-state space (Zeng et al., 9 Feb 2026). Before emitting token $\mathbf{x}_t$5, the model produces a variable-length latent trajectory

$\mathbf{x}_t$6

where $\mathbf{x}_t$7 is learned adaptively. The trajectory is implemented by repeated passes of the same LLaMA-style Transformer stack under a 2D causal attention mask indexed by $\mathbf{x}_t$8, with

$\mathbf{x}_t$9

No new Transformer layers are introduced; the full stack is reused at each latent step.

Adaptive halting is controlled by a router head. The continuation gate is

$$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$0

with reach and exit probabilities

$$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$1

$$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$2

Threshold pruning removes tokens whose next-step reach probability falls below $$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$3, so adaptivity affects actual FLOPs during both training and inference. Because the process may be truncated at $$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$4, the exit distribution is mass-preserving, and the final latent representation is a weighted mixture,

$$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$5

Training is one-stage on general text, with the Pile corpus at 26B tokens and LLaMA-style architectures at 410M and 1.4B parameters (Zeng et al., 9 Feb 2026). The loss is

$$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$6

where the adaptive term penalizes continuing when the model is already confident: $$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$7 The paper characterizes the resulting behavior as an emergent curriculum of compute: easy tokens tend to execute approximately 0–1 extra steps, while difficult tokens approach the maximum latent depth.

Empirically, the adaptive latent CoT models achieve the lowest perplexity across baselines on The Pile, WikiText, and LAMBADA. Under matched compute budgets, they outperform Vanilla LLaMA, LoopedLM, PauseLM, PonderLM, PonderLM2, and Mixture-of-Recursions on downstream benchmarks including LAMBADA, SciQ, HellaSwag, PIQA, WinoGrande, ARC-E, ARC-C, and RACE. The paper further reports that the 410M model with $$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$8 and similar training compute outperforms a vanilla 1.4B LLaMA on average accuracy, and that the 1.4B model with $$\mathcal{C}(t) \equiv \mathbb{E}_{\mathbf{x}_0,\mathbf{z}}\bigl[\|\hat{\mathbf{x}}_0(\mathbf{x}_t^*, t)-\mathbf{x}_0\|^2\bigr],$$9 achieves better perplexity than PonderLM2 with less than half the FLOPs, 7.47 versus 17.47 $\mathbf{x}_t^*$0 (Zeng et al., 9 Feb 2026). A shared linear router already improves CE and yields about 12% pruning.

A recurrent misconception about chain-of-thought is that it must be verbalized. This work directly contradicts that assumption: the trajectory is continuous, latent, and never exposed as text, yet it is still treated as a CoTj mechanism because each token is produced after traversing a token-specific chain of internal states (Zeng et al., 9 Feb 2026).

6. Action-space CoTj in robotic manipulation

CoA implements CoTj directly in continuous action space (Zhang et al., 11 Jun 2025). The task setting uses a 7-DoF Franka Panda in RLBench and a Fetch arm in real-world experiments. Observations combine multi-view RGB images from 4 cameras at $\mathbf{x}_t^*$1 with proprioceptive state, while actions are 8D end-effector controls consisting of 3D position, 4D unit quaternion orientation, and 1D gripper open/close. A keyframe is identified using a heuristic from C2F-ARM—timesteps where the gripper state changes or joint velocities approach zero—and the next keyframe action $\mathbf{x}_t^*$2 is treated as the goal action.

The defining factorization is reverse temporal: $\mathbf{x}_t^*$3 During training, actions are encoded into continuous latent tokens, reversed, and decoded with teacher forcing in reversed time. During inference, the model starts from an SOS token, generates the keyframe first, and continues backward until dynamic stopping indicates that the backward-generated trajectory has reached the present. The stopping criterion is based on Euclidean distance between the decoded predicted action and the current end-effector pose,

$\mathbf{x}_t^*$4

CoA avoids action discretization by using continuous action tokens,

$\mathbf{x}_t^*$5

with a linear decoder back to action space. To stabilize autoregressive generation in latent space, it adds a latent consistency loss,

$\mathbf{x}_t^*$6

The architecture uses 4 ResNet-18 visual encoders, a 4-layer Transformer encoder, and a 7-layer Transformer decoder. CoA also adds reverse temporal ensemble, which aligns multiple backward trajectories at the keyframe, and multi-token prediction (MTP), which supervises local chunks during training through

$\mathbf{x}_t^*$7

The empirical results emphasize trajectory-level reasoning and spatial generalization. On 60 RLBench tasks, CoA achieves average success rate 0.552, compared with 0.389 for ACT and 0.326 for Diffusion Policy; it beats ACT on 81.7% of tasks and Diffusion Policy on 80% of tasks. On 10 highlighted tasks, CoA reaches 0.756 average success rate, compared with 0.488 for ACT, 0.416 for Diffusion Policy, and 0.644 for Octo. On 8 real-world kitchen tasks, CoA reaches 0.613 versus 0.463 for ACT (Zhang et al., 11 Jun 2025).

The extrapolation analysis on Push Button is especially diagnostic: CoA scores 0.94 in interpolation and 0.48 in extrapolation, while ACT scores 0.54 and 0.08, and Diffusion Policy scores 0.18 and 0.04 (Zhang et al., 11 Jun 2025). The ablations further isolate the CoTj ingredients: reverse autoregression reaches 0.756, forward autoregression 0.668, the hybrid variant 0.600, and the ACT baseline 0.488; replacing latent consistency with action consistency drops performance from 0.756 to 0.212; removing reverse ensemble drops it from 0.756 to 0.660. The paper interprets these results as evidence that a goal-anchored backward chain gives each local action explicit awareness of the final objective.

7. Comparative structure, common misconceptions, and open questions

Across the cited works, CoTj is not tied to one substrate or one learning regime. The trajectory can be a path of diffusion timesteps, a sequence of semantic behavior codes, a chain of latent token states, or a backward plan in continuous action space.

Domain	Trajectory substrate	Chaining mechanism
Diffusion sampling	Timesteps with Diffusion DNA costs	DAG shortest-path planning
Sequential recommendation	Semantic-ID tokens	Autoregressive future-sequence generation
Language modeling	Hidden-state trajectory $\mathbf{x}_t^*$8	Shared-stack latent iteration with halting
Robotic manipulation	Continuous action tokens	Reverse autoregressive goal-to-state decoding

Several misconceptions are directly contradicted by the literature. First, CoTj is not synonymous with explicit textual chain-of-thought: the language-model variant is fully latent, the robotics variant reasons in action space, and the diffusion variant reasons over schedules rather than words (Zeng et al., 9 Feb 2026, Zhang et al., 11 Jun 2025, Chen et al., 16 Mar 2026). Second, CoTj is not necessarily a training-time intervention: the diffusion formulation is an inference-time layer that leaves the base model untouched (Chen et al., 16 Mar 2026). Third, CoTj is not restricted to fixed-length reasoning: adaptive latent CoT uses token-wise halting, CoA uses distance-based dynamic stopping, and diffusion CoTj uses an explained-gain stopping threshold (Zeng et al., 9 Feb 2026, Zhang et al., 11 Jun 2025, Chen et al., 16 Mar 2026).

The open problems are domain-specific. In diffusion CoTj, the paper notes prediction errors in DNA, heuristic threshold choice for $\mathbf{x}_t^*$9, model-specific DNA behavior, and the absence of online correction during execution (Chen et al., 16 Mar 2026). In adaptive latent CoT, latent trajectories are not human-readable, the correctness-aware penalty is heuristic, and scaling beyond the reported 410M and 1.4B regimes remains open (Zeng et al., 9 Feb 2026). In GCB, performance depends on discretization quality, experiments focus on $t$0, and the two-stage pipeline plus beam search with $t$1 introduces cost (Huang et al., 26 Jan 2026). CoA demonstrates strong results, but its behavior is tied to keyframe heuristics, dynamic stopping in continuous action space, and the quality of backward goal anchoring (Zhang et al., 11 Jun 2025).

Taken together, these works indicate that CoTj is a general strategy for reallocating modeling capacity and computation from isolated local predictions to structured, multi-step trajectories. In diffusion, this means globally planning denoising schedules; in recommendation, it means generating future behavior blocks in a semantic latent space; in language modeling, it means allocating internal reasoning depth per token; and in robotics, it means synthesizing full action segments from a goal keyframe backward to the present. This suggests that the enduring contribution of CoTj is not a particular module, but a shift in what is treated as the primitive object of inference: not a single step, but a chain of trajectories.