---
title: Trajectory Stitching Definition, Methods & Examples
url: https://www.emergentmind.com/topics/trajectory-stitching
type: topic
---

# Trajectory Stitching Definition, Methods & Examples

Trajectory stitching is the construction of a new trajectory by combining states, actions, or sub-trajectories that occur separately in existing data or generated trajectories. In offline reinforcement learning (RL), it generally denotes recombining a useful prefix from one trajectory with a useful continuation from another, often by generating or selecting a connecting transition. The term is also used for diffusion-based planning, motion-primitive search, video stitching, hybrid language-model decoding, and flow-based inverse-problem solvers. Across these settings, the common operation is the creation of a coherent sequence from locally available trajectory segments, while the admissibility criterion—value improvement, dynamical feasibility, temporal consistency, spatial alignment, or model confidence—depends on the application.

## 1. Conceptual foundations and terminology

In offline RL, a dataset consists of trajectories collected by a behavior policy. A trajectory-stitching method seeks behavior that is better than any complete trajectory in the dataset by combining segments that are individually available but not jointly observed. One trajectory may contain a useful prefix but fail later, whereas another may contain a high-quality continuation from a relevant state. Stitching attempts to construct the missing connection.

This operation differs from ordinary behavioral cloning (BC). BC learns to reproduce observed state–action pairs and has no intrinsic mechanism for distinguishing actions from successful and failed trajectories. It can therefore imitate suboptimal behavior, suffer from state-distribution shift, and overfit when demonstrations are limited. Trajectory stitching instead changes the training distribution or generates a new plan before policy extraction.

The term does not imply a single algorithmic mechanism. Existing approaches include:

- **Observed-state recombination**: connecting states already present in an offline dataset and synthesizing only the connecting action or reward.
- **Model-based data augmentation**: generating intermediate states, actions, or rewards between source and target trajectories.
- **Latent or representation-level stitching**: using a learned embedding to identify compatible fragments and condition a policy on expert-like context.
- **Diffusion composition**: generating overlapping trajectory chunks whose local interfaces are jointly denoised.
- **Graph search over motion primitives**: concatenating dynamically feasible closed-form trajectory segments.
- **State-level retrieval**: retaining recorded predecessor transitions whose successors are similar to expert states.
- **Sampling-path switching**: replacing part of a generative trajectory with the trajectory segment of another model.
- **Token-level model routing**: interleaving tokens generated by small and large language models.

A central distinction is between **literal trajectory concatenation** and **trajectory reconstruction**. Literal concatenation joins recorded segments, whereas reconstruction generates a connecting action sequence, intermediate state sequence, latent segment, or model transition. The generated connection may be evaluated by a learned dynamics model, a value function, a consistency penalty, or an environment rollout.

## 2. Model-based stitching in offline reinforcement learning

The most direct offline-RL formulation is introduced by "Model-based Trajectory Stitching for Improved Offline Reinforcement Learning" [2211.11603] and "Model-based trajectory stitching for improved behavioural cloning and its applications" [2212.04280]. The method, called Trajectory Stitching (TS), improves a fixed dataset before applying ordinary BC. It searches for a candidate observed destination state \(\hat{s}'\) for an observed transition \((s,a,r,s')\), then replaces that transition with

\[
(s,\hat{a},\hat{r},\hat{s}'),
\]

where \(\hat{a}\) and \(\hat{r}\) are synthetic.

A candidate destination must satisfy two conditions. First, it must be dynamically plausible under a learned state-only forward model. Second, it must have higher estimated continuation value:

\[
V(\hat{s}')>V(s').
\]

The forward model is represented as a Gaussian conditional distribution,

\[
\hat p_\xi(s'|s)=\mathcal N(\mu_\xi(s),\Sigma_\xi(s)),
\]

and is trained as an ensemble to estimate epistemic uncertainty. The operational reachability criterion is conservative:

\[
\min_i \hat p_\xi^i(\hat{s}'|s)
>
\operatorname{mean}_i\hat p_\xi^i(s'|s).
\]

Candidate states are prefiltered using Euclidean neighborhoods, but Euclidean distance is only a computational shortcut; the learned likelihood performs the reachability test. The method generates the connecting action with an inverse-dynamics conditional variational autoencoder and predicts the synthetic reward with a conditional Wasserstein GAN. The state-value function is trained using a squared Bellman error, with two value networks and the minimum used as a conservative estimate.

TS performs one-step stitching between observed states rather than long model rollouts. A stitched trajectory has a prefix from one historical trajectory, a synthetic connecting transition, and a suffix from another. After processing a trajectory sequentially, TS accepts the modified trajectory only if its predicted cumulative reward exceeds the original by a margin:

\[
\sum_i \hat r_i>(1+\tilde p)\sum_i r_i,
\]

with \(\tilde p=0.1\) in the reported experiments. Accepted trajectories replace the originals; they are not simply appended to the replay buffer. BC is then trained on the resulting dataset, producing TS+BC.

On D4RL MuJoCo tasks—Hopper, HalfCheetah, and Walker2d—TS+BC improves over ordinary BC on the listed medium, medium-expert, medium-replay, and expert datasets, with particularly large gains on mixed-quality and replay data. Reported examples include Hopper-medium, \(55.3\) to \(64.3\pm4.2\), Hopper-medium-expert, \(62.3\) to \(94.8\pm11.7\), and Walker2d-medium-replay, \(34.7\) to \(61.5\pm5.6\). Gains are small on expert-quality data, where limited improvement is available, and Walker2d-expert changes from \(109.0\) to \(108.9\pm0.2\) in one reported table. A value-weighted BC ablation improves Hopper-medium from \(55.3\) to \(59.21\pm3.4\), but remains below TS+BC at \(64.3\pm4.2\), supporting the interpretation that trajectory reconstruction contributes beyond value weighting.

The method relies on observed-state coverage, local model accuracy, approximate Markov structure, reliable value ranking, and plausible reward generation. It does not establish a formal guarantee of true policy improvement under learned-model error. Its central limitation is that it can recombine useful observed states but cannot invent high-value states absent from the dataset.

## 3. Alternatives to Bellman-based stitching

"Free from Bellman Completeness: Trajectory Stitching via Model-based Return-conditioned Supervised Learning" [2310.19308] studies stitching through return-conditioned supervised learning (RCSL). RCSL predicts actions conditioned on state and return-to-go instead of applying Bellman backups. The paper argues that dynamic programming requires Bellman completeness of the value-function class, which is stronger than realizability and can be difficult to satisfy under function approximation.

The paper proves that Markovian RCSL cannot generally stitch. There exists a deterministic MDP and uniformly covering dataset for which a Markovian RCSL algorithm has a suboptimality gap of at least \(1/2\), even though every state–action pair appears in the dataset. A related Decision Transformer construction shows that complete state–action coverage does not guarantee an optimal policy when the optimal trajectory itself is absent.

To restore stitching, the paper proposes model-based return-conditioned supervised learning (MBRCSL). It learns a dynamics-and-reward model

\[
T_\theta(s',r\mid s,a)
\]

and a behavior policy \(\mu_\psi(a\mid s)\), then forward-samples trajectories. Only generated trajectories whose predicted return exceeds the maximum dataset return are retained. RCSL is trained on these model-generated trajectories. The model is therefore used for data generation rather than Bellman backups.

MBRCSL differs from TS in that it can generate multi-step trajectories through forward sampling, whereas TS performs conservative one-step connections between observed endpoint states. Its assumptions include learnable dynamics, sufficient behavior-policy coverage, an expressive multimodal behavior model, useful initial-state coverage, and a realizable final RCSL policy. The paper does not provide a finite-sample performance theorem for the complete MBRCSL pipeline.

In a Point Maze experiment, the offline dataset contains \(S\to A\to B\to G\) and \(S\to M\), while the optimal route is \(S\to M\to G\). MBRCSL obtains \(91.5\pm7.1\), compared with \(57.2\pm4.1\) for Decision Transformer and \(54.0\pm9.2\) for top-\(10\%\) BC. In simulated robotics tasks, MBRCSL reports success rates of \(0.48\pm0.04\) on PickPlace, \(0.51\pm0.12\) on ClosedDrawer, and \(0.68\pm0.09\) on BlockedDrawer. The results provide empirical evidence that model-generated trajectories can supply the missing compositions required by RCSL.

Other RCSL-oriented methods operate in latent context spaces. "Context-Former: Stitching via Latent Conditioned Sequence Modeling" [2401.16452] treats stitching as expert matching. A BERT-style hindsight-information encoder maps trajectory fragments to latent representations, and a trainable expert embedding \(z^*\) is learned to be close to expert representations and far from nonexpert representations. A latent-conditioned Decision Transformer then predicts actions using the expert latent throughout inference. This is latent-conditioned action substitution and continuation rather than literal trajectory concatenation. The method is strongest when a small number of expert demonstrations overlap with useful fragments in a larger suboptimal dataset.

## 4. Generative and diffusion-based trajectory stitching

Diffusion methods expand stitching beyond one-step state recombination. "DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching" [2402.02439] generates an intermediate sub-trajectory between the endpoint of a source trajectory and the entry point of a target trajectory. It first estimates the required bridge length by generating an imagined continuation and selecting the state most similar to the target entry state. A conditional diffusion model then inpaints the masked state sequence between the two endpoints.

Actions are inferred by an inverse-dynamics model and rewards by a reward model. A learned forward model qualifies the bridge by rejecting transitions whose predicted successor differs from the generated or target successor by more than a threshold. Accepted trajectories are added to a mixture of original and augmented data before training IQL, TD3+BC, or Decision Transformer.

On MuJoCo averages, DiffStitch obtains \(83.8\) with IQL, \(80.2\) with TD3+BC, and \(76.1\) with Decision Transformer. On Adroit-Pen, it obtains \(75.7\), \(20.6\), and \(64.9\), respectively. Its ablations show that fixed bridge lengths and the removal of the forward-model qualification reduce performance. Using only generated data is reported to be catastrophic, while mixing original and augmented data is important.

"Generative Trajectory Stitching through Diffusion Composition" [2503.05153] proposes CompDiffuser, which divides a trajectory into overlapping chunks and trains one bidirectional diffusion model to denoise each chunk conditioned on neighboring chunks. Interior chunks use neighboring noisy chunks; the first and last chunks use start and goal states. Autoregressive denoising passes a newly denoised chunk to the next chunk, improving interface consistency relative to parallel sampling. Overlapping regions are blended after denoising.

The approach provides compositional generalization without requiring a complete training trajectory for every long-horizon task. In PointMaze, CompDiffuser reaches \(100\%\) success on U-, Medium-, and Large-maze tasks and \(68\%\) on Giant. It reports \(96\%\), \(86\%\), and \(65\%\) on AntMaze Medium, Large, and Giant, respectively. Limitations include manual selection of the number of chunks, accumulated local-generation errors, increased inference cost, and the possibility that intermediate chunks become dynamically infeasible.

"State-Covering Trajectory Stitching for Diffusion Planners" [2506.00895] uses reward-free augmentation. SCoTS learns a temporal-distance-preserving representation \(\phi\), retrieves candidate segments whose initial states are close to the current endpoint in latent space, and selects among them using directional progress and novelty:

\[
S_j=P_j+\beta N_j.
\]

A conditional diffusion stitcher refines the connection between endpoints, and inverse dynamics reconstructs actions. The method augments datasets for diffusion planners and offline goal-conditioned RL.

On PointMaze Stitch, SCoTS reports \(100\%\) success on Medium, Large, and Giant. On AntMaze Stitch, it reports \(97\%\), \(93\%\), and \(87\%\), and on AntMaze Explore, \(99\%\) and \(98\%\) on Medium and Large. Average success rates for GCIQL, CRL, and HIQL increase from \(12.6\), \(8.4\), and \(36.4\) on original data to \(20.7\), \(27.9\), and \(73.4\) with SCoTS augmentation. Its temporal-distance representation is symmetric, however, so it may be inadequate for asymmetric reachability, irreversible actions, or one-way environments.

"Improving Trajectory Stitching with Flow Models" [2505.07802] proposes Flow Planner (FP), a conditional flow-matching planner designed to compose local trajectory patterns. It uses a temporal UNet, inpainting-based boundary conditioning, independent action-noise augmentation during data collection, and trajectory splitting during training and inference. The reported stitching error is \(0.16\pm0.07\), compared with \(0.71\pm0.24\) for a directly conditioned UNet and \(1.00\pm0.97\) for the MPD baseline. On a Franka Panda, the method generates obstacle-avoiding trajectories despite training data consisting mostly of straight-line motions. The method requires local regularity, compatible local fragments, smoothing after action-noise augmentation, and trajectory lengths compatible with the UNet structure.

" What Do You Need for Diverse Trajectory Stitching in Diffusion Planning?" [2505.18083] identifies local receptiveness and positional equivariance as the main properties supporting compositional diffusion planning. Local receptiveness restricts each denoising update to a bounded neighborhood, while positional equivariance allows the same local pattern to appear at different sequence positions. The proposed Eq-Net uses stride-one convolutions without pooling or downsampling. In a \(6\times6\) grid with lava pits, Eq-Net achieves an average completion rate of \(83.3\%\) on unseen start–goal pairs, approximately \(7.5\) times that of the locality-only Eq-Net + PosEnc configuration. The paper concludes that locality is more influential than positional equivariance in the tested tasks, although both are important.

## 5. Dynamics-guided, retrieval-based, and planning-oriented variants

"Robust Offline Imitation Learning Through State-level Trajectory Stitching" [2503.22524] applies stitching to offline imitation learning with a small expert dataset and a larger imperfect dataset. Its state-based retrieval (SBR) method learns a dynamics-aware latent representation using an encoder, latent dynamics predictor, and decoder. Similarity is defined as

\[
S(s_i,s_j)=-\|q_\phi(s_i)-q_\phi(s_j)\|_2.
\]

When an imperfect state is sufficiently similar to an expert state, SBR retrieves its recorded predecessor and associated action. The process expands backward iteratively, creating training data that describe transitions from imperfect or off-distribution states toward expert-like states. Unlike generative stitching, the retrieved transitions are observed transitions from the original data.

SBR then trains BC on expert and retrieved useful data. It reports best or competitive results on seven of eleven D4RL tasks, including \(95.6\pm6.28\) on pen-cloned-v1, \(25.6\pm15.9\) on door-cloned-v1, \(119\pm0.90\) on hammer-cloned-v1, and \(69.9\pm6.38\) on relocate-cloned-v1. In real-world tasks—Pour Tea, Clean Whiteboard, and Pick Pepper—the paper reports an improvement exceeding \(23\%\), although exact per-task rates are not supplied in the provided text. Its limitations include reliance on latent similarity, absence of explicit cross-fragment physical compatibility checks, and a fully observable MDP formulation that does not formally address POMDP inference.

"ASTRO: Adaptive Stitching via Dynamics-Guided Trajectory Rollouts" [2511.23442] combines temporal-distance target selection with a sequence-level dynamics diffusion model and an action stitch planner. The temporal-distance representation selects targets according to an approximate number of environment steps rather than raw Euclidean distance. Candidate masked sequences are filtered by temporal-consistency error. The action planner is guided by Rollout Deviation Feedback (RDF), defined as the discrepancy between the desired state sequence and the state sequence predicted by rolling out a candidate action sequence through the learned dynamics model.

ASTRO uses RDF during action denoising and includes self-correction and deviation-regularization losses. The final augmented states are generated by rolling out the planned actions through the sequence-level dynamics model. With IQL, average performance increases from \(36.08\) to \(45.52\); with FQL, it increases from \(55.52\) to \(65.71\). On AntMaze-Large, the reported dynamics-violation rate decreases to \(5.3\%\). ASTRO’s limitations include computational overhead, dependence on temporal-distance and dynamics-model accuracy, and the possibility of model error outside the data distribution.

STITCHER is a separate planning-oriented use of the concept. "STITCHER: Real-Time Trajectory Planning with Motion Primitive Search" [2412.21180] constructs long trajectories by graph-searching over short dynamically feasible motion primitives. A geometric A* search first generates a collision-free waypoint path. A velocity graph supplies an admissible double-integrator cost-to-go, and a higher-order motion-primitive A* search selects closed-form trajectory segments that connect the waypoints.

Adjacent primitives share boundary position and velocity; acceleration continuity can be handled by augmenting graph nodes with compatible acceleration states. Candidate primitives are checked for state constraints and collision using point-cloud queries and cached free spheres. The method is optimal only relative to the selected waypoint path, finite velocity and acceleration samples, primitive class, and collision-checking discretization. It reports millisecond-to-subsecond planning times for trajectories spanning tens of meters and outperforms the evaluated optimization-based planners in reported computation-time comparisons.

## 6. Broader uses, theoretical issues, and limitations

Trajectory stitching is not restricted to offline RL or planning. In video stitching, "Eliminating Warping Shakes for Unsupervised Online Video Stitching" [2403.06378] and "StabStitch++: Unsupervised Online Video Stitching with Spatiotemporal Bidirectional Warps" [2505.05001] define a stitching trajectory as the motion of a mesh vertex after spatial warping and temporal camera motion have been combined. For a spatial mesh \(M^S(t)\) and temporal mesh \(M^T(t)\), the relative stitching motion is

\[
s(t)=\mathcal T_{M^{Rig}\to M^S(t-1)}(M^T(t))-M^S(t),
\]

and the accumulated trajectory is

\[
S_i(t)=\sum_{\tau=1}^{t}s_i(\tau).
\]

SmoothNet modifies the spatial mesh so that these trajectories are temporally smooth while preserving alignment and mesh shape. StabStitch++ adds bidirectional warping through a virtual midplane and an online-alignment term. On StabStitch-D, it reports alignment of \(30.88/0.898\), stability \(41.70\), and distortion \(0.371\), compared with \(29.89/0.890\), \(48.74\), and \(0.674\) for StabStitch.

In diffusion-model sampling, "T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching" [2402.14167] uses a small denoiser for early, high-noise sampling steps and a large denoiser for later refinement steps. The intermediate sampling state is passed directly from the small model to the large model. On DiT-XL, replacing approximately \(40\%\) of early steps with DiT-S yields roughly \(1.5\times\) speedup with little quality degradation in the reported configuration. The method is training-free, but depends on compatible model distributions and degrades when the small model is used too far into late refinement.

In language-model reasoning, "R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning" [2507.17307] interleaves a small language model and a large language model at token level. The small model generates by default; low confidence triggers the large model, which replaces the uncertain token and continues until its confidence recovers. Separate key-value caches and partial prefill avoid full recomputation. The method reports up to \(85\%\) latency reduction in mathematical-reasoning benchmarks, but confidence is an imperfect proxy for correctness and the final sequence is not guaranteed to equal pure large-model decoding.

For off-policy evaluation, "STITCH-OPE: Trajectory Stitching with Guided Diffusion for Off-Policy Evaluation" [2505.20781] generates short target-policy trajectory windows with diffusion, guides them using the target-policy score minus a behavior-policy score, and concatenates the windows through shared boundary states. If the window size is \(w\), the likelihood-ratio dependence is \(\kappa^w\) rather than \(\kappa^T\) under the paper’s bounded-ratio assumptions. The theoretical MSE bound therefore replaces full-horizon exponential dependence with dependence on the local window length and polynomial dependence on the number of windows. The method remains dependent on behavior coverage and learned window-distribution accuracy.

Finally, "Trajectory Stitching for Solving Inverse Problems with Flow-Based Models" [2602.08538] applies multiple shooting to flow-based inverse problems. Instead of optimizing one initial latent through an entire flow trajectory, MS-Flow introduces intermediate states and penalizes local defects between independently optimized states and short flow integrations. Its objective combines measurement fidelity, an initial-state prior, terminal coupling, and local trajectory-matching penalties. This reduces the long computational dependency of single shooting, but finite consistency penalties permit deviations from the exact flow trajectory.

Across these applications, several limitations recur. Stitching cannot reliably create behavior, states, transitions, or representations absent from the data or model support. Learned dynamics, inverse dynamics, reward models, temporal embeddings, and diffusion denoisers can all introduce errors. Endpoint compatibility does not guarantee that the connection is feasible, and individually in-distribution states can form an out-of-distribution pair. Local greedy selection may miss better long-horizon combinations. More generally, empirical improvements do not imply a universal guarantee of policy improvement, global optimality, physical feasibility, or compositional generalization.

The main methodological trade-off is between **conservatism** and **expressiveness**. One-step observed-state methods reduce model-error accumulation but cannot form connections requiring multiple intermediate steps. Long-horizon generative methods can produce more flexible compositions but accumulate generation and conditioning errors. Retrieval methods preserve logged transitions but may lack novel actions. Diffusion and flow methods can generate new bridges but require careful architectural locality, conditioning, filtering, or dynamics guidance. Consequently, trajectory stitching is best understood as a family of data-centric, model-based, and compositional procedures whose common objective is to expose a learner or planner to coherent combinations that were not explicitly present as complete trajectories.

Source: https://www.emergentmind.com/topics/trajectory-stitching