TrajLoc: Trajectory Similarity & Motion Control

• TrajLoc is a dual-method framework: one method embeds piecewise-linear trajectories into a Euclidean space using landmark distances, while the other employs attention localization for video diffusion.
• The landmark-based approach computes trajectory distances by mapping GPS data to fixed landmarks, ensuring metric properties under density and general-position conditions.
• The attention-based method replaces object token attention with Gaussian heatmaps to precisely control multi-object motion in image-to-video generative tasks.

TrajLoc is a name used for two distinct trajectory-centered methods. In "Simple Distances for Trajectories via Landmarks" (Phillips et al., 2018), TrajLoc denotes a landmark-based embedding for piecewise-linear trajectories in $\mathbb R^2$, with induced distances $D^{\parallel}$ and $D^\pi$ that can become bona-fide metrics under density and general-position assumptions on the landmark set. In "TrajLoc: Trajectory-Attention Localization for Multi-Object Motion Control" (Sela et al., 1 Jul 2026), TrajLoc denotes a control mechanism for text-and-image-conditioned video diffusion in which each object token’s cross-attention weights are overwritten by a Gaussian heatmap centered on the target location at every frame. The two uses are unrelated in task formulation: the former is a trajectory similarity and indexing method, whereas the latter is a multi-object motion-control method for image-to-video generation.

1. Disambiguation and scope

The term TrajLoc refers to two separate constructions documented in the literature (Phillips et al., 2018, Sela et al., 1 Jul 2026).

Usage of TrajLoc	Defining representation	Main use
"Trajectories via Landmarks"	$$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$ with induced distances $D^{\parallel}$ and $D^\pi$	trajectory similarity, k-means clustering, classification, approximate nearest-neighbor search
"Trajectory-Attention Localization"	per-object Gaussian heatmap overwrite of cross-attention, plus trajectory and appearance token embeddings	precise multi-object motion control in image-to-video diffusion

This disambiguation matters because the two methods operate on different objects, optimize different objectives, and are evaluated with different metrics. The landmark-based method maps trajectories to Euclidean vectors and then uses standard Euclidean machinery. The diffusion-based method modifies attention maps and prompt tokens inside pretrained image-to-video backbones. A plausible implication is that the shared name should not be taken to indicate architectural continuity.

2. Landmark-distance TrajLoc: formal construction

In the landmark-based formulation, $\tau$ is a piecewise-linear trajectory in $\mathbb R^2$, and $L=\{\ell_1,\ldots,\ell_m\}$ is a fixed set of $m$ landmark points. The embedding is defined by

$D^{\parallel}$0

where, in the version called $D^{\parallel}$1, the landmark coordinate is

$D^{\parallel}$2

The induced distance between two trajectories $D^{\parallel}$3 is the Euclidean distance in the embedding space,

$D^{\parallel}$4

Symmetry and the triangle inequality follow immediately from the Euclidean norm in $D^{\parallel}$5. The nontrivial issue is positive-definiteness. The stated theorem assumes that every critical vertex of $D^{\parallel}$6 lies at least distance $D^{\parallel}$7 from all other segments and that $D^{\parallel}$8 is chosen so that for every such critical point $D^{\parallel}$9 there are at least three landmarks in general position. In particular, if $D^\pi$0 is a sufficiently dense grid with step size $D^\pi$1 over a domain $D^\pi$2 containing all trajectories, then $D^\pi$3 is injective on that family and $D^\pi$4 is a metric (Phillips et al., 2018). The proof sketch uses three landmarks around each critical point whose distance-circles intersect only at that point; equality of all landmark distances then forces the second trajectory to pass through the same critical points and ultimately coincide everywhere.

A second variant, $D^\pi$5, retains closest-point coordinates rather than only scalar distances. For each landmark $D^\pi$6,

$D^\pi$7

and

$D^\pi$8

This is a pseudometric via embedding into $D^\pi$9 and becomes a metric under essentially the same density and general-position assumptions. The key structural point is that TrajLoc replaces alignment-heavy trajectory comparison with a fixed-dimensional Euclidean embedding.

3. Landmark-distance TrajLoc: algorithms and empirical profile

Landmark selection admits two stated modes. Random landmarks may be chosen uniformly or by Poisson process in $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$0. Data-driven landmarks may be selected from a small random sample of all GPS fixes or from points of interest known to be semantically important. For a trajectory with $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$1 linear segments and $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$2, each coordinate $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$3 can be computed in $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$4 time by projecting $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$5 onto each segment, so computing $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$6 takes $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$7 time and $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$8 space. Once the embeddings are precomputed, evaluating $$\phi(\tau) = (d(\tau,\ell_1), \ldots, d(\tau,\ell_m))$$9 takes $D^{\parallel}$0 time. Standard Lloyd’s algorithm can then be applied directly for k-means, with cost $D^{\parallel}$1 for $D^{\parallel}$2 trajectories, $D^{\parallel}$3 clusters, and $D^{\parallel}$4 iterations. The same embeddings can be stored in Euclidean ANN indices such as FLANN, Annoy, or KGraph, with typical build time $D^{\parallel}$5 and sub-millisecond query time even for $D^{\parallel}$6, $D^{\parallel}$7 (Phillips et al., 2018).

The reported empirical evaluation spans clustering, classification, sensitivity to landmark weighting, and ANN search. In Geolife user 155 with 42 trajectories, using $D^{\parallel}$8 random Beijing POIs and Lloyd’s algorithm with $D^{\parallel}$9 or $D^\pi$0 yielded coherent "central city" versus "northern" clusters. In driver classification on Geolife, with 128 users and up to 200 trajectories each for a total of approximately 20k trajectories, 5-NN in $D^\pi$1 yielded mean error $D^\pi$2, comparable to the best prior $D^\pi$3LCSS, DTW $D^\pi$4 and better than Euclidean-critical-point distance $D^\pi$5. When fed to a Gaussian-kernel SVM, $D^\pi$6 and $D^\pi$7 both achieved approximately $D^\pi$8 error versus approximately $D^\pi$9 for the raw Euclidean-critical-point embedding.

On the UCI "GPS Trajectories" Bus vs Car task with 123 trajectories, 5-NN in $\tau$0 with $\tau$1 gave $\tau$2 error versus approximately $\tau$3-$\tau$4 for DTW, Hausdorff, Euclidean, and related baselines, and a Gaussian-SVM on the $\tau$5-vectors reduced error to $\tau$6. A landmark-sensitivity experiment used two synthetic classes of 30 trajectories each that differed only in whether they passed near a special POI $\tau$7. A small uniform landmark set produced approximately $\tau$8-$\tau$9 error, while re-weighting $\mathbb R^2$0’s coordinate by $\mathbb R^2$1 with weights summing to $\mathbb R^2$2 reduced 5-NN error in $\mathbb R^2$3 to $\mathbb R^2$4 and weighted Gaussian-SVM error to $\mathbb R^2$5. For ANN search on the full Geolife dataset with $\mathbb R^2$6, building a KGraph index on $\mathbb R^2$7 for $\mathbb R^2$8 took $\mathbb R^2$9-$L=\{\ell_1,\ldots,\ell_m\}$0 and used approximately $L=\{\ell_1,\ldots,\ell_m\}$1 memory, while each 1-NN query took $L=\{\ell_1,\ldots,\ell_m\}$2-$L=\{\ell_1,\ldots,\ell_m\}$3 at recall $L=\{\ell_1,\ldots,\ell_m\}$4. By contrast, state-of-the-art dedicated DFT/DTW indices were reported to require minutes on hundreds of cores for a single query.

These results establish the landmark-based TrajLoc as a Euclideanization strategy: trajectory analysis is reduced to vector analysis once the landmark embedding has been computed.

4. Trajectory-attention TrajLoc: localization inside video diffusion

The 2026 TrajLoc method addresses precise multi-object motion control in text-and-image-conditioned video diffusion. Its central operation is performed inside cross-attention layers. In a standard layer, visual queries $L=\{\ell_1,\ldots,\ell_m\}$5 attend to text keys $L=\{\ell_1,\ldots,\ell_m\}$6 with

$L=\{\ell_1,\ldots,\ell_m\}$7

and the output is

$L=\{\ell_1,\ldots,\ell_m\}$8

For each object $L=\{\ell_1,\ldots,\ell_m\}$9 and frame $m$0, TrajLoc constructs a 2D Gaussian heatmap over the latent grid,

$m$1

These per-frame maps are bilinearly downsampled to the latent grid, flattened over $m$2 to a vector $m$3, and normalized so that $m$4 (Sela et al., 1 Jul 2026).

If $m$5 is the column index in $m$6 corresponding to the text token for object $m$7, the method replaces that column with $m$8 and then renormalizes every row: $m$9

$D^{\parallel}$00

The modified output is

$D^{\parallel}$01

which is used in place of the standard cross-attention output. In architectures such as WaN-2.1, where the full matrix $D^{\parallel}$02 is explicitly built, this replacement is exact. In CogVideoX-5B, the full attention matrix is too large to materialize, approximately $D^{\parallel}$03 per layer, so the method uses an efficient two-SDPA approximation that splits keys and values into supervised columns $D^{\parallel}$04 and the remaining set $D^{\parallel}$05, computes two small SDPA calls, and mixes them according to the total heatmap mass $D^{\parallel}$06. The reported practical effect is that the attention cost per layer is doubled while remaining within memory limits.

The method’s stated departure point is that existing approaches entangle multiple trajectories within a shared, dense conditioning signal, which makes object-level correspondence difficult to preserve in crowded scenes. TrajLoc instead enforces a strict, per object spatial constraint that isolates instances independently. The same per object token interface carries trajectory and depth through a learned embedding and preserves identity by encoding first frame appearance in place of an object token. This design is reported to scale to scenes with up to 20 simultaneously controlled objects and to remain effective under occlusions and path crossings.

5. Token conditioning, optimization, and reported results in diffusion TrajLoc

Trajectory and appearance are encoded as dedicated text tokens. For object $D^{\parallel}$07, the trajectory input is a sequence

$D^{\parallel}$08

where $D^{\parallel}$09 is per-frame depth and $D^{\parallel}$10 is a normalized time channel. The encoder $D^{\parallel}$11 consists of three 1D convolution layers with stride $D^{\parallel}$12 and channels $D^{\parallel}$13, followed by BN and GELU, then flattening to a $D^{\parallel}$14-dimensional vector $D^{\parallel}$15, and finally a two-layer MLP $D^{\parallel}$16 producing a $D^{\parallel}$17-dimensional trajectory token $D^{\parallel}$18. Pretraining freezes the I2V model’s text encoder and learns $D^{\parallel}$19 with

$D^{\parallel}$20

After $D^{\parallel}$21 steps and approximately $D^{\parallel}$22 parameters, the decoder is discarded and $D^{\parallel}$23 is frozen.

The appearance encoder $D^{\parallel}$24 takes the first-frame VAE latent $D^{\parallel}$25 and object center $D^{\parallel}$26. It uses one stride-2 convolution $D^{\parallel}$27 and two stride-1 convolutions $D^{\parallel}$28 with BN and GELU, producing an $D^{\parallel}$29-channel feature map at $D^{\parallel}$30. The $D^{\parallel}$31-dimensional feature sampled at $D^{\parallel}$32 is projected by a $D^{\parallel}$33 linear layer to $D^{\parallel}$34. This replaces the generic category token embedding such as "girl" or "ball" in the prompt. Prompt construction uses a template of the form: "Scene where $D^{\parallel}$35 moves [traj$D^{\parallel}$36] and $D^{\parallel}$37 moves [traj$D^{\parallel}$38] …". Each $D^{\parallel}$39 is replaced with the frozen output $D^{\parallel}$40, and each category embedding $D^{\parallel}$41 is replaced with $D^{\parallel}$42.

Fine-tuning uses LoRA on all cross-attention layers of a pretrained I2V diffusion model, specifically CogVideoX-5B or WaN 2.1-14B, with rank $D^{\parallel}$43 and $D^{\parallel}$44. Only the LoRA weights and the appearance encoder are trained; the original model weights and $D^{\parallel}$45 remain frozen. The objective combines standard diffusion noise-prediction MSE over all pixels with the same noise-MSE restricted to small bounding boxes centered on each trajectory point: $D^{\parallel}$46 Training uses AdamW and bf16. The reported schedules are $D^{\parallel}$47 steps for CogVideoX-5B with batch size $D^{\parallel}$48 H100s, approximately $D^{\parallel}$49, and $D^{\parallel}$50 steps for WaN 2.1-14B with batch size $D^{\parallel}$51 H100s, approximately $D^{\parallel}$52. Diffusion uses $D^{\parallel}$53 steps, classifier-free guidance $D^{\parallel}$54 for CogVideoX or $D^{\parallel}$55 for WaN, and $D^{\parallel}$56 px (Sela et al., 1 Jul 2026).

Evaluation covers six datasets and $D^{\parallel}$57 total clips with up to $D^{\parallel}$58 simultaneous objects, using static-camera scenes of $D^{\parallel}$59 frames at $D^{\parallel}$60. The in-distribution synthetic sets are MoVi-Extended, Pool, Football, and MOTSynth; the out-of-distribution real-world sets are MOT17 and DAVIS 2017. Metrics are PSNR, LPIPS, FVD, and EPE, where EPE is the $D^{\parallel}$61 distance between ground-truth and tracked object centers in the generated video. Across both backbones and all six datasets, TrajLoc is reported to outperform four baselines—Tora, MagicMotion, ATI, and Wan-Move—by an average gain of $D^{\parallel}$62 PSNR with range $D^{\parallel}$63-$D^{\parallel}$64 and an average $D^{\parallel}$65 reduction in EPE with range $D^{\parallel}$66-$D^{\parallel}$67. On MoVi, example numbers are: for CogVideoX, $D^{\parallel}$68 versus $D^{\parallel}$69-$D^{\parallel}$70 and EPE $D^{\parallel}$71 versus $D^{\parallel}$72-$D^{\parallel}$73; for WaN, $D^{\parallel}$74 versus $D^{\parallel}$75-$D^{\parallel}$76 and EPE $D^{\parallel}$77 versus $D^{\parallel}$78-$D^{\parallel}$79.

Ablations identify attention localization as the most critical component. Removing attention localization on WaN-14B causes PSNR to drop by $D^{\parallel}$80 on MoVi and EPE to increase by a factor of $D^{\parallel}$81. Removing trajectory tokens reduces PSNR by $D^{\parallel}$82 and increases EPE by up to $D^{\parallel}$83. Removing the depth channel yields a similar drop in depth-sensitive scenes, and removing the appearance encoder causes moderate FVD degradation. For CogVideoX, the self-attention approximation ablation reports that the 2-SDPA replacement is stronger on crowded datasets such as MOTSynth and MOT17, whereas the additive correction variant is slightly stronger on far-out-of-distribution DAVIS. Qualitatively, the reported failure modes of baselines are missing objects under occlusion, drifting over time, hallucinating duplicates, and failing in crowded crossings.

6. Relation to adjacent trajectory methods

TrajLoc should be distinguished from trajectory methods that solve next-location prediction rather than similarity search or generative motion control. TTDM, the "Travel Time Difference Model," defines a user trajectory as

$D^{\parallel}$84

uses the actual travel time from all passed locations to each candidate next location, and compares it with a shortest-path travel time on a time-slot-indexed road-network graph. For a candidate location $D^{\parallel}$85, the model aggregates

$D^{\parallel}$86

and forms the average excess

$D^{\parallel}$87

A decreasing function such as $D^{\parallel}$88 or $D^{\parallel}$89 converts this to

$D^{\parallel}$90

which can be linearly interpolated with a first-order Markov model,

$D^{\parallel}$91

The reported top-1 accuracy results are $D^{\parallel}$92, $D^{\parallel}$93, and $D^{\parallel}$94 for MM, TTDM, and TTDM+MM on VPR, and $D^{\parallel}$95, $D^{\parallel}$96, and $D^{\parallel}$97 on Taxi, with best interpolation weight $D^{\parallel}$98-$D^{\parallel}$99 (Liu et al., 2020).

The conceptual distinction is straightforward. Landmark-based TrajLoc embeds a completed trajectory into $D^\pi$00 and studies metric geometry, Euclidean learning, and ANN search. Diffusion-based TrajLoc injects trajectory and appearance into cross-attention and prompt tokens to control future video frames. TTDM predicts a next location from a partial trajectory using travel-time differences and Markov interpolation. A common source of confusion is therefore terminological rather than methodological: the three methods all process trajectories, but they belong to different task families, use different representations, and are evaluated with different criteria.