Unified Trajectory Schema for Spatiotemporal Data

• Unified Trajectory Schema is a rigorously specified standard that reconciles diverse trajectory data formats by enforcing strict field naming, coordinate realignment, and semantic unification.
• It implements agent-centric spatial normalization and fixed temporal sampling to remove global coordinate biases and ensure consistent data representation across datasets.
• The schema enhances data scalability, transfer learning, and reproducible benchmarking in applications ranging from autonomous vehicle research to human mobility analytics.

A unified trajectory schema is a rigorously specified, task-agnostic data and annotation standard designed to reconcile the divergent formats, conventions, and semantic labelings across heterogeneous trajectory datasets and tasks. Such schemas enable robust scaling, evaluation, and transfer of predictive or generative models in autonomous vehicle research, multi-agent simulation, reinforcement learning world models, human mobility analytics, and broader spatiotemporal intelligence domains. Recent efforts center on formalizing minimal yet expressive field requirements, coordinate realignments, semantic unification, and interface contract guarantees, yielding datasets and APIs in which downstream models can operate without per-dataset or per-task code branches.

1. Core Schema Fields and Normalization

Recent unified trajectory schemas, such as those in UniTraj and related frameworks, prescribe strict field name conventions, fully specified meanings, and normalization procedures for every data sample. For vehicle-centric contexts, each sample is anchored to a target agent at a reference time $t=0$, collecting the following minimal fields (Feng et al., 2024):

• Agent-state representation:
  • agent.id: string, globally unique agent or scenario identifier
  • agent.type: integer class index (0=vehicle, 1=pedestrian, 2=cyclist)
  • agent.type_onehot: 3-dimensional one-hot encoding
  • agent.timestamps: $[T_\mathrm{hist} + T_\mathrm{fut}]$ array, seconds relative to reference
  • agent.pos: $[T_\mathrm{hist} + T_\mathrm{fut}] \times 2$ array, agent-centric $(x, y)$ positions
  • agent.vel, agent.acc, agent.heading: past-step velocities, accelerations, headings
• Map context:
  • map.lanes: $N_\mathrm{lanes}$ arrays of $K \times 2$ lane-center polylines
  • map.lane_dirs: $N_\mathrm{lanes} \times K \times 2$ unit direction vectors
  • map.lane_types: $N_\mathrm{lanes} \times 5$ one-hot semantic codes
  • map.resolution: inter-point spacing (default 0.5 m)
  • map.range: spatial context radius (e.g., 100 m)

Normalization procedures ensure strict agent-centricity: all coordinates and headings are rotated and translated so the origin is at the target agent’s $t=0$ pose, with $x$ aligned to heading. Map polylines are resampled such that consecutive points are exactly 0.5 m apart using linear interpolation.

Temporal alignment: All data is resampled or interpolated to a 10 Hz grid, with truncation or padding to a fixed window, typically 2 s history and 6 s future (80 total points). Missing velocities or accelerations are reconstructed by finite differences. Map and agent semantic vocabularies are collapsed and encoded as fixed-length one-hots (3 agent classes, 5 line/semantic types).

Example schema entry: $[T_\mathrm{hist} + T_\mathrm{fut}]$2 This rigid schema ensures that any dataset processed through the conversion framework will yield an identical, semantically aligned structure, guaranteeing interchangeability (Feng et al., 2024).

2. Semantic Annotation and Collapsing Taxonomies

Diverse datasets encode agent types, map elements, and behavioral semantics in incompatible categorical systems. The unified schema enforces a minimal set:

• Agent types: Collapsed to {0: vehicle, 1: pedestrian, 2: cyclist}, presented as integer and one-hot formats.
• Map line types: Collapsed to five {center_lane, lane_boundary, crosswalk, speed_bump, stop_sign}, encoded as 5-dimensional one-hots.

Source datasets with richer or more nonstandard semantic vocabularies must be mapped, merged, or dropped during conversion (Feng et al., 2024). This ensures models trained on the unified schema do not experience identifier or taxonomy mismatch when ingesting data from multiple sources.

3. Coordinate Frames, Temporal Alignment, and Imputation

Uniformity is enforced at the representational level:

• Spatial: All inputs are in local, agent-centric frames, removing global coordinate biases.
• Temporal: All data sampled at 10 Hz, with fixed-length historical and predictive windows. Datasets with non-conforming sampling rates are resampled by linear interpolation in time.
• Imputation: Short trajectories are padded using last-seen constant-velocity extrapolation or are dropped; velocities/accelerations are derived by finite differences if not originally present.

Bounding-box geometries, which may vary in presence and format, are dropped, further homogenizing the input space (Feng et al., 2024).

4. Examples of Unified Schemas Across Research Domains

Vehicle Trajectory Prediction

UniTraj’s schema formalizes requirements for scalable vehicle trajectory prediction, harmonizing input for models across nuScenes, Argoverse, Lyft, and other datasets. This harmonization exposes domain shifts that cause performance drops when models are transferred, but also enables large-scale aggregation and generalization studies. Enlarging data size and diversity in the unified schema produces state-of-the-art results on major benchmarks (Feng et al., 2024).

World Models for Heterogeneous Control

In RL world modeling, UniTraj (2502.01366) scalarizes agent trajectories from 80 distinct environments, each possibly with different sensor and actuator dimensions: $[T_\mathrm{hist} + T_\mathrm{fut}]$0 Trajectories are tabularized into matrices $[T_\mathrm{hist} + T_\mathrm{fut}]$1, then discretized and binned to enable a discrete-token Transformer world model. This enables transfer learning and in-context adaptation across morphologically divergent RL tasks.

Human Trajectory and Crowd Simulation

Human trajectory APIs such as trajdata (Ivanovic et al., 2023) express all datasets’ agent trajectories as typed Arrow tables:

• Required columns: scene_id, timestamp, track_id, x, y, z
• Derived columns: vx, vy, heading, acceleration (auto-computed if absent)

Agent and map objects are converted into unified VectorMap abstractions, with coordinate transforms and scene tags standardized for cross-dataset batching and querying.

5. Model Interface Guarantees and Interoperability

The unified schema acts as a contract: any model consuming these fields expects precisely aligned data—names, types, units, temporal and spatial frames, semantics—independent of source dataset. This enables:

• Plug-and-play batch iteration and data loading (standard JSON, arrow, numpy formats)
• Downstream model invariance, with no need for dataset-specific preprocessing branches
• Joint training/evaluation over merged, de-duplicated sources
• Reproducible benchmarking under strictly fixed windows, rates, and semantics

APIs for multi-task or simulation frameworks (e.g., PyTorch Datasets, CARLA digital twins, RL replay buffers) are built on top of these schema definitions for seamless hand-off between data, model, and evaluation environments (Feng et al., 2024, Ivanovic et al., 2023, Zheng et al., 13 Apr 2026).

6. Implications for Generalization, Transfer, and Scaling

Empirical evaluations confirm that strict adherence to a unified schema exposes cross-domain generalization gaps—models trained on one domain can drop in performance when deployed on another, even under schema alignment (Feng et al., 2024). However, schema-level unification enables scaling to larger, more diverse datasets and facilitates formal analysis of transfer barriers. Further, it is foundational for developing model architectures (e.g., universal transformers, 2D attention models, world models) able to exploit cross-domain commonalities and adapt to residual distribution shifts, as demonstrated in RL world-model pretraining scenarios (2502.01366).

Overall, the unified trajectory schema is positioned as the enabling contract for scalable, reproducible, and transferable learning across the rapidly growing corpus of spatiotemporal datasets in robotics, intelligent transportation, and multi-agent modeling. By guaranteeing field-level, temporal, spatial, and semantic alignment, it unlocks both methodological rigor and practical interoperability for the next generation of trajectory-centric research.