X-TRAJ: Autonomous Trajectory Prediction

• X-TRAJ is a trajectory prediction framework that employs an extended LSTM architecture to capture long-range temporal dependencies in multi-agent traffic environments.
• X-TRACK, its physics-aware extension, integrates vehicle kinematic constraints via a differentiable kinematic layer to generate physically feasible trajectories.
• Empirical evaluations on highD and NGSIM demonstrate state-of-the-art performance with reduced displacement errors and improved non-holonomic consistency.

X-TRAJ is a trajectory prediction framework for autonomous driving that leverages the extended Long Short Term Memory (xLSTM) architecture to model long-range temporal dependencies in multi-agent traffic environments. Its physics-aware variant, X-TRACK, incorporates vehicle kinematic constraints into the learning process, enabling generation of physically feasible and realistic trajectories. Empirical evaluations on the highD and NGSIM datasets demonstrate that X-TRACK achieves state-of-the-art accuracy on standard prediction metrics while maintaining non-holonomic consistency for vehicle motion (Chugh et al., 31 Oct 2025).

1. Theoretical Foundation: The xLSTM Architecture

Traditional LSTM networks are limited by a scalar cell state, challenges with memory revision, and sequential computation bottlenecks. The xLSTM family, including the single-head sLSTM and multi-head mLSTM, overcomes these issues via exponential gating—which enables more flexible (open/close) gate operation—and introduces a “normalizer” state for improved stabilization, along with multi-headed memory mixing for richer memory representations.

The sLSTM, deployed as the primary encoder in X-TRAJ, computes the following for timestep $t$ given input $x_t \in \mathbb{R}^{d_{in}}$ and previous hidden state $h_{t-1} \in \mathbb{R}^d$:

$$\begin{aligned} z_{t} &= \phi(W_z^T x_t + R_z h_{t-1} + b_z)\ i_{t} &= \exp(W_i^T x_t + R_i h_{t-1} + b_i)\ f_{t} &= \sigma(W_f^T x_t + R_f h_{t-1} + b_f)\ o_{t} &= \sigma(W_o^T x_t + R_o h_{t-1} + b_o)\ c_{t} &= f_{t} \odot c_{t-1} + i_{t} \odot z_{t}\ n_{t} &= f_{t} \odot n_{t-1} + i_{t}\ h_{t} &= o_{t} \odot(c_{t} \odot n_{t}^{-1}) \end{aligned}$$

Here, $c_t$ is the cell state, $n_t$ is the normalizer, and $h_t$ is the output. $\phi(\cdot)$ denotes the activation function (typically tanh), and $\sigma(\cdot)$ is the sigmoid.

The mLSTM variant generalizes to matrix-valued memories with a covariance-style update, but empirical results on X-TRAJ favored the sLSTM encoder.

2. X-TRAJ and Physics-Aware X-TRACK: Model Structure

X-TRAJ processes each vehicle’s temporal feature sequence via a single-layer sLSTM encoder, where each embedded feature vector $e_i^t$ (per vehicle per timestep) yields hidden states $x_t \in \mathbb{R}^{d_{in}}$0 that encode 3-second-long observation windows. Interaction modeling is performed by a two-layer multi-head Graph Attention Network (GAT) with four heads, reflecting influences from up to eight neighbors (preceding, following, and adjacent lanes).

In the base X-TRAJ model, the decoder is a standard LSTM equipped to output future positions.

X-TRACK, the physics-aware extension, replaces the position decoder with a motion-parameter decoder that predicts per-timestep longitudinal acceleration $x_t \in \mathbb{R}^{d_{in}}$1 and yaw-rate $x_t \in \mathbb{R}^{d_{in}}$2. A differentiable kinematic bicycle model integrates these controls to generate full trajectories $x_t \in \mathbb{R}^{d_{in}}$3, ensuring non-holonomic and physical realism.

3. Kinematic Layer and Learning Integration

The X-TRACK model’s core novelty is its differentiable kinematic integration layer. At each step, the decoder yields control outputs $x_t \in \mathbb{R}^{d_{in}}$4; these are passed through the kinematic equations:

$x_t \in \mathbb{R}^{d_{in}}$5

Predicted $x_t \in \mathbb{R}^{d_{in}}$6 and $x_t \in \mathbb{R}^{d_{in}}$7 are clamped with $x_t \in \mathbb{R}^{d_{in}}$8 and $x_t \in \mathbb{R}^{d_{in}}$9 to respect physical bounds.

Backpropagation through the kinematic layer ensures that the network learns to output controls resulting in feasible and accurate trajectories.

4. Loss Functions and Training Regimen

Trajectory prediction is supervised via a position-based MSE loss over all predicted timesteps (Eq. 12):

$h_{t-1} \in \mathbb{R}^d$0

For X-TRACK, an auxiliary control-parameter loss can be used:

$h_{t-1} \in \mathbb{R}^d$1

Optional constraint penalty via ReLU ensures soft enforcement of physical limits.

$h_{t-1} \in \mathbb{R}^d$2

The combined loss is:

$h_{t-1} \in \mathbb{R}^d$3

In practice, pure positional MSE was used with hard-clamped controls.

Training utilizes batch size 32, Adam optimizer (learning rate 1e–3), and proceeds until validation ADE convergence. Ghost vehicles are inserted for missing neighbors.

5. Experimental Setup and Evaluation

Datasets and Features

• highD (Germany): 25 Hz, 110,000 vehicles, 6 locations; 13,722 scenarios (train/val/test split: 9604/1371/2747).
• NGSIM (US): 10 Hz, 45 minutes; 2,337 scenarios (1634/232/471).

Each scenario uses a 3-s observed history to predict a 5-s future.

• Input features for X-TRAJ: $h_{t-1} \in \mathbb{R}^d$4 per timestep per vehicle.
• Input for X-TRACK: $h_{t-1} \in \mathbb{R}^d$5.
• Up to 8 neighbors are considered.

Architecture

• Embedding: Fully connected layer + LeakyReLU (32-D).
• Encoder: single-layer sLSTM, hidden size 64.
• Interaction: two-layer multi-head GAT (4 heads).
• Decoder: standard LSTM, hidden size 128, with FC+LeakyReLU, producing controls or positions.
• Physics-aware kinematic layer (X-TRACK only).

Evaluation Metrics

• ADE (Average Displacement Error): mean Euclidean distance over prediction horizon.
• FDE (Final Displacement Error): Euclidean distance at $h_{t-1} \in \mathbb{R}^d$6 s.
• RMSE(t): time-dependent root-mean-squared error.

6. Quantitative Results and Comparative Analysis

highD Dataset (5 s horizon)

Architecture	ADE [m]	FDE [m]
X-TRACK (ours)	0.56	1.76
X-TRAJ (ours)	1.14	2.65
GFTNNv2	0.92	2.20
GSTAN	1.28	2.66
CS-LSTM	2.14	5.09

• X-TRACK achieves a 51% reduction in ADE and 34% in FDE compared to the base X-TRAJ.
• Against GFTNNv2, X-TRACK leads by 39% in ADE and 20% in FDE.

NGSIM Dataset (5 s horizon)

Architecture	ADE [m]	FDE [m]
GSTAN	1.94	4.50
X-TRAJ (ours)	1.99	4.99
X-TRACK (ours)	2.11	5.17

• X-TRACK ranks among the top methods, with GSTAN attaining the best scores by a narrow margin.

RMSE over Time (highD)

Horizon [s]	1	2	3	4	5
X-TRACK	0.10	0.31	0.71	1.31	2.16
GFTNNv2	0.47	0.61	1.05	1.75	2.69

X-TRACK demonstrates a 79% reduction in 1 s RMSE and a 20% improvement at 5 s relative to GFTNNv2.

7. Conclusion and Significance

X-TRAJ harnesses xLSTM’s advanced temporal modeling capabilities for trajectory forecasting, capturing extended traffic dependencies crucial for robust autonomous navigation. The integration of a differentiable kinematic layer in X-TRACK guarantees trajectories adhere to vehicle non-holonomic constraints, offering physical plausibility alongside competitive predictive accuracy. Empirical results on highD and NGSIM datasets establish X-TRACK’s state-of-the-art performance in ADE, FDE, and RMSE, supporting its applicability to safety-critical autonomous driving scenarios that demand both accuracy and physical realism (Chugh et al., 31 Oct 2025).