---
title: 'X-TRAJ: Autonomous Trajectory Prediction'
url: https://www.emergentmind.com/topics/x-traj
type: topic
---

# X-TRAJ: Autonomous Trajectory Prediction

X-TRAJ is a trajectory prediction framework for autonomous driving that leverages the extended Long Short Term Memory (xLSTM) architecture to model long-range temporal dependencies in multi-agent traffic environments. Its physics-aware variant, X-TRACK, incorporates vehicle kinematic constraints into the learning process, enabling generation of physically feasible and realistic trajectories. Empirical evaluations on the highD and NGSIM datasets demonstrate that X-TRACK achieves state-of-the-art accuracy on standard prediction metrics while maintaining non-holonomic consistency for vehicle motion [2511.00266].

## 1. Theoretical Foundation: The xLSTM Architecture

Traditional LSTM networks are limited by a scalar cell state, challenges with memory revision, and sequential computation bottlenecks. The xLSTM family, including the single-head sLSTM and multi-head mLSTM, overcomes these issues via exponential gating—which enables more flexible (open/close) gate operation—and introduces a “normalizer” state for improved stabilization, along with multi-headed memory mixing for richer memory representations.

The sLSTM, deployed as the primary encoder in X-TRAJ, computes the following for timestep $t$ given input $x_t \in \mathbb{R}^{d_{in}}$ and previous hidden state $h_{t-1} \in \mathbb{R}^d$:

\[
\begin{aligned}
z_{t} &= \phi(W_z^T x_t + R_z h_{t-1} + b_z)\\
i_{t} &= \exp(W_i^T x_t + R_i h_{t-1} + b_i)\\
f_{t} &= \sigma(W_f^T x_t + R_f h_{t-1} + b_f)\\
o_{t} &= \sigma(W_o^T x_t + R_o h_{t-1} + b_o)\\
c_{t} &= f_{t} \odot c_{t-1} + i_{t} \odot z_{t}\\
n_{t} &= f_{t} \odot n_{t-1} + i_{t}\\
h_{t} &= o_{t} \odot(c_{t} \odot n_{t}^{-1})
\end{aligned}
\]

Here, $c_t$ is the cell state, $n_t$ is the normalizer, and $h_t$ is the output. $\phi(\cdot)$ denotes the activation function (typically tanh), and $\sigma(\cdot)$ is the sigmoid.

The mLSTM variant generalizes to matrix-valued memories with a covariance-style update, but empirical results on X-TRAJ favored the sLSTM encoder.

## 2. X-TRAJ and Physics-Aware X-TRACK: Model Structure

X-TRAJ processes each vehicle’s temporal feature sequence via a single-layer sLSTM encoder, where each embedded feature vector $e_i^t$ (per vehicle per timestep) yields hidden states $h_i^t$ that encode 3-second-long observation windows. Interaction modeling is performed by a two-layer multi-head Graph Attention Network (GAT) with four heads, reflecting influences from up to eight neighbors (preceding, following, and adjacent lanes).

In the base X-TRAJ model, the decoder is a standard LSTM equipped to output future positions.

X-TRACK, the physics-aware extension, replaces the position decoder with a motion-parameter decoder that predicts per-timestep longitudinal acceleration $a_x^t$ and yaw-rate $\dot{\psi}^t$. A differentiable kinematic bicycle model integrates these controls to generate full trajectories $(x, y, v, \psi)$, ensuring non-holonomic and physical realism.

## 3. Kinematic Layer and Learning Integration

The X-TRACK model’s core novelty is its differentiable kinematic integration layer. At each step, the decoder yields control outputs $\hat{u}^t = [a_x^t; \dot{\psi}^t]$; these are passed through the kinematic equations:

\[
\begin{aligned}
x^{t+\Delta t} &= x^t + v^t \cos \psi^t \Delta t + \frac{1}{2} \left[a_x^t \cos\psi^t - \dot{\psi}^t v^t \sin\psi^t\right] (\Delta t)^2 \\
y^{t+\Delta t} &= y^t + v^t \sin \psi^t \Delta t + \frac{1}{2} \left[a_x^t \sin\psi^t + \dot{\psi}^t v^t \cos\psi^t\right] (\Delta t)^2 \\
v^{t+\Delta t} &= v^t + a_x^t \Delta t \\
\psi^{t+\Delta t} &= \psi^t + \dot{\psi}^t \Delta t
\end{aligned}
\]

Predicted $a_x^t$ and $\dot{\psi}^t$ are clamped with $|a_x^t| \leq 9 \,\text{m/s}^2$ and $|\dot{\psi}^t| \leq 1.243\,\text{rad/s}$ to respect physical bounds.

Backpropagation through the kinematic layer ensures that the network learns to output controls resulting in feasible and accurate trajectories.

## 4. Loss Functions and Training Regimen

Trajectory prediction is supervised via a position-based MSE loss over all predicted timesteps (Eq. 12):

\[
L_{\text{traj}} = \frac{1}{N} \sum_{n=1}^N \sum_{t=t_{\text{obs}}+1}^{t_{\text{obs}}+t_f} \| \hat{y}_n^t - y_n^t \|^2_2
\]

For X-TRACK, an auxiliary control-parameter loss can be used:

\[
L_{\text{ctrl}} = \frac{1}{N} \sum_{n=1}^N \sum_{t=t_{\text{obs}}+1}^{t_{\text{obs}}+t_f} \| \hat{u}_n^t - u_n^t \|^2_2
\]

Optional constraint penalty via ReLU ensures soft enforcement of physical limits.

\[
L_{\text{bound}} = \sum_{n, t} \left[ \text{ReLU}(|a_{x,n}^t| - a_{x,\text{max}})^2 + \text{ReLU}(|\dot{\psi}_n^t| - \dot{\psi}_{\text{max}})^2 \right]
\]

The combined loss is:

\[
L_{\text{total}} = L_{\text{traj}} + \lambda_{\text{ctrl}} L_{\text{ctrl}} + \lambda_{\text{bound}} L_{\text{bound}}
\]

In practice, pure positional MSE was used with hard-clamped controls.

Training utilizes batch size 32, Adam optimizer (learning rate 1e–3), and proceeds until validation ADE convergence. Ghost vehicles are inserted for missing neighbors.

## 5. Experimental Setup and Evaluation

### Datasets and Features
- **highD** (Germany): 25 Hz, 110,000 vehicles, 6 locations; 13,722 scenarios (train/val/test split: 9604/1371/2747).  
- **NGSIM** (US): 10 Hz, 45 minutes; 2,337 scenarios (1634/232/471).

Each scenario uses a 3-s observed history to predict a 5-s future.

- Input features for X-TRAJ: $(x, y, v, a_x)$ per timestep per vehicle.
- Input for X-TRACK: $(a_x, \dot{\psi})$.
- Up to 8 neighbors are considered.

### Architecture
- Embedding: Fully connected layer + LeakyReLU (32-D).
- Encoder: single-layer sLSTM, hidden size 64.
- Interaction: two-layer multi-head GAT (4 heads).
- Decoder: standard LSTM, hidden size 128, with FC+LeakyReLU, producing controls or positions.
- Physics-aware kinematic layer (X-TRACK only).

### Evaluation Metrics
- **ADE** (Average Displacement Error): mean Euclidean distance over prediction horizon.
- **FDE** (Final Displacement Error): Euclidean distance at $t=5$ s.
- **RMSE(t)**: time-dependent root-mean-squared error.

## 6. Quantitative Results and Comparative Analysis

### highD Dataset (5 s horizon)

| Architecture      | ADE [m] | FDE [m] |
|-------------------|---------|---------|
| X-TRACK (ours)    | 0.56    | 1.76    |
| X-TRAJ (ours)     | 1.14    | 2.65    |
| GFTNNv2           | 0.92    | 2.20    |
| GSTAN             | 1.28    | 2.66    |
| CS-LSTM           | 2.14    | 5.09    |

- X-TRACK achieves a 51% reduction in ADE and 34% in FDE compared to the base X-TRAJ.
- Against GFTNNv2, X-TRACK leads by 39% in ADE and 20% in FDE.

### NGSIM Dataset (5 s horizon)

| Architecture      | ADE [m] | FDE [m] |
|-------------------|---------|---------|
| GSTAN             | 1.94    | 4.50    |
| X-TRAJ (ours)     | 1.99    | 4.99    |
| X-TRACK (ours)    | 2.11    | 5.17    |

- X-TRACK ranks among the top methods, with GSTAN attaining the best scores by a narrow margin.

### RMSE over Time (highD)

| Horizon [s] | 1    | 2    | 3    | 4    | 5    |
|-------------|------|------|------|------|------|
| X-TRACK     | 0.10 | 0.31 | 0.71 | 1.31 | 2.16 |
| GFTNNv2     | 0.47 | 0.61 | 1.05 | 1.75 | 2.69 |

X-TRACK demonstrates a 79% reduction in 1 s RMSE and a 20% improvement at 5 s relative to GFTNNv2.

## 7. Conclusion and Significance

X-TRAJ harnesses xLSTM’s advanced temporal modeling capabilities for trajectory forecasting, capturing extended traffic dependencies crucial for robust autonomous navigation. The integration of a differentiable kinematic layer in X-TRACK guarantees trajectories adhere to vehicle non-holonomic constraints, offering physical plausibility alongside competitive predictive accuracy. Empirical results on highD and NGSIM datasets establish X-TRACK’s state-of-the-art performance in ADE, FDE, and RMSE, supporting its applicability to safety-critical autonomous driving scenarios that demand both accuracy and physical realism [2511.00266].

Source: https://www.emergentmind.com/topics/x-traj