X-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Prediction

Abstract: Recent advancements in Recurrent Neural Network (RNN) architectures, particularly the Extended Long Short Term Memory (xLSTM), have addressed the limitations of traditional Long Short Term Memory (LSTM) networks by introducing exponential gating and enhanced memory structures. These improvements make xLSTM suitable for time-series prediction tasks as they exhibit the ability to model long-term temporal dependencies better than LSTMs. Despite their potential, these xLSTM-based models remain largely unexplored in the context of vehicle trajectory prediction. Therefore, this paper introduces a novel xLSTM-based vehicle trajectory prediction framework, X-TRAJ, and its physics-aware variant, X-TRACK (eXtended LSTM for TRAjectory prediction Constraint by Kinematics), which explicitly integrates vehicle motion kinematics into the model learning process. By introducing physical constraints, the proposed model generates realistic and feasible trajectories. A comprehensive evaluation on the highD and NGSIM datasets demonstrates that X-TRACK outperforms state-of-the-art baselines.

• The paper demonstrates that integrating a physics-based kinematic layer into xLSTM significantly enhances vehicle trajectory predictions by enforcing non-holonomic constraints.
• It leverages an encoder-decoder architecture, combining sLSTM, LSTM, and graph attention modules to model temporal dynamics and inter-vehicle interactions.
• Experimental results indicate notable improvements, with up to 79% RMSE improvement at short horizons and significant reductions in displacement errors on high-quality datasets.

Physics-Aware xLSTM for Realistic Vehicle Trajectory Prediction: The X-TRACK Framework

Introduction

The paper presents X-TRACK, a physics-aware trajectory prediction framework for highway vehicles, leveraging the extended Long Short Term Memory (xLSTM) architecture. The approach addresses the limitations of conventional LSTM-based models in capturing long-term temporal dependencies and ensuring physical feasibility of predicted trajectories. By integrating a kinematic bicycle model as a physics-based layer, X-TRACK enforces non-holonomic constraints, resulting in predictions that are both accurate and physically plausible. The framework is evaluated on the highD and NGSIM datasets, demonstrating superior performance over state-of-the-art baselines in terms of displacement and root mean square errors.

Model Architecture

X-TRACK employs an encoder-decoder architecture with the following key components:

• sLSTM Encoder: Encodes the temporal evolution of each vehicle's historical trajectory, utilizing exponential gating and memory mixing for enhanced representational capacity.
• Graph Attention Network (GAT) Module: Models social interactions among vehicles by constructing a graph where nodes represent vehicles and edges encode attention-based interactions.
• LSTM Decoder: Predicts future motion parameters using the concatenated target vehicle encoding and interaction context.
• Physics-Based Kinematic Layer: Transforms predicted motion parameters (longitudinal acceleration and yaw rate) into position coordinates, enforcing physical constraints on vehicle motion.

The overall architecture is depicted in the following figure:

Figure 1: The X-TRACK architecture integrates sLSTM encoding, GAT-based interaction modeling, and a kinematic layer for physically consistent trajectory prediction.

Problem Formulation and Physics Integration

The trajectory prediction task is formulated as a sequence-to-sequence problem, where the model receives historical states of the target and neighboring vehicles and outputs future positions. The input features for the physics-aware variant are longitudinal acceleration and yaw rate, rather than direct position coordinates. The kinematic layer applies the following update equations for position, velocity, and heading:

$$\begin{align} x^{t+\Delta t} &= x^t + v^t \cos(\psi^t)\,\Delta t + \left(a_x^t \cos(\psi^t) - \dot{\psi}^t v^t \sin(\psi^t)\right)\frac{\Delta t^2}{2} \ y^{t+\Delta t} &= y^t + v^t \sin(\psi^t)\,\Delta t + \left(a_x^t \sin(\psi^t) + \dot{\psi}^t v^t \cos(\psi^t)\right)\frac{\Delta t^2}{2} \ v^{t+\Delta t} &= v^t + a_x^t\,\Delta t \ \psi^{t+\Delta t} &= \psi^t + \dot{\psi}^t\,\Delta t \end{align}$$

Physical limits are imposed on $a_x$ and $\dot{\psi}$ to ensure feasibility. This explicit integration of vehicle dynamics distinguishes X-TRACK from purely data-driven models, which may produce statistically plausible but physically infeasible trajectories.

Experimental Setup

Datasets

• highD: Drone-captured highway trajectories in Germany, with over 110,000 vehicles and 147 hours of recordings.
• NGSIM: US highway trajectories, annotated at 10 Hz, with significant scenario imbalance addressed via preprocessing.

Balanced subsets are created for both datasets to avoid bias toward dominant lane-keeping scenarios.

Training Details

• Implemented in PyTorch and PyTorch Geometric.
• sLSTM encoder (64-dim), LSTM decoder (128-dim), GAT with four-head attention.
• LeakyReLU activations, batch size 32, Adam optimizer.

Evaluation Metrics

• Average Displacement Error (ADE)
• Final Displacement Error (FDE)
• Root Mean Square Error (RMSE) over prediction horizons

Results

X-TRACK achieves the lowest ADE and FDE on the highD dataset, outperforming all baselines. On NGSIM, X-TRAJ (without the kinematic layer) slightly outperforms X-TRACK, likely due to annotation inaccuracies and reduced scenario count in the balanced subset.

• On highD, X-TRACK yields a 51% improvement in ADE and 34% in FDE over X-TRAJ.
• RMSE improvements are 79% at 1s and 32% at 5s prediction horizons.
• Compared to GFTNNv2, X-TRACK shows 78.7% and 19.7% improvement at 1s and 5s, respectively.

Predicted trajectories demonstrate that X-TRACK aligns more closely with ground truth, especially in lane change scenarios:

Figure 2: Predicted trajectories on highD for keep lane and lane change scenarios, showing X-TRACK's superior alignment with ground truth compared to X-TRAJ.

Ablation studies reveal that the combination of sLSTM encoder and LSTM decoder yields the best performance, with up to 10.77% improvement in ADE and 6.5% in FDE over conventional LSTM encoder-decoder architectures.

Discussion

The integration of physics-based priors with xLSTM sequence modeling bridges the gap between predictive accuracy and physical consistency. The kinematic layer enforces non-holonomic constraints, preventing implausible trajectory drift and ensuring that predictions are executable by real vehicles. The pronounced performance gain on highD highlights the importance of high-quality, well-annotated data for physics-aware models. In contrast, the reduced scenario count and annotation noise in NGSIM limit the statistical significance of results.

The framework's modularity allows for further extension, such as incorporating richer map and road structure information, or adapting the architecture to urban environments with more complex interactions.

Conclusion

X-TRACK demonstrates that physics-aware sequence modeling with xLSTM and kinematic constraints yields state-of-the-art performance in highway vehicle trajectory prediction. The approach achieves substantial improvements in displacement and RMSE metrics, particularly on high-quality datasets. The results underscore the necessity of combining data-driven learning with explicit physical modeling for safety-critical applications in autonomous driving. Future work may explore integration with additional contextual information and evaluation on diverse traffic scenarios to further enhance generalization and robustness.