Future-aware Trajectory Generator (FaTG)

• Future-aware Trajectory Generation is a method that incorporates anticipated future scene evolution into trajectory planning, leveraging multi-modal sensors and transformer-based encoding.
• It employs bidirectional reasoning, what-if scene rollouts, and token injections to generate context-rich trajectories in dynamic, multi-agent environments.
• Iterative fusion of current and predicted information leads to enhanced safety metrics and improved planning accuracy in benchmark evaluations.

A Future-aware Trajectory Generator (FaTG) is an architectural module designed to explicitly incorporate anticipated future scene evolution into the trajectory planning pipeline for autonomous agents, particularly in end-to-end autonomous driving. Rather than treating trajectory generation as a one-shot mapping from current perception, FaTG architectures jointly reason about future world dynamics—either by bidirectionally modeling the evolution of the surrounding scene and agent (SeerDrive), encoding “what-if” scene rollouts under potential actions (MindDrive), or injecting future-mode tokens into spatiotemporal interaction graphs (FINet). This enables the planner to incorporate longer-horizon anticipatory context, increasing safety, feasibility, and robustness in complex multi-agent environments (Zhang et al., 13 Oct 2025, Suna et al., 4 Dec 2025, Li et al., 9 Mar 2025).

1. Underlying Principles of Future-aware Trajectory Generation

FaTG designs emerge from the observation that the optimal trajectory for an ego vehicle is bidirectionally entangled with future environment states: the agent’s own actions shape the scene, and an accurate forecast of the scene enhances the agent’s planning (Zhang et al., 13 Oct 2025). Classical pipelines operate under a static, context-only paradigm, mapping current sensor data directly to immediate plans, and thus tend to underappreciate the effects of longer-term scene evolution, especially when dealing with dynamic elements or rare events.

The fundamental goal of FaTG is to maximize not only the accuracy of the planned motion under the current context but also its feasibility and safety with respect to plausible future scene configurations, often leveraging explicit world modeling within the planning loop (Suna et al., 4 Dec 2025).

2. Architectural Realizations

a) SeerDrive/Closed-loop BEV World Models

The SeerDrive framework features a FaTG module that alternates between two main computational flows: (1) future BEV feature prediction and (2) trajectory refinement. Multi-modal sensor data (cameras, LiDAR) are encoded into BEV feature maps using a TransFuser-style backbone. Anchored candidate trajectories are encoded together with the ego state, producing a set of ego-centric mode features (Zhang et al., 13 Oct 2025).

These features, concatenated with spatial BEV tokens, pass through a Transformer encoder (BEVWorldModel) predicting scene evolution and yielding future BEV representations. The FaTG utilizes two Transformer decoders: one fusing current context, the other fusing predicted future context into trajectory mode features. A Motion-aware LayerNorm then merges these embeddings for final trajectory decoding. The loop is iteratively executed (optimal N=2) with the latest planned trajectory embedding reinjected for recalibrated future scene prediction.

b) MindDrive/Ego-conditioned What-if World Modeling

MindDrive’s FaTG leverages a World Action Model (WAM) conditioned on discrete anchors reflecting different high-level intentions. Anchor-level intent tokens are bilinearly injected into local BEV feature maps, generating N alternative scene-variant tensors. These are stacked and processed by a spatial Transformer encoder, followed by a temporal rollout via cascaded state-space (Mamba) blocks, and finally spatial decoding back to BEV (Suna et al., 4 Dec 2025).

Candidate trajectories are then generated by cross-attending intent tokens over both current and future BEV tensors, with offsets decoded and applied to the initial anchors, producing a diverse, future-aware set of trajectory proposals.

c) FINet/Token-based Multi-modal Scene Encoding

The Future-Aware Interaction Network (FINet) implements a FaTG for agent prediction domains by introducing explicit future-mode tokens alongside agent and map tokens. Spatial-temporal dependencies are modeled via state-space models (Mamba) with adaptive reordering to respect spatial context relevance. Temporal trajectory refinement further smooths each candidate’s prediction over time, with intermediate supervision at each stage (Li et al., 9 Mar 2025).

3. Mathematical Formalism

At the core of all FaTG variants is the explicit fusion of future-aware scene encoding with candidate trajectory generation. The following paradigm is common:

• Scene-variant construction: For each candidate action/trajectory mode $n$, compute an intent token $a^{(n)}_{token}$, inject into BEV features via bilinear interpolation, forming a scene-variant tensor $s^{(n)}_f$ (Suna et al., 4 Dec 2025).
• World model rollout: Evolve the scene features under ego actions. For SeerDrive, this involves Transformer-based $F_{\mathrm{scene}}$ encodings and iterative future BEV prediction; for MindDrive, a Transformer–Mamba–Transformer architecture evolves scene variants over time steps.
• Trajectory fusion and decoding: Fuse current and predicted future BEV (or other contextually-rich) features with trajectory mode tokens, typically through cross-attention or parallel decoding, before turning into final waypoints. Motion-aware normalization or spatio-temporal smoothing may further regularize the final prediction (Zhang et al., 13 Oct 2025, Suna et al., 4 Dec 2025, Li et al., 9 Mar 2025).

The overall loss combines map reconstruction (BEV semantics, supervised via cross-entropy/focal or L2 losses at every predicted step) with trajectory regression (L2 or smooth $L_1$ displacement plus orientation, often with winner-takes-all selection) and alignment/auxiliary losses to guide context sorting and intermediate reasoning (Zhang et al., 13 Oct 2025, Li et al., 9 Mar 2025).

4. Implementation Details and Network Components

The principal FaTG components span perception, world modeling, and trajectory planning:

Module	Implementation	Key Hyperparameters/Features
BEV Encoder	TransFuser (ResNet-34/50)	C=256, H×W=8×8 or 100×100→8×8
Ego/Intent Encoder	MLP + Anchor Clustering	M=256 (NAVSIM), M=6 (nuScenes/NAVSIM-v2)
World Model	Transformer, Mamba (SSM)	6 layers, 8 heads, feedforward=1024
Planning Decoder	Transformer decoder/Mamba	MLN fusion, cross-attention
Iteration	N=2 (SeerDrive/MindDrive)
Optimization	AdamW/Adam; LR 1–2e-4	Batch 16/GPU, epochs 12–30, ≈66M params

Evaluation and ablation protocols regularly utilize NAVSIM (PDM Score/PDMS) and nuScenes (open-loop L2, collision %) for benchmarking (Zhang et al., 13 Oct 2025, Suna et al., 4 Dec 2025). Scene-variant anchoring, hybrid backbone use, linearized SSM, and iterative supervision are central to top performance.

5. Comparative Evaluation and Ablation Studies

FaTG inclusion yields substantial gains over prior state-of-the-art, with effects detailed in the following performance table:

Model Variant	NAVSIM PDMS ↑	nuScenes L2 ↓	Col. Rate ↓
DiffusionDrive (prior)	88.1	—	—
SeerDrive FaTG	88.9	0.43	0.06%
SeerDrive† (V2-99 backbone)	90.7	—	—
SparseDrive (prior, nuScenes)	—	0.61	0.08%
MindDrive (with full FaTG/VLoE)	88.9	—	—

Ablation studies highlight that disabling future BEV injection or bidirectional iteration both degrade results (e.g., PDMS drop from 88.9 to 87.9 and 88.1, respectively (Zhang et al., 13 Oct 2025)), while the MindDrive WAM’s hybrid Transformer–Mamba–Transformer “sandwich” configuration outperforms either alone (Suna et al., 4 Dec 2025). Two-step temporal rollout surpasses a single-step (88.9 vs 87.5 PDMS).

A plausible implication is that anticipatory context and iterative optimization are necessary for reliable long-horizon planning under dynamic uncertainty.

6. Training Objectives, Losses, and Supervision

FaTG systems universally employ compound supervision:

• BEV map loss: $\mathcal{L}_{\mathrm{map}}$, often a weighted sum of current and future step semantic cross-entropy/focal or L2.
• Trajectory loss: $\mathcal{L}_{\mathrm{traj}}$, sum over all modes and iterations, winner-takes-all on best mode, combining L2 (or smooth-$L_1$) with heading error.
• Auxiliary losses: Alignment for sorting reference points (FINet), intermediate decoding losses, and multi-stage supervision.
• Total objective: For SeerDrive, $$\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{map}} + \mathcal{L}_{\mathrm{traj}}$$; for MindDrive, L2 on map and L1/L2 on trajectories (Zhang et al., 13 Oct 2025, Suna et al., 4 Dec 2025, Li et al., 9 Mar 2025).

Weighting for NAVSIM is $\lambda_1=10$, $\lambda_2=0.1$, $\lambda_3=1$; for nuScenes, all $\lambda=1$ (Zhang et al., 13 Oct 2025).

7. Impact and Prospects in Autonomous Planning

FaTG establishes a structured, future-centric approach to trajectory generation, demonstrably increasing the reliability and safety of autonomous driving systems under uncertainty. Explicit world modeling, intent-conditioned simulation, and iterative feedback correction set the FaTG apart from classical static or one-shot planners.

Empirically, FaTG alone yields a 2.5-point PDMS gain in closed-loop driving on NAVSIM (from 84.1 to 86.6), rising to 88.9 with integration into the full MindDrive framework (Suna et al., 4 Dec 2025). SeerDrive FaTG achieves state-of-the-art on both NAVSIM and nuScenes, decreasing open-loop L2 and collision error rates (Zhang et al., 13 Oct 2025). This suggests that the principal future extension will be in integrating even richer, multi-agent world models and multi-objective evaluation, as signaled by the MindDrive VLoE integration (Suna et al., 4 Dec 2025).

Future research will likely focus on scaling FaTG to more complex, closed-loop real-world tests, further reducing inference latency, and harmonizing anticipatory planning with vision-language reasoning for comprehensive, interpretable autonomous driving agents.