FMANet: Future-Aware Motion Forecasting

• FMANet is a future-aware interaction network for motion forecasting that integrates potential future trajectories with historical and contextual scene data.
• It employs an efficient Mamba state-space model with adaptive reordering to encode precise spatiotemporal interactions, reducing computational load compared to transformer models.
• FMANet demonstrates improved accuracy and efficiency on benchmarks like Argoverse, making it ideal for real-time autonomous driving applications.

FMANet refers to several distinct neural network architectures sharing similar nomenclature across recent machine learning literature, each addressing specialized domains such as motion forecasting in autonomous driving, visual inspection of textured surfaces, and micro-expression recognition. This article focuses primarily on FMANet for motion forecasting in autonomous systems (Li et al., 9 Mar 2025), but incorporates context for related “FMANet” variants to clarify distinctions and lineage.

1. Conceptual Overview

FMANet, in the context of future-aware motion forecasting, is an interaction-based network architecture designed for trajectory prediction in autonomous driving. Its principal innovation lies in integrating representations of potential future trajectories into the scene encoding process, alongside historical agent motion and map elements. This facilitates joint optimization and comprehensive scene representation, enabling the system to produce accurate, smooth, and diverse future trajectory distributions for road agents.

In contrast to previous methods that typically insert future prediction modules after scene encoding, FMANet internalizes the predictive modeling of possible futures at the earliest stages, influencing all subsequent spatiotemporal reasoning. This interaction-based and future-aware strategy is paired with an efficient State Space Model (SSM), specifically the Mamba architecture, which replaces transformer-based spatiotemporal modules to address scalability and efficiency.

2. Integration of Future-Aware Interaction

Traditional motion forecasting approaches often operate in a staged fashion: scene elements (agents and map) are encoded, queries (historical agent states) are processed, and possible futures are generated. FMANet departs by integrating future trajectory tokens within the scene encoding:

• Future Trajectory Tokenization: Let $\mathcal{T}_0^{\mathrm{hist}}$ denote the historical motion feature of the focal agent, $\mathcal{T}^{\mathrm{DI}}$ the driver intention tokens, and $\mathcal{T}^{\mathrm{bias}}$ an inductive bias. The set of future-aware tokens is constructed as

$$\mathcal{T}^{\mathrm{fut}} = \mathcal{T}_0^{\mathrm{hist}} + \mathcal{T}^{\mathrm{bias}} + \mathcal{T}^{\mathrm{DI}}$$

• Concatenation in Scene Encoding: These tokens are concatenated with all other scene tokens (including agents and lane features), forming a joint representation on which spatial interactions operate.
• Implication: This structure ensures that every part of the encoded scene “knows” about plausible future motion, promoting mutual constraint between current state, map context, and predicted future evolutions. The result is an encoded tensor wherein trajectories, agents, and map elements are all embedded in a “future-aware” latent space.

This framework contrasts directly with MLP-based or query-driven forecasting models, which risk suboptimal coordination between historical evidence and predicted futures due to their strictly serial flow.

3. State Space Model and Adaptive Reordering for Spatial Interaction

To efficiently encode spatiotemporal interactions, FMANet employs the Mamba SSM, which provides linear complexity with respect to sequence length—an advantage over transformer modules prone to quadratic scaling.

• Spatial Application of Mamba: Spatial interactions using Mamba require sequence-ordered (not unordered) data. However, typical scene tokens lack such inherent order.
• Adaptive Reorder Strategy (ARS): FMANet predicts a reference position $\mathcal{P}_R$ (from the focal agent’s history via MLPs), and all tokens are ordered according to spatial proximity to $\mathcal{P}_R$. The scene tokens $\mathcal{ST}$ are thus reordered by

$$\mathcal{OS}_1 = \mathrm{Reorder}(\mathcal{ST}, \mathcal{P}_R)$$

The focal agent's token is forced to the end of the sequence to maximize its influence over future token ordering.

• Sequential Modeling: This surrogate ordering transforms the set into a spatial sequence, enabling Mamba to model fine-grained spatial relationships that would otherwise be inaccessible without explicit ordering. The use of Mamba blocks after ARS captures both local and global spatial dependencies, necessary for accurate inter-agent interaction modeling.

4. Temporal Refinement and Diverse Prediction

• Temporal Modeling: After spatial encoding, future trajectory tokens are further refined using bidirectional Mamba modules that model the evolution of predicted trajectories over time. This process ensures not only accurate end-point predictions but also smooth and plausible intermediate trajectories.
• Temporal Enhanced Decoder (TEDec):
  • An extension process interpolates between initial and predicted future states:

  $$\mathrm{IDT}^{\mathrm{fut}} = \frac{t}{T_f} \times \mathcal{ST}^{\mathrm{fut}}, \quad \mathrm{DT}^{\mathrm{fut}} = \mathcal{ST}_0^{\mathrm{scene}} + \mathrm{IDT}^{\mathrm{fut}}$$ - Cross-attention and bidirectional Mamba blocks are then applied:

  $$\mathrm{DT}^{\mathrm{fut}}_{\mathrm{ca}} = \mathrm{CA}(\mathrm{DT}^{\mathrm{fut}}, \mathrm{ST})$$

  $$\mathrm{DT}^{\mathrm{fut}}_m = \mathrm{M}(\mathrm{DT}^{\mathrm{fut}}_{\mathrm{ca}})$$ - This promotes temporal consistency across steps and suppresses trajectory “jumps.”

• Diversity: Because each future trajectory token interacts with agents and other futures, FMANet generates sets of plausible, context-sensitive future predictions, addressing the multi-modality of road agent behavior.

5. Computational Efficiency and Benchmarking

FMANet’s architecture exhibits several advantages in efficiency and accuracy, as demonstrated on the Argoverse 1 and Argoverse 2 benchmarks:

Metric	Transformer Baseline	FMANet
FLOPs	28.0G	1.47G
Final Displacement Error	Higher	Lower (up to 10% rel)
Miss Rate (MR)	Higher	Lower
Model Size / Latency	Larger, Slower	Smaller, Faster

• Efficiency: The use of Mamba state-space modeling, lightweight PointNet modules, and ordered token processing yields orders-of-magnitude reductions in FLOPs, model size, and latency.
• Accuracy: On Argoverse 1, FMANet achieves approximately a 10% improvement in minADE and consistently reduces final displacement error and miss rates relative to state-of-the-art baselines.
• Qualitative Consistency: Visualizations show that FMANet’s predictions are both diverse and faithful to ground truth agent behaviors, while transformer-based models occasionally produce implausible outputs.

6. Applications and Deployment in Autonomous Systems

FMANet’s design is well-suited for the unique requirements of autonomous driving:

• Deployment: The high efficiency makes it practical for real-time inference in vehicles with limited hardware.
• Covering Complex Scenarios: The model's capacity for multi-modal and diverse predictions allows risk assessment and planning modules to consider a broad set of plausible futures, enhancing safety.
• Integration: FMANet’s architecture can be directly embedded within an autonomous vehicle's perception-planning pipeline, providing a unified, future-aware representation for downstream navigation and decision-making.

7. Future Research Directions

Several potential avenues are highlighted for advancing FMANet:

• Enhanced Intention Modeling: Integration of richer driver intention cues or sensor modalities.
• Ordering Strategy Refinement: Further development of ARS could improve sensitivity to spatial context or agent grouping.
• Longer Horizon Forecasting: Combating the challenges associated with predicting over longer time frames and in more densely populated scenarios, possibly via deeper state-space modeling.
• Generalization: Investigating domain robustness for deployment in heterogeneous traffic conditions or novel cities.
• Public Availability: Release of code and models is intended upon acceptance, facilitating reproducibility and fostering further exploration by the research community.

---

FMANet’s emergence as a “future-aware interaction network” for motion forecasting marks a significant evolution in trajectory prediction system design. By fusing prospective information at the core of scene encoding and leveraging efficient SSM-based modeling, FMANet not only establishes new benchmarks in accuracy and diversity for motion forecasting but also presents a scalable solution for real-world autonomous driving applications (Li et al., 9 Mar 2025).