FASTopoWM: Unified Lane Topology

• FASTopoWM is a unified framework for lane topology reasoning using a dual fast-slow decoder that integrates temporal BEV fusion and latent world models.
• It mitigates previous limitations by reducing error accumulation and pose-estimation sensitivity through a fallback fast branch and robust latent projections.
• Empirical results demonstrate significant mAP gains in lane detection and topology inference on large-scale benchmarks, underscoring its practical impact.

FASTopoWM is a unified framework for lane segment topology reasoning, introducing a fast-slow architecture augmented by latent world models to advance bird’s-eye view (BEV) road scene understanding. Designed for robust perception in autonomous driving, FASTopoWM addresses deficiencies in temporal reasoning that hinder prior models, achieving state-of-the-art results for lane detection and topological inference on large-scale benchmarks (Yang et al., 31 Jul 2025).

1. Problem Statement and Limitations of Previous Methods

Lane segment topology reasoning involves inferring the spatial arrangement and topological connectivity of lane segments from sequences of multi-view images, typically lifted into a BEV representation. This modality enables high-level planning in autonomous systems by providing structured, map-like perception.

Earlier single-frame methods—such as MapTR, TopoLogic, and TopoFormer—process each frame in isolation, failing to enforce temporal consistency and thereby producing outputs with jitter or discontinuities in global coordinates. More recent stream-based temporal propagation methods (e.g., StreamMapNet, SQD-MapNet) attempt to leverage temporal context by warping past “stream queries” and BEV features into the present frame using estimated ego-motion. While this approach introduces temporal cues, it displays three critical weaknesses:

• Over-reliance on historical queries during training (via Hungarian matching) can marginalize newly initialized queries, resulting in degraded first-frame predictions and error accumulation.
• Sensitivity to pose-estimation errors, such as those arising under GNSS denial or IMU drift, can cause catastrophic performance drops since historical information becomes misaligned.
• Inadequate temporal modeling, as simple warping or fixed MLPs fail to capture nuanced spatiotemporal evolution in BEV and query representations.

2. FASTopoWM Architecture

FASTopoWM introduces three central innovations: a unified fast-slow decoder, learned latent world models for queries and BEV features, and robust mode switching under pose failure.

1. PV-to-BEV Encoding: Input multi-view images are processed by a ResNet-50 backbone with FPN, followed by a BevFormer-based spatiotemporal BEV encoder, yielding $$F^{T}_{\text{bev}} \in \mathbb{R}^{H \times W \times C}$$.
2. Query Initialization: At each timestep $T$, $N$ learnable queries $Q^T \in \mathbb{R}^{N \times C}$ are initialized.
3. Action Latent Construction: Relative ego-motion between $T-1$ and $T$ is flattened into an action latent $\Psi \in \mathbb{R}^d$, encapsulating rotation and translation.
4. Latent World Models (Slow Pipeline):
   • Query World Model (QWM): Projects historical queries $Q^{T-1}$ and action latent $\Psi$ into current stream queries $\tilde Q^T$ using Transformer blocks.
   • BEV World Model (BWM): Propagates $F^{T-1}_{\text{bev}}$ and $\Psi$ into $\tilde F^T_{\text{bev}}$ with temporal self-attention.
5. Fusion: Fused BEV representation for the slow branch is obtained by combining $\tilde F^T_{\text{bev}}$ with $F^T_{\text{bev}}$ via a GRU along the channel dimension.
6. Unified Fast–Slow Decoder:
   • Initial transformer layer is shared.
   • Layers 1–5 split into fast (purely from $Q^T$, $F^T_{\text{bev}}$) and slow (combining top-K stream queries $\tilde Q^T$ with up-to-date BEV features) branches.
   • Both branches are supervised in parallel with dedicated Hungarian-matching loss heads.
7. Inference Switching: The output is selected from the slow branch when reliable pose deltas are available; otherwise, the system falls back to the fast (single-frame) branch.

3. Mathematical Formulation of Latent World Models

The temporal evolution of queries and BEV features is governed by the following:

• Action-Aware Projection:

$$\tilde Q^{T-1} = \mathrm{MLP}\left([Q^{T-1}, \Psi]\right), \quad \tilde F_{\text{bev}}^{T-1} = \mathrm{MLP}\left([F_{\text{bev}}^{T-1}, \Psi]\right)$$

• Temporal Propagation with Transformers:

$$\tilde Q^{T} = \mathrm{QueryWorldModel}\left(\tilde Q^{T-1}\right), \quad \tilde F_{\text{bev}}^{T} = \mathrm{BEVWorldModel}\left(\tilde F_{\text{bev}}^{T-1}\right)$$

Both world models use stacks of Transformer blocks with (spatial and temporal) self-attention followed by feed-forward networks.

• Temporal BEV Fusion:

$$F_{\text{bev}}^{\text{fused}} = \mathrm{GRU}\left(F^T_{\text{bev}}, \tilde F^T_{\text{bev}}\right)$$

This learned fusion augments the current frame with temporally propagated context.

4. Training Objectives and Optimization

The overall objective function comprises both fast and slow branch losses:

• Lane-Segment Loss $L_{ls}$: DETR-style Hungarian matching between predicted lane segments and ground truth, aggregating $L_1$ coordinate loss, focal and cross-entropy for class labels, and connectivity accuracy for topology edge prediction.
• Latent-Model Supervision $L_{\text{latent}}$ (slow branch):

$$\mathcal{L}_{\text{bev}} = \|\tilde F^T_{\text{bev}} - F^T_{\text{bev}}\|_2$$

$$\mathcal{L}_{\text{query}} = L_1\left(\tilde{\mathbf L}_T, \mathbf L_T\right) + L_{\mathrm{Focal}}(\tilde{\mathit{Class}}_T, \mathit{Class}_T) + L_{\mathrm{CE}}(\tilde{\mathit{Type}}_T, \mathit{Type}_T) + L_{\mathrm{CE}}(\tilde{\mathbf M}_T, \mathbf M_T) + L_{\mathrm{Dice}}(\tilde{\mathbf M}_T, \mathbf M_T)$$

$$\mathcal{L}_{\text{latent}} = \mathcal{L}_{\text{bev}} + \mathcal{L}_{\text{query}}$$

The aggregate loss:

$$\mathcal{L}_{\text{slow}} = \alpha_1 \, \mathcal{L}_{ls}(\tilde Q^T_{\text{out}}, GT) + \alpha_2 \, \mathcal{L}_{\text{latent}}, \quad \mathcal{L}_{\text{fast}} = \mathcal{L}_{ls}(Q^T_{\text{out}}, GT), \quad \mathcal{L}_{\text{total}} = \mathcal{L}_{\text{slow}} + \mathcal{L}_{\text{fast}}$$

with $\alpha_1 = 1.0$, $\alpha_2 = 0.3$.

Ground-truth for the slow branch is aligned by warping the frame $T-1$ annotations to $T$ using the known relative pose. Both branches are trained in parallel, ensuring that newly initialized queries remain competitive even in frames lacking history.

5. Robustness and Temporal Adaptation

FASTopoWM is designed for resilience against pose-estimation imperfections. The dual-branch structure ensures that if pose deltas become unavailable or inaccurate (as in GNSS denial, IMU drift, or tunnels), the system reverts seamlessly to the single-frame (fast) branch, with performance degrading only to single-frame baseline rather than collapsing. This contrasts with prior stream-based propagation, where heavy reliance on pose warping amplifies errors under failure scenarios.

Temporal fusion via the GRU module enables integration of fine-grained, up-to-date contextual information with temporally propagated features, mitigating historical drift and capturing both short- and long-range dependencies in BEV space.

6. Empirical Results and Ablation Studies

Experimental validation on the OpenLane-V2 benchmark demonstrates the empirical benefits of FASTopoWM:

Benchmark Task	Prior SOTA Fast mAP	Prior SOTA Stream mAP	FASTopoWM Fast	FASTopoWM Slow
Lane Segment Detection + Topology	33.6% (Topo2Seq)	26.0% (SQD-MapNet)	34.1%	37.4%
Centerline Perception + Topology	41.5% (TopoFormer)	—	41.8%	46.3%

Key findings:

• The fast-only branch outperforms previous single-frame and stream-based methods.
• The slow branch (temporal world models and fusion) yields an additional 3.8% absolute mAP improvement in lane detection and 4.8% in centerline perception.
• Fast-slow system sans world models delivers 1.6% mAP gain; introduction of QueryWM and BEVWM adds another 2.1% mAP.
• Conditioning on action latents outperforms trajectory conditioning or absence of action context.

Qualitative analysis reveals reductions in endpoint misalignments, hallucinated lanes, and coverage gaps, especially in long sequences and high-dynamic scenarios.

7. Significance and Applications

FASTopoWM constitutes an advance in temporal perception for autonomous driving, enabling more accurate lane topology reasoning under a variety of operational constraints. Its decoupled fast-slow architecture ensures robust operation regardless of pose-estimation fidelity, while its latent world models introduce learned spatiotemporal propagation mechanisms absent from previous approaches.

The architecture is directly applicable to planning-oriented, end-to-end autonomous driving stacks, where temporally stable topology reasoning is a prerequisite for motion planning and behavior prediction. By providing a unified, parallel solution to both single-frame and temporally-aware scenarios, FASTopoWM establishes a benchmark for future lane topology reasoning systems (Yang et al., 31 Jul 2025).