MapDiffusion: Generative HD Map Construction

• MapDiffusion is a generative diffusion framework that constructs online vectorized HD maps from multi-sensor BEV inputs, modeling spatial uncertainty in autonomous driving.
• It employs a BEV encoder and a DETR-style graph diffusion decoder that refines polyline queries over several diffusion steps using deformable cross-attention.
• The system aggregates multiple map samples to quantify uncertainty, enhancing downstream planning by addressing challenges like occlusions and ambiguous lane markings.

MapDiffusion is a generative diffusion framework for online vectorized HD map construction in autonomous driving. It addresses the task of inferring a local, instance-level map around the ego vehicle in real time from multi-sensor inputs, primarily six monocular cameras, represented in bird’s-eye view (BEV). Rather than producing a single deterministic vector map, it models the conditional distribution $p(M \mid I)$ over plausible vectorized maps, enabling multiple samples, sample aggregation, and a spatial uncertainty estimator $U(x,y)$ that reflects ambiguity induced by occlusions, missing lane markings, lane-width ambiguities, and temporally inconsistent construction zones (Monninger et al., 29 Jul 2025).

1. Task definition and problem setting

Online vectorized HD map construction seeks a local map expressed as a set of vector primitives in the ego-centric BEV frame. In MapDiffusion, the scene map is

$$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$

where the primitives are lane dividers, lane boundaries, and pedestrian crossings. Each primitive $P$ is parameterized as a polyline,

$P = [(x_j, y_j)]_{j=1}^{N_P},$

that is, a sequence of 2D control points. The learning objective is to infer a map estimate $\hat{M}$ from camera frames $I = \{i_1,\dots,i_m\}$, while simultaneously providing an uncertainty estimator $U(x,y)$ over BEV locations.

The motivating distinction from prior online map construction systems is that earlier vector decoders such as VectorMapNet, MapTR, and StreamMapNet produce a single point estimate. In the setting described by MapDiffusion, that determinism is a structural limitation rather than merely an implementation choice: real scenes may admit several plausible interpretations because parts of the static environment are invisible, partially erased, or geometrically ambiguous. A single decoded polyline set can therefore be both inaccurate and unsafe for downstream planning. Distributional modeling, by contrast, permits multiple plausible samples, aggregation for improved robustness, and spatially resolved uncertainty derived from sample disagreement (Monninger et al., 29 Jul 2025).

2. Representation and architecture

The model combines a learned BEV encoder with a diffusion-based vector decoder. Multi-camera inputs are fused into a latent BEV grid $B$ using a learned encoder with temporal aggregation in BEV space, following the policy used in StreamMapNet. This preserves online operation while introducing temporal context at the encoder level.

The decoder operates on a fixed set of $l$ DETR-style queries. During training, ground-truth polylines are perturbed by a forward diffusion process $U(x,y)$0 to create noisy query inputs, while unused query slots are padded with Gaussian noise because the number of ground-truth instances is variable and may be smaller than $U(x,y)$1. During inference, all queries are initialized randomly. The denoising network is a graph diffusion decoder that iteratively refines the query set over $U(x,y)$2 steps. Conditioning on the BEV latent $U(x,y)$3 is implemented through deformable cross-attention, and the decoder also receives a time-step embedding $U(x,y)$4.

A notable architectural decision is that classification scores are predicted only at the final step. The diffusion process itself therefore remains in vector space, acting on polyline coordinates rather than on joint coordinate-class states. The decoder regresses control-point coordinates and predicts one class score per query, yielding a sampled map as a set of denoised polylines with class assignments. Topology is not explicitly modeled as a separate constraint during diffusion; instead, topology is treated as emerging from consistent geometry and class prediction. The temporal policy is similarly asymmetric: temporal aggregation is used in the encoder, but previously refined queries are not injected into the decoder for the next frame, so each frame’s samples are generated purely from random noise, preserving diversity in generative sampling (Monninger et al., 29 Jul 2025).

3. Diffusion formulation in vector space

MapDiffusion applies DDPM-style noising directly to the tokenized vector map representation. Let $U(x,y)$5 denote the concatenated polyline coordinates across queries, and let $U(x,y)$6, $U(x,y)$7, and $U(x,y)$8. The forward process is

$U(x,y)$9

$$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$0

The reverse process is conditioned on the BEV latent grid $$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$1, written as $$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$2, and parameterized by the denoising network: $$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$3 In practice, conditioning enters through deformable cross-attention from the diffusion queries into the BEV latent grid.

Training uses a cosine noise schedule with $$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$4 steps and a standard $$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$5-prediction objective,

$$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$6

Final-step classification and coordinate regression are optimized jointly but remain outside the diffusion trajectory so that denoising remains a coordinate-space process. This design makes the method a diffusion decoder for vectorized geometry rather than a raster denoiser or a class-conditional token generator. A plausible implication is that the model’s expressivity is concentrated on geometric ambiguity, which is precisely the source of uncertainty emphasized in the task formulation (Monninger et al., 29 Jul 2025).

4. Sampling, aggregation, and uncertainty estimation

Inference begins by computing the BEV latent grid $$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$7 once from the camera inputs. A set of $$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$8 queries is then initialized with clipped Gaussian noise, and DDIM sampling is run for $$M = (P_{\mathrm{div}}, P_{\mathrm{bound}}, P_{\mathrm{ped}}),$$9 denoising steps; the default configuration uses $P$0 and $P$1. After each intermediate step, low-confidence queries are filtered using a score threshold $P$2; the best setting reported is $P$3. Queries that fall below threshold before the final step are reinitialized randomly rather than discarded, which is intended to avoid poor local minima while preserving sample diversity. Repeating the process with different random seeds yields multiple map samples, with the BEV encoding shared across all samples and therefore parallelizable.

Aggregation is performed in raster space. For each sample $P$4 and class $P$5, predicted polylines are rasterized and weighted by their confidence scores, then Gaussian-smoothed and clipped to produce a class-specific raster probability map $P$6. The mean prediction over $P$7 samples is

$P$8

and the spatial uncertainty map is obtained by summing per-class sample variances,

$P$9

The paper also defines per-class empirical mean and variance explicitly as

$P = [(x_j, y_j)]_{j=1}^{N_P},$0

This uncertainty measure is intended to quantify disagreement across plausible map samples rather than epistemic uncertainty from model ensembles. In the reported experiments, invisible or occluded areas exhibit higher uncertainty than visible areas, with mean $P = [(x_j, y_j)]_{j=1}^{N_P},$1 versus $P = [(x_j, y_j)]_{j=1}^{N_P},$2, corresponding to $P = [(x_j, y_j)]_{j=1}^{N_P},$3, and a one-sided $P = [(x_j, y_j)]_{j=1}^{N_P},$4-test yields $P = [(x_j, y_j)]_{j=1}^{N_P},$5 at $P = [(x_j, y_j)]_{j=1}^{N_P},$6. ROC-based evaluation of the rasterized aggregated prediction shows that increasing the number of samples from $P = [(x_j, y_j)]_{j=1}^{N_P},$7 to $P = [(x_j, y_j)]_{j=1}^{N_P},$8 improves AUC from $P = [(x_j, y_j)]_{j=1}^{N_P},$9 to $\hat{M}$0, indicating that explicit distribution modeling is beneficial not only for uncertainty visualization but also for prediction quality after aggregation (Monninger et al., 29 Jul 2025).

5. Training regime, inference configuration, and empirical performance

The reported experiments use nuScenes at $\hat{M}$1, with six monocular cameras as input and vectorized ground-truth labels for lane dividers, lane boundaries, and pedestrian crossings. The StreamMapNet training and validation split without geospatial overlap is used. Camera features are fused into a BEV latent grid of size $\hat{M}$2, covering $\hat{M}$3 around the ego vehicle, and temporal aggregation is performed in BEV space.

The encoder is StreamMapNet-style, while the decoder is a DETR-style transformer with deformable cross-attention and diffusion time-step embeddings. Training runs for 24 epochs with batch size 1, AdamW, cosine learning-rate scheduling, initial learning rate $\hat{M}$4, and 8 NVIDIA V100 GPUs. Diffusion training uses the cosine noise scheduler with $\hat{M}$5. At inference, the default settings are DDIM sampling with $\hat{M}$6, $\hat{M}$7 diffusion steps, $\hat{M}$8 samples for aggregation, and $\hat{M}$9 for query filtering. Rasterization uses pixel width 1 for polylines with score $I = \{i_1,\dots,i_m\}$0, Gaussian kernel size $I = \{i_1,\dots,i_m\}$1 (approximately $I = \{i_1,\dots,i_m\}$2), and $I = \{i_1,\dots,i_m\}$3 (approximately $I = \{i_1,\dots,i_m\}$4).

Under vector decoding metrics, MapDiffusion reaches single-sample mAP of $I = \{i_1,\dots,i_m\}$5, compared with $I = \{i_1,\dots,i_m\}$6 for the StreamMapNet baseline, a $I = \{i_1,\dots,i_m\}$7 relative gain. Under rasterized evaluation, aggregation improves AUC from $I = \{i_1,\dots,i_m\}$8 at $I = \{i_1,\dots,i_m\}$9 to $U(x,y)$0 at $U(x,y)$1, a $U(x,y)$2 relative gain. Latency is governed mainly by the denoising loop because the encoder is executed only once: on an NVIDIA A10 GPU, each extra denoising step adds approximately $U(x,y)$3; $U(x,y)$4 yields $U(x,y)$5 FPS, while $U(x,y)$6 yields $U(x,y)$7 FPS. The qualitative trend reported in the paper is that samples agree strongly in visible regions and diverge in occluded regions, aligning the sample-variance map with scene ambiguity (Monninger et al., 29 Jul 2025).

6. Comparative position, ablations, limitations, and downstream use

MapDiffusion is positioned against two groups of baselines. Relative to deterministic vector decoders such as VectorMapNet, MapTR, MapTRv2, and StreamMapNet, it adds explicit distribution modeling and uncertainty estimation. Relative to diffusion systems that operate on raster semantics, such as DiffMap and DifFUSER, it generates vectorized polylines directly. The paper also distinguishes it from diffusion methods for other structured outputs, including DiffBEV, DiffusionDet, and HouseDiffusion, and from PolyDiffuse, which is described as using diffusion only for refining coarse predictions rather than sampling diverse map configurations.

The ablation study attributes performance gains to several specific design choices. Increasing the number of diffusion steps improves accuracy but saturates around $U(x,y)$8. The DDIM stochasticity parameter $U(x,y)$9 has a minor effect, with the best result at $B$0. Query thresholding is sensitive: $B$1 is best, while $B$2 and $B$3 reduce mAP to $B$4 and $B$5, respectively. BEV encoder pretraining accelerates early optimization but reduces final mAP to $B$6, whereas training from scratch reaches $B$7. For padding unused query slots during training, Gaussian noise performs best, while repeating existing polylines performs worst because it confuses the decoder.

The stated limitations are also specific. Under heavy occlusion or sparse sensing, uncertainty increases; this is useful for planning but may require more samples for robust aggregation. The decoder deliberately omits temporal query carryover, which preserves generative diversity but may reduce temporal consistency. Aggregation is currently performed in raster space; the paper suggests that direct set aggregation in vector space could improve geometry fidelity. These limitations imply that the method trades some temporal determinism and vector-set exactness for explicit multimodal sampling and uncertainty quantification.

In an autonomy stack, the uncertainty map $B$8 can be used for risk-aware planning, including cost-map modulation, slow-down policies, or requesting additional sensor evidence in uncertain regions. Multiple sampled maps $B$9 can support hypothesis testing for maneuver evaluation under ambiguous map geometry. The refined raster distributions $l$0 are also compatible with downstream modules that already consume occupancy-style grids, while the sampled polylines and associated confidences remain available for HD-map-aware motion planning. This suggests a perception-planning interface in which vectorized map construction is no longer treated as a deterministic preprocessing stage but as a probabilistic component with explicit spatial uncertainty (Monninger et al., 29 Jul 2025).