---
title: 'HDMapNet: Online HD Semantic Mapping'
url: https://www.emergentmind.com/topics/hdmapnet-framework
type: topic
---

# HDMapNet: Online HD Semantic Mapping

HDMapNet is an online high-definition semantic map construction and evaluation framework designed to support autonomous driving through scalable, real-time generation of BEV (bird’s-eye-view) vectorized road maps from onboard sensor observations. Unlike traditional mapping pipelines reliant on extensive offline SLAM and manual annotation, HDMapNet enables dynamic inference of road semantics, supporting downstream tasks such as path prediction and planning. The framework represents semantic map elements as polylines in the BEV domain, employs unified evaluation protocols, and demonstrates robust performance improvements over previous projection-based approaches [2107.06307].

## 1. Problem Definition and Objectives

High-definition semantic map learning is formulated as an online estimation problem:

- **Inputs**: Surround-view camera images $\{I_i \in \mathbb{R}^{H_{pv} \times W_{pv} \times 3}\}_{i=1}^{N_m}$ and/or a 3D LiDAR sweep $P = \{p_n=(x_n, y_n, z_n)\}_{n=1}^N$.
- **Outputs**: A local HD semantic map $M$, composed of vectorized map elements $\{C^k\}$, where each $C^k$ is a polyline in the ego-vehicle BEV frame.

The main goals are:
- Elimination of costly pre-built global maps.
- Real-time, scalable local map construction from sensor data.
- Provision of unified semantic- and instance-level evaluation protocols.

## 2. Architectural Modules

HDMapNet comprises four key components:

| Module                      | Input Type           | Function              |
|-----------------------------|---------------------|-----------------------|
| Perspective-View Image Encoder $\phi_I$ | Camera             | Multi-scale PV features |
| Neural View Transformer $\phi_V$        | PV features         | PV $\rightarrow$ BEV mapping |
| Pillar-based LiDAR Encoder $\phi_P$     | LiDAR              | BEV pillar features   |
| BEV Map Decoder $\phi_M$                | Fused BEV tensor   | Vectorized map output |

- **Camera-only**: $I \rightarrow \phi_I \rightarrow \phi_V \rightarrow \phi_M$
- **LiDAR-only**: $P \rightarrow \phi_P \rightarrow \phi_M$
- **Fusion**: Camera-derived BEV and LiDAR BEV features are concatenated pre-$\phi_M$, maximizing information content.

The BEV Decoder $\phi_M$ includes three output heads for semantic segmentation, instance embedding, and direction classification.

## 3. Neural View Transformation and Polyline Vectorization

The camera branch leverages a neural view transformer for perspective-to-BEV mapping:

- For each PV image, $\phi_I$ extracts $F_{I_i}^{pv}$.
- A multi-layer perceptron aggregates PV pixels per BEV cell: $F_{I_i}^{c}[h,w] = \phi_{V_i}^{hw}(\{F_{I_i}^{pv}[u,v]\}_{u,v})$.
- Camera extrinsics warp PV features to BEV space.
- Multi-view BEV features are averaged: $F_{I}^{bev} = \frac{1}{N_m}\sum_{i=1}^{N_m} F_{I_i}^{bev}$.

Instance polylines are constructed by clustering embedding maps and applying greedy polyline tracing based on predicted direction bins:

$$
c_{t+1} = c_t + \Delta_{\text{step}} \cdot d(c_t), \quad d(c_t) \in \{\text{unit vectors in }N_d\text{ bins}\}
$$

The map representation is a vectorized set of polylines $C = \{c_1, ..., c_L\}$ with $c_\ell \in \mathbb{R}^2$, rather than a dense occupancy grid.

## 4. Loss Functions and Training Protocol

HDMapNet's total loss function combines semantic, instance, and direction losses:

$$
L = L_{\text{seg}} + L_{\text{inst}} + L_{\text{dir}}
$$

- **Semantic segmentation**: Pixel-wise cross-entropy.
- **Instance embedding**: Discriminative loss with variances and inter-instance distances:
  $$
  L_{\text{inst}} = \alpha\,L_{\text{var}} + \beta\,L_{\text{dist}}, \quad \alpha=\beta=1
  $$
- **Direction classification**: Cross-entropy on direction classes, lane pixels only.
- **Optimization**: Adam ($1 \times 10^{-3}$), weight decay ($1 \times 10^{-7}$), decays by 0.1 every 10 epochs.

## 5. Sensor Fusion and Performance

Three sensor integration modes are supported:

- **HDMapNet(Surr)**: Cameras-only, adept at lane dividers and crosswalks.
- **HDMapNet(LiDAR)**: LiDAR-only, excels at geometry but less effective for lane markings.
- **HDMapNet(Fusion)**: Concatenation of camera and LiDAR BEV features before $\phi_M$.

Fusion yields significant improvements:

| Method            | IoU (All %) | CD (m) | mAP (All %) |
|-------------------|-------------|--------|-------------|
| IPM(CB)           | 32.4        | 0.839  | 19.7        |
| Lift-Splat-Shoot  | 30.8        | 0.968  | 17.4        |
| VPN               | 29.3        | 1.337  | 17.5        |
| HDMapNet(Surr)    | 32.9        | 0.834  | 22.7        |
| HDMapNet(LiDAR)   | 29.5        | 1.101  | 11.6        |
| HDMapNet(Fusion)  | **44.5**    | **0.639** | **30.6**   |

Fusion achieves a 12.1 point absolute IoU gain and a 10.9 point mAP gain over best camera-based baselines [2107.06307].

## 6. Evaluation Metrics and Temporal Consistency

HDMapNet employs both Eulerian and Lagrangian evaluation protocols:

- **Semantic IoU**:
  $$
  \mathrm{IoU} = \frac{|\hat S \cap S|}{|\hat S \cup S|}
  $$
- **Chamfer Distance (CD)** for vectorized curves:
  $$
  \mathrm{CD}(C^A, C^B) = \frac{1}{|C^A|} \sum_{x\in C^A} \min_{y\in C^B} \|x-y\| + \frac{1}{|C^B|} \sum_{y\in C^B} \min_{x\in C^A} \|y-x\|
  $$
- **Instance-level mAP**: Average precision over recall thresholds, with true positives defined by Chamfer distance criteria.

Temporal fusion via max-pooling BEV probabilities across ego poses supports locally consistent map accumulation, improving robustness to sensor variability and environmental changes.

## 7. Limitations, Extensions, and Related Frameworks

Key limitations include:

- **Heuristic vectorization**: Polyline tracing is greedy; learned graph-generation may improve topology.
- **Simple fusion**: Camera-LiDAR fusion is concatenation; uncertainty-aware fusion mechanisms could further enhance complementarity.
- **Accuracy tradeoff**: Online maps do not match offline map precision but offer scalability.

*This suggests* that future extensions should consider advanced fusion strategies, temporal sequence modeling, and expansion to richer semantic layers (e.g., curbs, signage). Related approaches such as the input-level raster fusion and online map prediction in HDNET [2012.11704], explicit height modeling and foreground-background masking in HeightMapNet [2411.01408], and global vector map construction with GlobalMapNet [2409.10063] expand HDMapNet’s methodology to 3D object detection, height-aware BEV learning, and global online mapping, respectively.

HDMapNet defines the formal problem of online HD semantic map learning, establishes comprehensive evaluation standards, and delivers substantial performance gains over prior BEV and projection-based semantic mapping strategies [2107.06307].

Source: https://www.emergentmind.com/topics/hdmapnet-framework