OptiMVMap: Offline Vectorized Map Construction via Optimal Multi-vehicle Perspectives

Abstract: Offline vectorized maps constitute critical infrastructure for high-precision autonomous driving and mapping services. Existing approaches rely predominantly on single ego-vehicle trajectories, which fundamentally suffer from viewpoint insufficiency: while memory-based methods extend observation time by aggregating ego-trajectory frames, they lack the spatial diversity needed to reveal occluded regions. Incorporating views from surrounding vehicles offers complementary perspectives, yet naive fusion introduces three key challenges: computational cost from large candidate pools, redundancy from near-collinear viewpoints, and noise from pose errors and occlusion artifacts. We present OptiMVMap, which reformulates multi-vehicle mapping as a select-then-fuse problem to address these challenges systematically. An Optimal Vehicle Selection (OVS) module strategically identifies a compact subset of helpers that maximally reduce ego-centric uncertainty in occluded regions, addressing computation and redundancy challenges. Cross-Vehicle Attention (CVA) and Semantic-aware Noise Filter (SNF) then perform pose-tolerant alignment and artifact suppression before BEV-level fusion, addressing the noise challenge. This targeted pipeline yields more complete and topologically faithful maps with substantially fewer views than indiscriminate aggregation. On nuScenes and Argoverse2, OptiMVMap improves MapTRv2 by +10.5 mAP and +9.3 mAP, respectively, and surpasses memory-augmented baselines MVMap and HRMapNet by +6.2 mAP and +3.8 mAP on nuScenes. These results demonstrate that uncertainty-guided selection of helper vehicles is essential for efficient and accurate multi-vehicle vectorized mapping. The code is released at https://github.com/DanZeDong/OptiMVMap.

• The paper introduces a novel Select–then–Fuse paradigm that optimally selects multi-vehicle perspectives to effectively reduce occlusion and uncertainty in HD mapping.
• The paper leverages cross-vehicle attention and semantic-aware noise filtering to mitigate pose errors and suppress redundant information during BEV fusion.
• The paper demonstrates significant mAP improvements on nuScenes and Argoverse2, validating its efficiency with minimal inference latency and computational overhead.

OptiMVMap: Offline Vectorized Map Construction via Optimal Multi-vehicle Perspectives

Introduction

Offline vectorized high-definition (HD) mapping is a core component of autonomous driving stacks, providing foundational infrastructure for localization, planning, and large-scale mapping. Conventional pipelines predominantly leverage single-vehicle ego-centric observations, which—despite extensions via temporal or memory-based aggregation—suffer from persistent viewpoint insufficiency, leading to incomplete recovery in occluded and long-range environments. OptiMVMap systematically addresses these limitations by introducing a selective, uncertainty-guided multi-vehicle fusion paradigm that targets information-deficient regions while mitigating computational, redundancy, and noise bottlenecks intrinsic to naively fused multi-agent observations.

Figure 1: Comparative schematics of single-vehicle memory-augmented approaches versus OptiMVMap's Select–then–Fuse pipeline.

Methodology

Select–then–Fuse Paradigm

OptiMVMap reformulates the multi-vehicle fusion problem from indiscriminate aggregation to a Select–then–Fuse process. An Optimal Vehicle Selection (OVS) module strategically identifies non-ego helper vehicles based on their ability to maximally reduce ego-centric BEV uncertainty in occluded and long-range regions. This selection encapsulates spatial relevance, geometric baseline diversity, and viewpoint complementarity, culling the candidate set to a computationally optimal compact subset ($K \ll M$). By focusing computation on those non-ego vehicles that are information-complementary relative to the ego’s uncertainty peaks, OVS suppresses collateral redundancy and obviates pose error noise propagation.

Figure 3: Full workflow of OptiMVMap: OVS ranks and selects non-ego views, followed by CVA-based alignment, semantic-aware noise denoising, BEV-level fusion, and vectorized decoding.

Pose-Tolerant Feature Fusion

The Cross-Vehicle Attention (CVA) mechanism addresses cross-vehicle misalignment stemming from pose estimation errors and time asynchrony, employing a lightweight deformable attention variant to learn adaptive spatial sampling and feature fusion between ego and non-ego BEV tensors. The Semantic-aware Noise Filter (SNF) further weights fusion inputs on a per-pixel, per-semantics basis, actively suppressing spurious or dynamically induced artifacts and modulating each vehicle’s contribution to the consolidated representation.

BEV Feature Pooling

The framework distinguishes between semantic and image features when transforming per-view to BEV space. For image features, Soft LSS (Lift, Splat, Shoot) suffices, but for semantics, Hard LSS is employed to avoid semantic ambiguities in overlapping regions—retaining only the highest-confidence assignment to each BEV cell.

Figure 2: Comparative effect between Soft LSS and Hard LSS for projecting per-view semantics into BEV, illustrating Hard-LSS’s superiority in maintaining semantic fidelity.

Training Protocol

Training is staged: CVA+SNF and the decoder are first pretrained using randomly sampled helpers; OVS is then trained end-to-end with the backbone frozen, employing BCE-loss against mAP-maximizing subsets. Downstream decoding is agnostic—OptiMVMap is compatible with major detection heads (e.g., MapTRv2).

Empirical Results

OptiMVMap establishes new benchmarks on nuScenes-MV and Argoverse2-MV, outperforming prior single-vehicle, memory-augmented, and multi-vehicle baselines under matched backbones:

• On nuScenes, OptiMVMap integrated with MapTRv2 yields +10.5 mAP over the baseline (61.5 → 72.0), outpacing HRMapNet and MVMap by +3.8 and +6.2 mAP, respectively.
• On Argoverse2, OptiMVMap achieves +9.3 mAP improvement on MapTRv2, surpassing the leading InteractionMap and MapExpert with only $K=2$ helpers.

The plug-and-play property holds across various detection heads, with, for example, +14.2 mAP on VectorMapNet and +6.1 mAP on MGMap. Ablations confirm that OVS-based selection dominates the performance gain, outstripping both random and closest-helper baselines by 4.4–6.9 mAP. Increasing $K$ beyond 2–5 yields diminishing returns, underscoring the efficacy of compact, complementary selection.

Figure 4: Qualitative outputs on nuScenes—OptiMVMap consistently recovers distant and occluded topology missed by both single-vehicle and memory-augmented methods.

Figure 5: Additional nuScenes scenes illustrating improved map completeness and noise suppression under severe occlusion and variable helper positions.

Computational Analysis

OVS adds negligible inference latency (~12.6ms per sample) due to restricted candidate set and low-resolution uncertainty representations. End-to-end throughput remains suitable for large-scale offline pipelines, scaling controllably with $K$.

Discussion and Implications

The theoretical contribution is a shift from brute-force multi-agent fusion to a utility-driven, selection-aware aggregation, setting precedent for broader multi-perspective selection architectures in collaborative perception. OptiMVMap’s modularity indicates direct applicability to other BEV-centric tasks like multi-agent scene flow, collaborative occupancy prediction, or scalable HD map auto-labeling. Robustness under extrinsic perturbations, shown by controlled pose noise experiments, highlights the necessity of explicit cross-agent alignment in real-world noisy datasets.

By demonstrating state-of-the-art results across datasets and decoders, this work motivates selection-aware, uncertainty-targeted fusion as a general principle in collaborative mapping. Broader theoretical questions regarding optimal selection in dynamic or resource-constrained cooperative perception remain compelling targets for future investigation.

Figure 6: Illustration of OptiMVMap's outputs on Argoverse2, establishing its cross-dataset generalization and robustness.

Conclusion

OptiMVMap presents a principled, uncertainty-guided, plug-and-play multi-vehicle mapping framework that consistently outperforms both single-vehicle and naive multi-agent baselines. By optimally selecting and robustly fusing complementary perspectives, it achieves superior topological and semantic map recovery, particularly in occluded and long-range scenarios. The methodology exemplifies the need for selective, quality-controlled collaboration in large-scale autonomous mapping and provides a strong foundation for successive research in selection-aware perception architectures (2604.17135).