MPTF-Net: Multi-view Pyramid Transformer Fusion Network for LiDAR-based Place Recognition

Abstract: LiDAR-based place recognition (LPR) is essential for global localization and loop-closure detection in large-scale SLAM systems. Existing methods typically construct global descriptors from Range Images or BEV representations for matching. BEV is widely adopted due to its explicit 2D spatial layout encoding and efficient retrieval. However, conventional BEV representations rely on simple statistical aggregation, which fails to capture fine-grained geometric structures, leading to performance degradation in complex or repetitive environments. To address this, we propose MPTF-Net, a novel multi-view multi-scale pyramid Transformer fusion network. Our core contribution is a multi-channel NDT-based BEV encoding that explicitly models local geometric complexity and intensity distributions via Normal Distribution Transform, providing a noise-resilient structural prior. To effectively integrate these features, we develop a customized pyramid Transformer module that captures cross-view interactive correlations between Range Image Views (RIV) and NDT-BEV at multiple spatial scales. Extensive experiments on the nuScenes, KITTI and NCLT datasets demonstrate that MPTF-Net achieves state-of-the-art performance, specifically attaining a Recall@1 of 96.31\% on the nuScenes Boston split while maintaining an inference latency of only 10.02 ms, making it highly suitable for real-time autonomous unmanned systems.

• The paper introduces MPTF-Net, a multi-view pyramid transformer fusion network that integrates NDT-based BEV encoding and dual-feature RIV representation to extract viewpoint-invariant global descriptors, achieving recall rates up to 99.43%.
• The method fuses multi-scale features using a pyramidal transformer with bi-directional cross-attention along azimuth bins, effectively addressing occlusions, scale sensitivity, and rotational shifts.
• Experimental results on datasets like nuScenes, KITTI, and NCLT show that MPTF-Net outperforms prior state-of-the-art approaches with improvements of up to 2.16% and supports real-time performance at over 50 Hz.

MPTF-Net: Multi-view Pyramid Transformer Fusion for LiDAR-based Place Recognition

Introduction and Motivation

LiDAR-based Place Recognition (LPR) is critical for robust global localization and loop-closure in large-scale SLAM, addressing environments where visual methods fail due to illumination and weather. Conventional LPR pipelines either project 3D point clouds into Range Image Views (RIV) or Bird's Eye View (BEV) representations. However, RIV suffers from occlusions and scale sensitivity, while BEV methods based on simple statistics (e.g., height pooling, occupancy) discard fine-grained geometric structure, limiting discriminativity in complex or repetitive urban scenes.

The paper introduces MPTF-Net, a multi-view, multi-scale Fusion Network, to address these limitations. The key innovations include a Normal Distribution Transform (NDT)-based BEV encoding capturing local geometric complexity and intensity distributions, and a multi-scale pyramidal Transformer fusion capturing cross-view interaction hierarchically. The architecture is tailored for viewpoint-invariant, highly discriminative global descriptor extraction under strict real-time constraints.

Figure 1: Overview of MPTF-Net, highlighting dual-branch input and multi-scale fusion for descriptor extraction.

Methodology

NDT-based BEV Encoding

The BEV branch leverages the Normal Distribution Transform to represent each cell in a polar grid with a multivariate Gaussian, parametrized by mean and covariance. This probabilistic modeling encodes not only the spatial occupancy but also second-order anisotropic structural cues, which are robust to sensor noise and outliers. The following feature channels are constructed per cell: geometric entropy, intensity entropy, geometric probability density score (PDS), and intensity PDS. These encode both the complexity ($\mathcal{H}$) and density consistency of point distributions.

Figure 3: Block diagram of the BEV multi-feature encoding structure. NDT enables computation of geometric and intensity statistics at the cell level.

Visualization demonstrates that geometric and radiometric features derived from the NDT-BEV encoding are complementary and highlight both local structure and material differences.

Figure 2: Visualization of geometric and radiometric entropy and PDS from BEV encoding.

RIV Dual-Feature Representation

RIV encodes each LiDAR scan into a $32 \times 1056$ spherical grid, with two channels for normalized range and intensity. An occlusion-aware filling strategy ensures near points overwrite farther ones, improving geometric consistency and complementing the BEV's global context with fine-grained elevation cues.

Multi-scale Pyramid Transformer Fusion

Both RIV and BEV features are processed by separate ResNet-based backbones. The network aligns both branches azimuthally and fuses them at multiple scales through a pyramidal Transformer module utilizing bi-directional cross-attention. Critically, attention is constrained along azimuth bins, enforcing geometric consistency and computational efficiency.

Figure 4: MPTF-Net pipeline—RIV and BEV processed in parallel, multi-scale Transformer modules fuse features, final NetVLAD aggregation produces descriptor.

Hierarchical, scale-wise aggregation enables the fusion module to capture both local and global context. The omission of absolute positional embeddings ensures the descriptors retain robustness to viewpoint shifts and axial rotations.

Invariance and Aggregation

Descriptor generation concludes with a context-gating enhanced NetVLAD module. Shift-equivariance in the azimuth dimension (via circular convolutions and aligned-attention) translates to strict yaw-rotation invariance after global aggregation— a property mathematically formalized and empirically validated.

Figure 6: Visual analysis of rotation invariance—global descriptors remain structurally consistent under $55^\circ$ yaw rotation.

Quantitative assessments across rotation angles further validate high Recall@1 stability under large viewpoint perturbations.

Figure 8: Recall@1 across yaw rotations—MPTF-Net maintains consistent, superior performance.

Experimental Results

Comprehensive experiments are conducted on nuScenes (Boston and Singapore splits), KITTI, and the NCLT dataset, evaluating generalization, robustness, and temporal resilience. MPTF-Net achieves Recall@1 of 96.31% on nuScenes Boston and 99.43% on Singapore (unseen), exceeding previous SOTA by a margin of up to 2.16% on critical splits and maintaining superiority in mean Recall scores across temporally variant NCLT sessions and the diverse KITTI dataset.

Ablations demonstrate that:

• Replacing NDT-BEV with first-order statistics causes a substantial Recall@1 drop (from 96.31% to 94.21%),
• Eliminating cross-attention severely degrades performance (Recall@1 drops to 90.22%),
• Increasing fusion scales to four provides clear gains over two-scale or single-scale setups.

  Figure 5: Multi-scale fusion strategy analysis—four-scale design offers highest recall.

Runtime and Efficiency

On a modern GPU, MPTF-Net achieves an average end-to-end runtime of 19.62 ms/query (35.83M parameters), supporting real-time deployment at over 50 Hz. This runtime is competitive—even as it outperforms other frameworks in accuracy.

Figure 7: Runtime and efficiency comparison—MPTF-Net achieves a strong balance between inference speed and recall.

Theoretical and Practical Implications

The explicit modeling of second-order BEV statistics via NDT represents a significant direction in robust scene encoding, particularly for domains sensitive to structural repeatability and noise. The integration of multi-scale cross-attention, restricted along aligned azimuth bins, shows efficient high-capacity fusion can be achieved without forfeiting spatial consistency or incurring prohibitive computation.

From a practical standpoint, MPTF-Net offers a deployable solution for autonomous vehicles requiring fast, viewpoint-invariant localization across domains and changing seasons. Its invariance and efficiency presage utility in large-scale SLAM, long-term map maintenance, and safety-critical robotic perception.

Future developments may extend NDT modeling to incorporate temporal priors, further optimize fusion scales with neural architecture search, or adapt the methodology to heterogenous sensor fusion (e.g., radar, camera, and LiDAR) under partial observations.

Conclusion

MPTF-Net presents a robust, multi-view, multi-scale framework for LiDAR-based place recognition, combining NDT-based BEV encoding and pyramid Transformer fusion into an architecture that excels in accuracy, invariance, and speed. Extensive empirical evidence and strong ablation results substantiate the advantages of this principled fusion approach, supporting its further adoption and extension in SLAM and autonomous systems research.

(2604.04513)