HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds (1906.05332v1)

Abstract: We present a novel deep neural network architecture for end-to-end scene flow estimation that directly operates on large-scale 3D point clouds. Inspired by Bilateral Convolutional Layers (BCL), we propose novel DownBCL, UpBCL, and CorrBCL operations that restore structural information from unstructured point clouds, and fuse information from two consecutive point clouds. Operating on discrete and sparse permutohedral lattice points, our architectural design is parsimonious in computational cost. Our model can efficiently process a pair of point cloud frames at once with a maximum of 86K points per frame. Our approach achieves state-of-the-art performance on the FlyingThings3D and KITTI Scene Flow 2015 datasets. Moreover, trained on synthetic data, our approach shows great generalization ability on real-world data and on different point densities without fine-tuning.

• The paper presents HPLFlowNet, a novel deep learning architecture for direct scene flow estimation from large-scale 3D point clouds, bypassing indirect stereo matching.
• HPLFlowNet introduces hierarchical layers (DownBCL, UpBCL, CorrBCL) and density normalization to efficiently handle varying point cloud densities and fuse temporal information.
• Empirical evaluation on FlyingThings3D and KITTI shows HPLFlowNet outperforms state-of-the-art models in accuracy and efficiency, demonstrating strong generalization capabilities.

Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds

The paper presents HPLFlowNet, a novel deep learning architecture tailored for the direct estimation of scene flow from large-scale 3D point clouds. This approach builds upon the core concept of Bilateral Convolutional Layers (BCL) but introduces DownBCL, UpBCL, and CorrBCL operations designed to address the challenges specific to unstructured point clouds while maintaining computational efficiency.

Motivations and Contributions

Scene flow representation is crucial for understanding 3D motion fields, essential in applications such as autonomous driving and robotics. Traditional methods often rely on stereo images to first reconstruct 3D motion, introducing additional computation and errors from stereo matching processes. HPLFlowNet circumvents these indirect processes by operating directly on point clouds.

The proposed architecture improves upon existing BCL methods by effectively handling two principal challenges faced in real-time 3D scene analysis: processing point clouds of varying densities and efficiently fusing temporal information across frames. The seminal contributions of this work are the novel layer designs and a robust network architecture that collectively enhance computation and generalization capabilities.

Network Architecture and Innovation

HPLFlowNet lays out an hourglass-like architecture incorporating:

• DownBCL & UpBCL: These are advanced versions of BCL optimized for hierarchical downsampling and upsampling. They manage computational resources by reducing complex operations in splatting and slicing, thus allowing the network to handle entire point cloud frames with fewer operations.
• CorrBCL: This layer fuses data between consecutive point cloud frames to enrich scene flow estimation. It employs a patch correlation technique combined with displacement filtering to robustly interpret motion through neighborhood aggregations and feature correlations.
• Density Normalization: To tackle variations in point cloud densities typically encountered in LiDAR data, the authors propose a unique normalization approach applied during signal splatting that mitigates the influence of non-uniform density distribution on network performance.

Empirical Evaluation

The architecture was rigorously evaluated on two prominent datasets: FlyingThings3D and KITTI Scene Flow 2015. Results from the experiments indicate that HPLFlowNet outperforms state-of-the-art models, including FlowNet3D, on critical metrics such as EPE3D and Acc3D (strict and relaxed criteria). Particularly, HPLFlowNet's ability to generalize from synthetic to real-world data was demonstrated by testing on the KITTI dataset without additional retraining or fine-tuning.

The comparative assessment further included efficiency analysis where the network showed reduced computational cost and improved speed while maintaining accuracy. Such attributes showcase the network's practical applicability in real-time systems where speed and memory resources are constrained.

Future Directions and Implications

The research opens avenues for further exploration into enhancing the scalability of 3D deep learning models in environments with dynamic, complex motions. Future work could integrate more complex sensor data or consider transfer learning techniques to adapt to evolving sensor technologies and dynamic scenes.

The developments in HPLFlowNet underscore the growing potential and applicability of hierarchical, lattice-based deep networks across various domains spanning autonomous systems, robotics, and AR/VR technologies, where precise and efficient motion and depth analyses are critical.