FLOT: Scene Flow on Point Clouds Guided by Optimal Transport (2007.11142v1)

Abstract: We propose and study a method called FLOT that estimates scene flow on point clouds. We start the design of FLOT by noticing that scene flow estimation on point clouds reduces to estimating a permutation matrix in a perfect world. Inspired by recent works on graph matching, we build a method to find these correspondences by borrowing tools from optimal transport. Then, we relax the transport constraints to take into account real-world imperfections. The transport cost between two points is given by the pairwise similarity between deep features extracted by a neural network trained under full supervision using synthetic datasets. Our main finding is that FLOT can perform as well as the best existing methods on synthetic and real-world datasets while requiring much less parameters and without using multiscale analysis. Our second finding is that, on the training datasets considered, most of the performance can be explained by the learned transport cost. This yields a simpler method, FLOT$_0$, which is obtained using a particular choice of optimal transport parameters and performs nearly as well as FLOT.

• The paper introduces FLOT, which innovatively applies optimal transport to establish accurate point correspondences between consecutive point cloud frames.
• It employs deep feature extraction and a variant of the Sinkhorn algorithm to compute soft correspondences, significantly reducing model parameters and complexity.
• Experimental results on FlyingThings3D and KITTI demonstrate that FLOT matches or exceeds state-of-the-art performance while ensuring high computational efficiency.

An Overview of FLOT: Scene Flow on Point Clouds Guided by Optimal Transport

The estimation of scene flow, which represents the 3D motion vectors of all points in a scene, is a fundamental task in computer vision, particularly valuable for applications like autonomous driving. The paper under consideration introduces FLOT (Flow on point clouds guided by Optimal Transport), a novel method for scene flow estimation from point clouds. This work presents a sophisticated yet efficient approach leveraging optimal transport theory to deliver high performance with reduced complexity compared to traditional methods.

Methodology

FLOT addresses scene flow estimation by leveraging optimal transport to establish correspondences between two consecutive point cloud frames capturing a scene. This innovative approach treats the task as a matching problem, where optimal transport is utilized to align points between these frames. The process consists of:

1. Problem Formulation: Initially, the problem is idealized to an environment where point correspondences form a perfect bijection represented as a permutation matrix. Scene flow estimation then reduces to effectively identifying this matrix.
2. Optimal Transport Application: The methodology involves defining a cost matrix based on feature similarity for optimal transport. This matrix is derived using deep features extracted from a neural network trained with full supervision on synthetic datasets. The optimal transport plan, indicating soft-correspondences between frames, is computed via a relaxed version of the traditional transport problem, allowing for efficient computation using a variant of the Sinkhorn algorithm.
3. Flow Estimation: Following the determination of soft-correspondences, a preliminary scene flow is computed using barycentric coordinates. A residual network subsequently refines this initial estimate, enhancing prediction accuracy.
4. Reduced Parameters and Simplicity: FLOT achieves competitive performance while maintaining a lower parameter count and eschews complex multiscale analyses. This is achieved by focusing on a singular scale and leveraging a straightforward approach where most of the process's effectiveness is attributed to the learned transport cost.

Findings and Experimental Results

The research highlights several key outcomes through comprehensive evaluation on datasets such as FlyingThings3D and KITTI Scene Flow:

• Performance Efficiency: FLOT demonstrates matching or superior performance compared to state-of-the-art techniques on both synthetic and real-world datasets. This includes a remarkable capability to maintain high accuracy with significantly fewer computational resources and reduced model complexity.
• Effectiveness of Learned Transport Costs: The analysis reveals that the transport cost, ideally learned from synthetic datasets, underpins most of FLOT's performance achievements. This insight led to the development of a simplified variant, FLOT$_0$, which also performs competitively by utilizing specific optimal transport parameters.
• Computational Speed: The computational demands of the optimal transport module are minimal compared to the overall method, emphasizing the efficiency and scalability of FLOT even in resource-constrained environments.

Implications and Future Developments

The implications of this research are manifold, both for practical and theoretical advancements in scene understanding. By introducing a transport-based method that proficiently links theoretical optimal transport with scene flow estimation, this work paves the way for more efficient real-world applications, particularly in autonomous systems that require rapid processing times.

Moreover, the exploration of occlusions remains an area for further enhancement. Currently, FLOT handles occlusions indirectly through the relaxation of transport constraints, but explicit treatment of these phenomena could further elevate its robustness and applicability.

In the broader context of artificial intelligence, the integration of optimal transport with deep learning for correspondence tasks opens up potential research pathways in other domains such as image registration, object tracking, and beyond. Future work could explore these relationships, enhance model interpretability, and further refine computational strategies to adapt the methodology to various complex real-world scenarios.