DKM: Dense Kernelized Feature Matching for Geometry Estimation (2202.00667v3)

Abstract: Feature matching is a challenging computer vision task that involves finding correspondences between two images of a 3D scene. In this paper we consider the dense approach instead of the more common sparse paradigm, thus striving to find all correspondences. Perhaps counter-intuitively, dense methods have previously shown inferior performance to their sparse and semi-sparse counterparts for estimation of two-view geometry. This changes with our novel dense method, which outperforms both dense and sparse methods on geometry estimation. The novelty is threefold: First, we propose a kernel regression global matcher. Secondly, we propose warp refinement through stacked feature maps and depthwise convolution kernels. Thirdly, we propose learning dense confidence through consistent depth and a balanced sampling approach for dense confidence maps. Through extensive experiments we confirm that our proposed dense method, \textbf{D}ense \textbf{K}ernelized Feature \textbf{M}atching, sets a new state-of-the-art on multiple geometry estimation benchmarks. In particular, we achieve an improvement on MegaDepth-1500 of +4.9 and +8.9 AUC$@5^{\circ}$ compared to the best previous sparse method and dense method respectively. Our code is provided at https://github.com/Parskatt/dkm

• The paper presents a novel dense matching methodology using Gaussian Process-based global matching that achieves significant performance gains on geometry benchmarks.
• It employs a warp refinement technique with depthwise separable convolutions to iteratively enhance the precision of initial coarse matches.
• The method integrates dense confidence learning with balanced sampling to reliably improve pose estimation accuracy across diverse benchmark datasets.

DKM: Dense Kernelized Feature Matching for Geometry Estimation

The presented paper introduces a novel approach to feature matching in computer vision, titled Dense Kernelized Feature Matching (DKM). Unlike traditional methods that rely on sparse feature matching, DKM emphasizes a dense matching paradigm, aiming to identify all correspondences between two images. This dense approach confronts the historical challenge where dense methods underperformed compared to sparse counterparts in geometry estimation.

Key Novel Contributions

DKM introduces three primary innovations:

1. Kernelized Global Matcher: The authors propose a Gaussian Process (GP) formulation for global matching, treating it as an embedded coordinate regression problem. This approach leverages a cosine similarity kernel to produce robust coarse matches, departing from the traditionally used dense correlation volumes.
2. Warp Refinement Technique: The method utilizes warp refinement through depthwise separable convolution kernels, fed by stacked feature maps. These refiners enhance the precision of the initial coarse warp, a significant departure from previous methods that relied on dense correlation-based refinement.
3. Dense Confidence Learning: By leveraging consistent depth in the 3D scene, DKM predicts a dense confidence map for matches. Through balanced sampling, it ensures reliability in downstream tasks, such as estimating geometry, by choosing diverse and accurate matches, rather than solely relying on certainty thresholds.

Experimental Evaluation and Results

The effectiveness of DKM is established through extensive experiments on several benchmarks. Notably, DKM sets a new state-of-the-art in multiple geometry estimation benchmarks:

• MegaDepth-1500 Benchmark: Achieves a remarkable improvement of +4.9 and +8.9 AUC$@5^{\circ}$ over the best existing sparse and dense methods, respectively.
• HPatches Homography: Demonstrates superiority with significant gains in matching accuracy for planar scenes.
• ScanNet-1500 Pose Estimation: Outperforms prior works in difficult indoor settings characterized by low-texture regions.

Architectural Insights and Methodological Innovations

The proposed approach for DKM provides significant insights:

• Global Matching as Regression: By treating global matching as a GP regression problem, DKM effectively manages multimodality issues inherent in dense matching, using innovative coordinate embeddings.
• Refinement Strategy: The use of large depthwise kernels combined with stacked feature maps enhances the capacity to improve upon initial coarse matches. This architectural choice shows clear performance benefits as validated by ablation studies.
• Certainty and Sampling Approaches: The method’s dense confidence maps, learned through depth consistency, coupled with a novel balanced match sampling strategy, directly contribute to improved pose estimation outcomes.

Implications and Future Directions

DKM's advances in dense feature matching have notable implications for 3D reconstruction, SLAM, and visual localization, potentially improving the precision and robustness in these applications.

Future directions could explore enhancements in handling depth discontinuities and refining confidence predictions for small or challenging texture features. Additionally, integrating uncertainty modeling directly could further enhance performance, especially in ambiguous regions.

In conclusion, DKM represents a substantial progression in feature matching methodologies, encouraging the adoption of dense strategies in complex geometry estimation tasks. The work not only addresses previous limitations of dense matching but also sets a new benchmark for future investigations in the field.