DeDoDe v2: Analyzing and Improving the DeDoDe Keypoint Detector (2404.08928v1)

Abstract: In this paper, we analyze and improve into the recently proposed DeDoDe keypoint detector. We focus our analysis on some key issues. First, we find that DeDoDe keypoints tend to cluster together, which we fix by performing non-max suppression on the target distribution of the detector during training. Second, we address issues related to data augmentation. In particular, the DeDoDe detector is sensitive to large rotations. We fix this by including 90-degree rotations as well as horizontal flips. Finally, the decoupled nature of the DeDoDe detector makes evaluation of downstream usefulness problematic. We fix this by matching the keypoints with a pretrained dense matcher (RoMa) and evaluating two-view pose estimates. We find that the original long training is detrimental to performance, and therefore propose a much shorter training schedule. We integrate all these improvements into our proposed detector DeDoDe v2 and evaluate it with the original DeDoDe descriptor on the MegaDepth-1500 and IMC2022 benchmarks. Our proposed detector significantly increases pose estimation results, notably from 75.9 to 78.3 mAA on the IMC2022 challenge. Code and weights are available at https://github.com/Parskatt/DeDoDe

• The paper introduces DeDoDe v2, which incorporates training-time non-maximum suppression to mitigate keypoint clustering.
• It leverages optimized data augmentation with 90° rotations and horizontal flips to boost the detector’s rotational robustness.
• The evaluation using a pretrained dense matcher for two-view pose estimation shows that shorter training times can yield superior performance.

Enhancements and Evaluation of the DeDoDe Keypoint Detector

Introduction to Keypoint Detection in Computer Vision

Keypoint detection is a critical process in computer vision, facilitating the identification of distinctive local features in images that are robust to variations in viewpoint and illumination. Building upon the foundation laid by the DeDoDe detector, this paper introduces an improved version dubbed DeDoDe v2. Despite the established utility of DeDoDe in extracting keypoints, it exhibited limitations such as keypoint clustering and sensitivity to large rotations. This paper details the analytical and experimental endeavors undertaken to address these issues, culminating in the development of DeDoDe v2. The enhancements include the incorporation of non-maximum suppression during training, refined data augmentation techniques, and a novel evaluation methodology leveraging a pretrained dense matcher for two-view pose estimation. Notably, the paper presents a streamlined training regime that significantly reduces computational requirements without sacrificing, and indeed improving, the detector's performance.

Key Contributions

• Refinements in Training Methodology: The introduction of non-maximum suppression into the training process addresses the clustering issue observed with the original DeDoDe detector, fostering the generation of more evenly distributed and diverse keypoints across images.
• Optimized Data Augmentation: By integrating 90-degree rotations and horizontal flips into the data augmentation strategy, the sensitivity of the DeDoDe detector to large in-plane rotations is mitigated, enhancing the rotational robustness of the keypoints detected.
• Evaluation Strategy: Evaluation of the keypoint detector's effectiveness for downstream tasks, particularly two-view pose estimation, is refined through the use of a pretrained dense matcher (RoMa). This approach allows for a more comprehensive assessment than the repeatability metric alone.
• Reduced Training Time: The paper outlines a significant reduction in training duration, from the protracted schedule of the original to approximately 20 minutes on a single A100 GPU, without compromising the quality of the generated keypoints.

Analysis and Improvements

The paper methodically addresses the deficiencies of the original DeDoDe detector through a combination of theoretical insights and empirical testing. For instance, the introduction of training-time non-maximum suppression effectively disperses the clustering of keypoints, a problem that previously led to diminished performance in downstream applications. Additionally, the adoption of targeted data augmentations ensures the detector's robustness to rotational variations in input imagery.

One of the standout revelations from this research is the counterproductive nature of prolonged training for keypoint detectors. By correlating training duration with performance metrics on two-view pose estimation, the authors adeptly demonstrate that shorter training periods can yield superior outcomes. This insight not only enhances the efficacy of the DeDoDe v2 detector but also significantly lowers the computational cost associated with its training.

Future Directions and Limitations

While DeDoDe v2 marks a considerable advancement in keypoint detection, the authors acknowledge the ongoing challenges in fully reconciling the objectives of high repeatability with the demands of downstream tasks like pose estimation. The exploration of detector objectives that harmonize with these tasks presents a fertile avenue for future research. Moreover, the paper's findings underscore the necessity of both theoretical and empirical approaches to improve the understanding and development of keypoint detectors in computer vision.

Conclusion

In summation, the development of DeDoDe v2 represents a significant step forward in the field of keypoint detection. By addressing the limitations observed in the original DeDoDe detector, the authors have not only enhanced its performance but also shed light on previously underexplored aspects of keypoint detector training and evaluation. The practical and theoretical insights offered by this paper are poised to inform and inspire future research in the field.