RIPE++: Reinforced Keypoint Learning from Positive Pairs Only

Abstract: Sparse keypoint extraction and matching underpin core tasks in geometric computer vision, including structure-from-motion, visual SLAM, augmented reality, and medical image registration. Learning robust local feature representations, however, typically requires accurate camera poses or depth supervision, which are often unavailable in real-world settings. Reinforcement learning (RL) has recently emerged as a promising alternative, requiring only the information if two images show the same scene or not. However, existing RL formulations such as RIPE rely on coarse binary rewards and carefully constructed negative training pairs, limiting training stability and descriptor discriminability. In this paper, we revisit RL-based keypoint learning and propose a reward that fully exploits the geometric consistency signal, deriving both reward and penalty from a single positive pair without contrasting against negatives. This richer signal provides sufficient supervisory contrast to learn discriminative detectors and descriptors from positive image pairs alone, enabling representation learning under extremely limited supervision. Furthermore, we show that the same RL objective can be extended to the matching stage by adapting LightGlue, raising AUC@5 on MegaDepth1500 from 56.58 to 59.65 and enabling weakly-supervised training of the full sparse matching pipeline from image pairs with partial visual overlap. We validate our approach on established benchmarks, demonstrating competitive results compared to fully-supervised methods. We further show that the method can be even trained on low texture medical video sequences, where camera poses are usually unavailable and standard SfM pipelines often fail. Code and data are available at https://github.com/fraunhoferhhi/RIPEpp .

• The paper introduces correspondence-level rewards that reinforce RANSAC inliers and penalize outliers within positive image pairs, eliminating negative examples while improving MegaDepth1500 AUC@5° by 3.11 percentage points over RIPE.
• The method combines entropy-regularized keypoint sampling with weakly supervised descriptor and LightGlue training, reducing training time from 72 to 26 hours and raising matcher AUC@5° from 56.58 to 59.65.
• The approach enables affordable domain adaptation from raw medical video, achieving 20.90/46.51/68.72 AUC at 5°/10°/20° on SCARED1500 and reducing false nearest-neighbor matches by 8.4% without negative pairs.

Motivation and problem setting

Sparse keypoint extraction and matching underpin structure-from-motion (SfM), visual SLAM, augmented reality, and medical image registration, yet learned pipelines such as SuperPoint, ALIKED, DeDoDe, and DaD depend on geometric ground truth — camera poses or depth derived from offline SfM reconstructions. The authors argue that supervision availability, not model capacity, is now the binding constraint in keypoint learning. RIPE (Künzel et al., 7 Jul 2025) reduced the annotation burden to a single bit per image pair by using RANSAC-based fundamental matrix estimation as a reinforcement learning (RL) reward, but it retained two weaknesses: a coarse binary reward that requires carefully curated negative pairs, and reliance on a matcher trained with full pose or depth supervision.

RIPE++ addresses both limitations. Its central claim is that the effectiveness of weakly-supervised RL depends on how geometric consistency is translated into a reward, and that a correspondence-level reward on positive pairs alone provides sufficient contrast to learn discriminative detectors, descriptors, and even matchers without any negative examples.

Method

Correspondence-level reward from positive pairs only

The framework follows RIPE: a VGG-19-based network predicts heatmaps divided into cells of 8 pixels; one keypoint location is sampled per cell from a categorical distribution, with an acceptance indicator $acc = \text{Sigmoid}(z)$ rejecting unreliable locations. Descriptors are hypercolumn features of the encoder, and gradients are estimated with REINFORCE over the combined log-probability matrix $\mathbf{L}$.

The key change is in the reward matrix. RIPE assigned rewards at the pair level: inliers were rewarded for positive pairs and penalized for negative pairs, while outliers were ignored. This had two consequences: mislabeled negatives destabilize training, and — more fundamentally — a network producing many false matches filtered out by RANSAC received a neutral gradient signal. RIPE++ instead assigns, within each positive pair, a reward $\rho_{\text{in}}$ to every RANSAC-verified inlier and a penalty $\rho_{\text{out}}$ to every outlier, with a negligible $\lambda$ for unmatched keypoints. This finer-grained signal makes negative pairs unnecessary, simplifies dataset curation (raw video streams suffice), and reduces training time from 72 to 26 hours on a single A100, since RANSAC can terminate early on positive pairs but must exhaust iterations on negatives.

Entropy regularization

RIPE's low-probability regularizer only penalizes the selected keypoint's probability, implicitly shaping the surrounding distribution. RIPE++ replaces it with the negative entropy of the per-cell distribution, which produces gradients for all positions and explicitly drives each cell toward a one-hot encoding. This improves heatmap sharpness particularly at low input resolutions, though the ablations show it is sensitive to weighting: $\omega = 10^{-5}$ degrades performance and $\omega = 10^{-4}$ collapses training entirely, so careful tuning is required.

Weakly-supervised LightGlue

The same objective is extended to the matching stage by adapting LightGlue via a DISK-style policy gradient. Because LightGlue's soft partial assignment decomposes into a matchability term and a bidirectional softmax match term, both computable in closed form, the expected-reward gradient can be evaluated exactly without sampling; variance arises only from the empirical expectation over feature sets. Inliers receive reward $\nu_{\text{in}}$, outliers $\nu_{\text{out}}$, and a non-matchable regularization term prevents the degenerate solution of labeling every keypoint unmatchable. This removes the last dependence on fully-supervised matching components in the pipeline.

Results

Relative pose estimation (MegaDepth1500). With mutual-nearest-neighbor matching and 2048 keypoints, RIPE++ reaches AUC@5°/10°/20° of 56.58 / 69.53 / 79.33, improving over RIPE by 3.11 pp AUC@5° while discarding its negative pairs. Notably, this places it essentially at parity with strongly-supervised extractors: it trails ALIKED by only 0.07 pp on average and outperforms DeDoDe-B and DaD, both trained with pose or depth supervision. RaCo reports higher absolute numbers, but only its detector is weakly supervised — its descriptor is the fully-supervised ALIKED — whereas RIPE++ trains both detector and descriptor from image pairs alone.

SCARED1500 benchmark. The authors introduce SCARED1500, derived from endoscopic video recorded with a da Vinci Xi robot, using provided poses only for evaluation. Zero-shot, all learned methods collapse on this domain. Retrained on raw video frames (17,514 pairs formed by a fixed frame offset plus random affine augmentation), RIPE++ Medical achieves AUC@5°/10°/20° of 20.90 / 46.51 / 68.72, outperforming every baseline including SuperPoint, RaCo, and DeDoDe-B. The operative advantage is not zero-shot transfer but cheap retraining wherever geometric ground truth is unavailable — something no pose- or depth-supervised method can do here, and RaCo cannot fully follow because its descriptor is frozen supervised ALIKED.

Weakly-supervised matcher. Training LightGlue with the proposed policy-gradient objective raises MegaDepth1500 AUC@5° from 56.58 to 59.65 (+3.07 pp), with +4.16 pp at 10° and +4.53 pp at 20°. Absolute accuracy remains below fully-supervised LightGlue variants (66.1% AUC@5° when paired with ALIKED), which the authors state plainly rather than claiming parity.

Aachen Day-Night v1.1. RIPE++ consistently improves over RIPE, with gains concentrated at night (+9.5 pp at 0.25 m/2°). Substituting 20% of training data with Tokyo 24/7 day/night pairs trades a small daytime degradation (-2.5 pp) for clear nighttime gains (+5.7 pp), confirming that the weakly-supervised scheme still absorbs additional unlabeled data easily.

Negative-pair analysis. An evaluation on 1500 cross-scene negative pairs shows that removing negatives does not increase spurious correspondences; false nearest-neighbor matches actually decrease by 8.4%, directly refuting the assumption that explicit negatives are needed to suppress false matches.

Ablations

Positive-only training alone improves AUC@5° from 51.83 to 52.42 relative to the RIPE-style baseline. Entropy regularization contributes the largest single gain (over 4 pp AUC@5° at $\omega = 10^{-6}$). Supplementary ablations show that replacing the contrastive descriptor loss with InfoNCE increases RANSAC inlier counts but does not translate into better pose accuracy; curriculum learning and a Sampson-distance-based continuous reward each help individually, but their combination yields no cumulative improvement. The best configuration uses positive-only training, entropy regularization, and the original contrastive descriptor loss.

Limitations and open questions

Several constraints are acknowledged or evident. The method assumes rigid scenes and pinhole cameras through the fundamental-matrix reward, excluding non-rigid deformation handling of the kind DEAL addresses. Entropy regularization is fragile to its weighting coefficient, with outright training collapse at higher values. The matcher result remains clearly below fully-supervised LightGlue, and the two-stage synthetic-pretraining protocol was retained rather than eliminated. On SCARED1500, the simple fixed-offset pairing strategy can produce degenerate pairs with negligible motion or no overlap; robustness to this noise is asserted empirically but not analyzed theoretically. Finally, whether the closed-form policy gradient for the matcher can close the remaining gap to depth-supervised training, or whether the reward formulation extends to non-rigid or non-pinhole settings, remains open.

Conclusion

RIPE++ demonstrates that a correspondence-level geometric reward — rewarding inliers and penalizing outliers within positive pairs — supersedes binary pair-level rewards in RL-based keypoint learning. It removes the need for negative pairs, matches fully-supervised extractors on MegaDepth1500 despite using neither pose nor depth, enables domain-specific training from raw medical video where standard SfM fails, and extends weakly-supervised RL training to transformer-based matching. The result establishes positive-pair-only RL as a viable recipe for learning sparse matching pipelines under minimal supervision and data curation.