Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Homography Estimation for Visual Place Recognition (2402.16086v2)

Published 25 Feb 2024 in cs.CV and cs.AI

Abstract: Visual place recognition (VPR) is a fundamental task for many applications such as robot localization and augmented reality. Recently, the hierarchical VPR methods have received considerable attention due to the trade-off between accuracy and efficiency. They usually first use global features to retrieve the candidate images, then verify the spatial consistency of matched local features for re-ranking. However, the latter typically relies on the RANSAC algorithm for fitting homography, which is time-consuming and non-differentiable. This makes existing methods compromise to train the network only in global feature extraction. Here, we propose a transformer-based deep homography estimation (DHE) network that takes the dense feature map extracted by a backbone network as input and fits homography for fast and learnable geometric verification. Moreover, we design a re-projection error of inliers loss to train the DHE network without additional homography labels, which can also be jointly trained with the backbone network to help it extract the features that are more suitable for local matching. Extensive experiments on benchmark datasets show that our method can outperform several state-of-the-art methods. And it is more than one order of magnitude faster than the mainstream hierarchical VPR methods using RANSAC. The code is released at https://github.com/Lu-Feng/DHE-VPR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Ali-bey, A.; et al. 2022. Gsv-cities: Toward appropriate supervised visual place recognition. Neurocomputing, 513: 194–203.
  2. Fast and Incremental Method for Loop-Closure Detection Using Bags of Visual Words. IEEE Transactions on Robotics, 24(5): 1027–1037.
  3. NetVLAD: CNN architecture for weakly supervised place recognition. In CVPR, 5297–5307.
  4. Speeded-up robust features (SURF). Computer vision and image understanding, 110(3): 346–359.
  5. Rethinking visual geo-localization for large-scale applications. In CVPR, 4878–4888.
  6. Viewpoint invariant dense matching for visual geolocalization. In ICCV, 12169–12178.
  7. Deep visual geo-localization benchmark. In CVPR, 5396–5407.
  8. Dsac-differentiable ransac for camera localization. In CVPR, 6684–6692.
  9. Brachmann, E.; et al. 2019. Neural-guided RANSAC: Learning where to sample model hypotheses. In ICCV, 4322–4331.
  10. Brachmann, E.; et al. 2021. Visual camera re-localization from RGB and RGB-D images using DSAC. IEEE transactions on pattern analysis and machine intelligence, 44(9): 5847–5865.
  11. Camara, L. G.; et al. 2020. Highly Robust Visual Place Recognition Through Spatial Matching of CNN Features. In ICRA, 3748–3755.
  12. Cao, B.; et al. 2020. Unifying deep local and global features for image search. In ECCV, 726–743. Springer.
  13. Deep learning features at scale for visual place recognition. In ICRA, 3223–3230.
  14. Only look once, mining distinctive landmarks from convnet for visual place recognition. In IROS, 9–16.
  15. ImageNet: A large-scale hierarchical image database. In CVPR, 248–255.
  16. DeTone, D.; et al. 2016. Deep image homography estimation. arXiv preprint arXiv:1606.03798.
  17. DeTone, D.; et al. 2018. Superpoint: Self-supervised interest point detection and description. In CVPR Workshops, 224–236.
  18. Scalable place recognition under appearance change for autonomous driving. In ICCV, 9319–9328.
  19. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR.
  20. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6): 381–395.
  21. Long-term loop closure detection through visual-spatial information preserving multi-order graph matching. In AAAI, volume 34, 10369–10376.
  22. Improving condition-and environment-invariant place recognition with semantic place categorization. In IROS, 6863–6870.
  23. SeqNet: Learning descriptors for sequence-based hierarchical place recognition. IEEE Robotics and Automation Letters, 6(3): 4305–4312.
  24. Garg, S.; et al. 2018. Don’t look back: Robustifying place categorization for viewpoint-and condition-invariant place recognition. In ICRA, 3645–3652.
  25. Self-supervising fine-grained region similarities for large-scale image localization. In ECCV, 369–386. Springer.
  26. Visual place recognition using HMM sequence matching. In IROS, 4549–4555.
  27. Hartley, R.; et al. 2003. Multiple view geometry in computer vision. Cambridge university press.
  28. Escaping the big data paradigm with compact transformers. arXiv preprint arXiv:2104.05704.
  29. Patch-netvlad: Multi-scale fusion of locally-global descriptors for place recognition. In CVPR, 14141–14152.
  30. Hierarchical multi-process fusion for visual place recognition. In ICRA, 3327–3333. IEEE.
  31. BoCNF: efficient image matching with Bag of ConvNet features for scalable and robust visual place recognition. Autonomous Robots, 42(6): 1169–1185.
  32. Learned contextual feature reweighting for image geo-localization. In CVPR, 2136–2145.
  33. Aggregating local descriptors into a compact image representation. In CVPR, 3304–3311.
  34. A Holistic Visual Place Recognition Approach Using Lightweight CNNs for Significant ViewPoint and Appearance Changes. IEEE Transactions on Robotics, 36(2): 561–569.
  35. Koguciuk, D.; et al. 2021. Perceptual loss for robust unsupervised homography estimation. In CVPR, 4274–4283.
  36. Deep homography estimation for dynamic scenes. In CVPR, 7652–7661.
  37. Densernet: Weakly supervised visual localization using multi-scale feature aggregation. In AAAI, volume 35, 6101–6109.
  38. Stochastic attraction-repulsion embedding for large scale image localization. In ICCV, 2570–2579.
  39. Lightweight, Viewpoint-Invariant Visual Place Recognition in Changing Environments. IEEE Robotics and Automation Letters, 3(2): 957–964.
  40. Visual place recognition: A survey. IEEE Transactions on Robotics, 32(1): 1–19.
  41. STA-VPR: Spatio-temporal alignment for visual place recognition. IEEE Robotics and Automation Letters, 6(3): 4297–4304.
  42. AANet: Aggregation and Alignment Network with Semi-hard Positive Sample Mining for Hierarchical Place Recognition. In ICRA, 11771–11778. IEEE.
  43. SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights. In ICRA, 1643–1649. IEEE.
  44. Semantics-aware visual localization under challenging perceptual conditions. In ICRA, 2614–2620.
  45. Robust visual robot localization across seasons using network flows. In AAAI, volume 28.
  46. Unsupervised deep homography: A fast and robust homography estimation model. IEEE Robotics and Automation Letters, 3(3): 2346–2353.
  47. Olid, D.; et al. 2018. Single-view place recognition under seasonal changes. arXiv preprint arXiv:1808.06516.
  48. Attentional pyramid pooling of salient visual residuals for place recognition. In ICCV, 885–894.
  49. Fine-tuning CNN image retrieval with no human annotation. IEEE transactions on pattern analysis and machine intelligence, 41(7): 1655–1668.
  50. Superglue: Learning feature matching with graph neural networks. In CVPR, 4938–4947.
  51. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
  52. Place Recognition With ConvNet Landmarks: Viewpoint-Robust, Condition-Robust, Training-Free. In RSS.
  53. Visual place recognition with repetitive structures. In CVPR, 883–890.
  54. TransVPR: Transformer-based place recognition with multi-level attention aggregation. In CVPR, 13648–13657.
  55. Mapillary street-level sequences: A dataset for lifelong place recognition. In CVPR, 2626–2635.
  56. Localizing Discriminative Visual Landmarks for Place Recognition. In ICRA, 5979–5985.
  57. A multi-domain feature learning method for visual place recognition. In ICRA, 319–324.
  58. Spatial pyramid-enhanced NetVLAD with weighted triplet loss for place recognition. IEEE transactions on neural networks and learning systems, 31(2): 661–674.
  59. ETR: An Efficient Transformer for Re-ranking in Visual Place Recognition. In WACV, 5665–5674.
  60. Content-aware unsupervised deep homography estimation. In ECCV 2020, 653–669. Springer.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Feng Lu (85 papers)
  2. Shuting Dong (7 papers)
  3. Lijun Zhang (239 papers)
  4. Bingxi Liu (10 papers)
  5. Xiangyuan Lan (25 papers)
  6. Dongmei Jiang (31 papers)
  7. Chun Yuan (127 papers)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets