Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

VPRTempo: A Fast Temporally Encoded Spiking Neural Network for Visual Place Recognition (2309.10225v2)

Published 19 Sep 2023 in cs.RO

Abstract: Spiking Neural Networks (SNNs) are at the forefront of neuromorphic computing thanks to their potential energy-efficiency, low latencies, and capacity for continual learning. While these capabilities are well suited for robotics tasks, SNNs have seen limited adaptation in this field thus far. This work introduces a SNN for Visual Place Recognition (VPR) that is both trainable within minutes and queryable in milliseconds, making it well suited for deployment on compute-constrained robotic systems. Our proposed system, VPRTempo, overcomes slow training and inference times using an abstracted SNN that trades biological realism for efficiency. VPRTempo employs a temporal code that determines the timing of a single spike based on a pixel's intensity, as opposed to prior SNNs relying on rate coding that determined the number of spikes; improving spike efficiency by over 100%. VPRTempo is trained using Spike-Timing Dependent Plasticity and a supervised delta learning rule enforcing that each output spiking neuron responds to just a single place. We evaluate our system on the Nordland and Oxford RobotCar benchmark localization datasets, which include up to 27k places. We found that VPRTempo's accuracy is comparable to prior SNNs and the popular NetVLAD place recognition algorithm, while being several orders of magnitude faster and suitable for real-time deployment -- with inference speeds over 50 Hz on CPU. VPRTempo could be integrated as a loop closure component for online SLAM on resource-constrained systems such as space and underwater robots.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Spiking neural networks and their applications: A review. Brain Sciences, 12(7), 2022.
  2. Spinnaker: A multi-core system-on-chip for massively-parallel neural net simulation. In IEEE Custom Integrated Circuits Conference, 2012.
  3. Efficient neuromorphic signal processing with Loihi 2. arXiv 2111.03746, 2021.
  4. A cerebellar-based solution to the nondeterministic time delay problem in robotic control. Science Robotics, 6(58), 2021.
  5. Event-driven vision and control for UAVs on a neuromorphic chip. arXiv 2108.03694, 2021.
  6. Towards grasping with spiking neural networks for anthropomorphic robot hands. Artificial Neural Networks and Machine Learning, 2017.
  7. Controlling a robot arm for target reaching without planning using spiking neurons. IEEE International Conference on Cognitive Informatics & Cognitive Computing, 2018.
  8. Ensembles of compact, region-specific & regularized spiking neural networks for scalable place recognition. In IEEE International Conference on Robotics and Automation, 2023.
  9. Spiking neural networks for visual place recognition via weighted neuronal assignments. IEEE Robotics and Automation Letters, 7(2), 2022.
  10. Visual place recognition: A tutorial. IEEE Robotics and Automation Magazine, 2023.
  11. NetVLAD: CNN architecture for weakly supervised place recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2016.
  12. Brain-inspired multimodal hybrid neural network for robot place recognition. Science Robotics, 8(78), 2023.
  13. 24/7 place recognition by view synthesis. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2015.
  14. Data-efficient large scale place recognition with graded similarity supervision. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  15. MixVPR: Feature mixing for visual place recognition. IEEE/CVF Winter Conference on Applications of Computer Vision, 2023.
  16. Leveraging deep visual descriptors for hierarchical efficient localization. arXiv 1809.01019, 2018.
  17. H. Durrant-Whyte and T. Bailey. Simultaneous localization and mapping: part I. IEEE Robotics & Automation Magazine, 13(2), 2006.
  18. Rafiqul Islam and H. Habibullah. A semantically aware place recognition system for loop closure of a visual SLAM system. International Conference on Mechatronics, Robotics and Automation, 2021.
  19. Real-time visual place recognition based on analyzing distribution of multi-scale CNN landmarks. Journal of Intelligent Robot Systems, 94, 2019.
  20. Online visual place recognition via saliency re-identification. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020.
  21. VPR-Bench: An open-source visual place recognition evaluation framework with quantifiable viewpoint and appearance change. International Journal of Computer Vision, 129(7), 2021.
  22. A survey on deep visual place recognition. IEEE Access, 9, 2021.
  23. Visual place recognition: A survey. IEEE Transactions on Robotics, 32(1), 2016.
  24. Navvab Kashiri et al. An overview on principles for energy efficient robot locomotion. Frontiers in Robotics and AI, 5, 2018.
  25. Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE, 111(9), 2023.
  26. Neural coding in spiking neural networks: A comparative study for robust neuromorphic systems. Frontiers in Neuroscience, 15, 2021.
  27. Ingvars Birznieks Roland S Johansson. First spikes in ensembles of human tactile afferents code complex spatial fingertip events. Nature Neuroscience, 7, 2004.
  28. Wolf Singer. Distributed processing and temporal codes in neuronal networks. Cognitive Neurodynamics, 3, 2009.
  29. Making a spiking net work: Robust brain-like unsupervised machine learning. arXiv 2208.01204, 2022.
  30. Modular neural networks: A survey. International Journal of Neural Systems, 09(02), 1999.
  31. Design and evolution of modular neural network architectures. Neural Networks, 7(6), 1994. Models of Neurodynamics and Behavior.
  32. Adaptive mixtures of local experts. Neural Computation, 3(1), 1991.
  33. SeqSLAM: visual route-based navigation for sunny summer days and stormy winter nights. In IEEE International Conference on Robotics and Automation, 2012.
  34. Patch-NetVLAD: Multi-scale fusion of locally-global descriptors for place recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
  35. Stochastic attraction-repulsion embedding for large scale image localization. In IEEE International Conference on Computer Vision, 2019.
  36. Real-time visual place recognition for personal localization on a mobile device. Wireless Personal Communications, 97(1), 2017.
  37. Single-view place recognition under seasonal changes. PPNIV Workshop at International Conference on Intelligent Robots and Systems, 2018.
  38. 1 year, 1000 km: The oxford robotcar dataset. The International Journal of Robotics Research, 36(1), 2017.
  39. Where is your place, visual place recognition? In International Joint Conference on Artificial Intelligence, 2021.
  40. Unsupervised learning methods for visual place recognition in discretely and continuously changing environments. In IEEE International Conference on Robotics and Automation, 2020.
  41. The revisiting problem in simultaneous localization and mapping: A survey on visual loop closure detection. IEEE Transactions on Intelligent Transportation Systems, 23(11), 2022.
  42. Jing Pei et al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature, 572(7767), 2019.
  43. Filipp Akopyan et al. TrueNorth: design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 34(10), 2015.
  44. An on-chip spiking neural network for estimation of the head pose of the icub robot. Frontiers in Neuroscience, 14, 2020.
  45. An end-to-end spiking neural network platform for edge robotics: From event-cameras to central pattern generation. IEEE Transactions on Cognitive and Developmental Systems, 14(3), 2022.
  46. Pose estimation and map formation with spiking neural networks: towards neuromorphic SLAM. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018.
  47. Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020.
  48. Spiking neural network on neuromorphic hardware for energy-efficient unidimensional SLAM. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2019.
  49. A neuromorphic controller for a robotic vehicle equipped with a dynamic vision sensor. Robotics Science and Systems, 2017.
  50. Live demo: Spiking ratSLAM: Rat hippocampus cells in spiking neural hardware. IEEE Biomedical Circuits and Systems Conference, 2012.
  51. Spiking neural network-based multi-task autonomous learning for mobile robots. Engineering Applications of Artificial Intelligence, 104, 2021.
  52. Sailboat navigation control system based on spiking neural networks. Control Theory and Technology, 2023.
  53. Spiking autoencoders with temporal coding. Frontiers in Neuroscience, 15, 2021.
  54. A survey of encoding techniques for signal processing in spiking neural networks. Neural Processing Letters, 53(6), 2021.
  55. Temporal pattern coding in deep spiking neural networks. In International Joint Conference on Neural Networks, 2021.
  56. Guo qiang Bi and Mu ming Poo. Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18(24), 1998.
  57. Deep spiking neural network with neural oscillation and spike-phase information. In AAAI Conference on Artificial Intelligence, volume 35, 2021.
  58. Eugene M. Izhikevich. Resonate-and-fire neurons. Neural Networks, 14(6), 2001.
  59. Deep spiking neural networks with resonate-and-fire neurons. arXiv 2109.08234, 2021.
  60. Adam Paszke et al. PyTorch: an imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32, 2019.
  61. Learning representations by back-propagating errors. Nature, 323(6088), 1986.
  62. The quest for the hippocampal memory engram: From theories to experimental evidence. Frontiers in Behavioral Neuroscience, 14, 2021.
  63. Intelligent reference curation for visual place recognition via bayesian selective fusion. IEEE Robotics and Automation Letters, 6(2), 2021.
  64. Rethinking visual geo-localization for large-scale applications. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  65. Multi-process fusion: Visual place recognition using multiple image processing methods. IEEE Robotics and Automation Letters, 4(2), 2019.
  66. Brian 2, an intuitive and efficient neural simulator. eLife, 8, 2019.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube