Hardware-Algorithm Re-engineering of Retinal Circuit for Intelligent Object Motion Segmentation
Abstract: Recent advances in retinal neuroscience have fueled various hardware and algorithmic efforts to develop retina-inspired solutions for computer vision tasks. In this work, we focus on a fundamental visual feature within the mammalian retina, Object Motion Sensitivity (OMS). Using DVS data from EV-IMO dataset, we analyze the performance of an algorithmic implementation of OMS circuitry for motion segmentation in presence of ego-motion. This holistic analysis considers the underlying constraints arising from the hardware circuit implementation. We present novel CMOS circuits that implement OMS functionality inside image sensors, while providing run-time re-configurability for key algorithmic parameters. In-sensor technologies for dynamical environment adaptation are crucial for ensuring high system performance. Finally, we verify the functionality and re-configurability of the proposed CMOS circuit designs through Cadence simulations in 180nm technology. In summary, the presented work lays foundation for hardware-algorithm re-engineering of known biological circuits to suit application needs.
- E. Sernagor, S. J. Eglen, and R. O. L. Wong, “Development of retinal ganglion cell structure and function,” Progress in Retinal and Eye Research, vol. 20, pp. 139–174, 2001. [Online]. Available: https://api.semanticscholar.org/CorpusID:7422356
- G. W. Schwartz and D. Swygart, “Object motion sensitivity,” in Retinal Computation. Elsevier, 2021, pp. 230–244.
- R. Benosman, C. Clercq, X. Lagorce, S.-H. Ieng, and C. Bartolozzi, “Event-based visual flow,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 2, pp. 407–417, 2014.
- X. Wang, J. Li, L. Zhu, Z. Zhang, Z. Chen, X. Li, Y. Wang, Y. Tian, and F. Wu, “Visevent: Reliable object tracking via collaboration of frame and event flows,” CoRR, vol. abs/2108.05015, 2021. [Online]. Available: https://arxiv.org/abs/2108.05015
- V. Vasco, A. Glover, E. Mueggler, D. Scaramuzza, L. Natale, and C. Bartolozzi, “Independent motion detection with event-driven cameras,” in 2017 18th International Conference on Advanced Robotics (ICAR), 2017, pp. 530–536.
- S. Taylor, “Ccd and cmos imaging array technologies: Technology review,” 1999.
- G. Gallego et al., “Event-based vision: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 1, pp. 154–180, 2020.
- K. Zaghloul and K. Boahen, “A silicon retina that reproduces signals in the optic nerve,” Journal of neural engineering, vol. 3, pp. 257–67, 01 2007.
- C. A. Mead and M. Mahowald, “A silicon model of early visual processing,” Neural Networks, vol. 1, no. 1, pp. 91–97, 1988.
- K. A. Boahen and A. G. Andreou, “A contrast sensitive silicon retina with reciprocal synapses,” in Proceedings of the 4th International Conference on Neural Information Processing Systems, ser. NIPS’91. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1991, p. 764–770.
- Z. Yin, M. A.-A. Kaiser, L. O. Camara, M. Camarena, M. Parsa, A. Jacob, G. Schwartz, and A. Jaiswal, “Iris: Integrated retinal functionality in image sensors,” Frontiers in Neuroscience, vol. 17, 2023.
- J. B. Selhorst, “The Retina: An Approachable Part of the Brain,” JAMA, vol. 260, no. 12, pp. 1792–1793, 09 1988. [Online]. Available: https://doi.org/10.1001/jama.1988.03410120138048
- B. Son, Y. Suh, S. Kim, H. Jung, J.-S. Kim, C. Shin, K. Park, K. Lee, J. Park, J. Woo et al., “4.1 a 640×\times× 480 dynamic vision sensor with a 9μ𝜇\muitalic_μm pixel and 300meps address-event representation,” in 2017 IEEE International Solid-State Circuits Conference (ISSCC). IEEE, 2017, pp. 66–67.
- D. P. Moeys, F. Corradi, C. Li, S. A. Bamford, L. Longinotti, F. F. Voigt, S. Berry, G. Taverni, F. Helmchen, and T. Delbruck, “A sensitive dynamic and active pixel vision sensor for color or neural imaging applications,” IEEE transactions on biomedical circuits and systems, vol. 12, no. 1, pp. 123–136, 2017.
- S. Snyder, H. Thompson, M. A.-A. Kaiser, G. Schwartz, A. Jaiswal, and M. Parsa, “Object motion sensitivity: A bio-inspired solution to the ego-motion problem for event-based cameras,” 2023.
- A. Kumar and S. S. Sodhi, “Comparative analysis of gaussian filter, median filter and denoise autoenocoder,” in 2020 7th International Conference on Computing for Sustainable Global Development (INDIACom), 2020, pp. 45–51.
- M. Abdullah-Al Kaiser and A. R. Jaiswal, “Hardware-algorithm co-design enabling processing-in-pixel-in-memory (p 2 m) for neuromorphic vision sensors,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 13 356–13 360.
- M. A.-A. Kaiser, G. Datta, P. A. Beerel, and A. R. Jaiswal, “Toward high performance, programmable extreme-edge intelligence for neuromorphic vision sensors utilizing magnetic domain wall motion-based mtj,” arXiv preprint arXiv:2402.15121, 2024.
- A. Mitrokhin, C. Ye, C. Fermuller, Y. Aloimonos, and T. Delbruck, “Ev-imo: Motion segmentation dataset and learning pipeline for event cameras,” 2020.
- C. M. Parameshwara, N. J. Sanket, A. Gupta, C. Fermüller, and Y. Aloimonos, “MOMS with events: Multi-object motion segmentation with monocular event cameras,” CoRR, vol. abs/2006.06158, 2020. [Online]. Available: https://arxiv.org/abs/2006.06158
- R. Kubendran, A. Paul, and G. Cauwenberghs, “A 256x256 6.3 pj/pixel-event query-driven dynamic vision sensor with energy-conserving row-parallel event scanning,” in 2021 IEEE custom integrated circuits conference (CICC). IEEE, 2021, pp. 1–2.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
