Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Brain-Inspired AI with Hyperbolic Geometry (2409.12990v3)

Published 4 Sep 2024 in q-bio.NC and cs.AI

Abstract: Artificial neural networks (ANNs) were inspired by the architecture and functions of the human brain and have revolutionised the field of AI. Inspired by studies on the latent geometry of the brain, in this perspective paper we posit that an increase in the research and application of hyperbolic geometry in ANNs and machine learning will lead to increased accuracy, improved feature space representations and more efficient models across a range of tasks. We examine the structure and functions of the human brain, emphasising the correspondence between its scale-free hierarchical organization and hyperbolic geometry, and reflecting on the central role hyperbolic geometry plays in facilitating human intelligence. Empirical evidence indicates that hyperbolic neural networks outperform Euclidean models for tasks including natural language processing, computer vision and complex network analysis, requiring fewer parameters and exhibiting better generalisation. Despite its nascent adoption, hyperbolic geometry holds promise for improving machine learning models through brain-inspired geometric representations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Navigable maps of structural brain networks across species. PLoS computational biology, 16(2): e1007584.
  2. Augustyn, A. 2024. Cognition.
  3. Understanding complexity in the human brain. Trends in cognitive sciences, 15(5): 200–209.
  4. Evolution, brain, and the nature of language. Trends in cognitive sciences, 17(2): 89–98.
  5. Hierarchical structure is employed by humans during visual motion perception. Proceedings of the National Academy of Sciences, 117(39): 24581–24589.
  6. Human Eye Optics within a Non-Euclidian Geometrical Approach and Some Implications in Vision Prosthetics Design. Biomolecules, 11(2): 215. Published online 2021 Feb 4.
  7. Coalescent embedding in the hyperbolic space unsupervisedly discloses the hidden geometry of the brain. arXiv preprint arXiv:1705.04192.
  8. Hyperbolic Geometry.
  9. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nature Human Behaviour, 7: 430–441.
  10. Hyperbolic Pre-Trained Language Model. IEEE ACM Trans. Audio Speech Lang. Process., 32: 3101–3112.
  11. Chen, Y. 2012. Zipf’s law, 1/f noise, and fractal hierarchy. Chaos, Solitons & Fractals, 45(1): 63–73.
  12. Hyperbolic Planforms in Relation to Visual Edges and Textures Perception. PLoS Comput Biol, 5(12): e1000625.
  13. Hyp-ow: Exploiting hierarchical structure learning with hyperbolic distance enhances open world object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 1555–1563.
  14. Evans, B. J. 2022. Chapter 1 - Nature of Binocular Vision Anomalies. In Evans, B. J., ed., Pickwell’s Binocular Vision Anomalies (Sixth Edition), 2–10. Philadelphia: Elsevier, sixth edition edition. ISBN 978-0-323-73317-5.
  15. HyperLib: Deep learning in the Hyperbolic space. https://github.com/nalexai/hyperlib.
  16. The neural basis of recursion and complex syntactic hierarchy. Biolinguistics, 5(1-2): 087–104.
  17. Geoopt. 2019. geoopt. https://github.com/geoopt/geoopt.
  18. Language models as hierarchy encoders. arXiv preprint arXiv:2401.11374.
  19. Neuroanatomy, Visual Cortex. Treasure Island (FL): StatPearls Publishing.
  20. Hyperbolic image embeddings. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6418–6428.
  21. Target cell-specific synaptic dynamics of excitatory to inhibitory neuron connections in supragranular layers of human neocortex. Elife, 12: e81863.
  22. Poincaré maps for analyzing complex hierarchies in single-cell data. Nature communications, 11(1): 2966.
  23. Hyperbolic geometry of complex networks. Phys. Rev. E, 82: 036106.
  24. Lewis, F. C. 2016. Grow those dendrites.
  25. Hyperbolic graph neural networks. Advances in neural information processing systems, 32.
  26. Luneburg, R. K. 1947. Mathematical analysis of binocular vision.
  27. Manifolds, J. 2020. Hyperbolic Learning Library. https://github.com/JuliaManifolds/Manifolds.jl.
  28. Organization and hierarchy of the human functional brain network lead to a chain-like core. Scientific reports, 7(1): 4888.
  29. Hyperbolic deep learning in computer vision: A survey. International Journal of Computer Vision, 1–25.
  30. Hierarchical modularity in human brain functional networks. Frontiers in neuroinformatics, 3: 571.
  31. Are biological systems poised at criticality? Journal of Statistical Physics, 144: 268–302.
  32. Moro, A. 2000. Dynamic antisymmetry. 38. MIT press.
  33. Newman, M. E. J. 2011. Resource Letter CS–1: Complex Systems. American Journal of Physics, 79(8): 800–810.
  34. Nickel, M. 2018. Poincaré Embeddings for Learning Hierarchical Representations. https://github.com/facebookresearch/poincare-embeddings.
  35. Poincaré embeddings for learning hierarchical representations. Advances in neural information processing systems, 30.
  36. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In International conference on machine learning, 3779–3788. PMLR.
  37. Brain hierarchy score: Which deep neural networks are hierarchically brain-like? iScience, 24(9): 103013.
  38. Hierarchical Organization of Human Auditory Cortex: Evidence from Acoustic Invariance in the Response to Intelligible Speech. Cerebral Cortex, 20(10): 2486–2495.
  39. Hyperbolic deep neural networks: A survey. IEEE Transactions on pattern analysis and machine intelligence, 44(12): 10023–10044.
  40. Hierarchical organization in complex networks. Physical review E, 67(2): 026112.
  41. Opportunities for neuromorphic computing algorithms and applications. Nature Computational Science, 2(1): 10–19.
  42. Sharpee, T. O. 2019. An argument for hyperbolic geometry in neural circuits. Current opinion in neurobiology, 58: 101–104.
  43. Emergence and Evolution of Hierarchical Structure in Complex Systems. arXiv:1805.04924.
  44. The global landscape of cognition: hierarchical aggregation as an organizational principle of human cortical networks and functions. Scientific reports, 5(1): 18112.
  45. van Spengler, M. 2023. Hyperbolic Learning Library. https://github.com/maxvanspengler/hyperbolic˙learning˙library.
  46. Hyperbolic disc embedding of functional human brain connectomes using resting-state fMRI. Network Neuroscience, 6(3): 745–764.
  47. UHCone: Universal Hyperbolic Cone For Implicit Hierarchical Learning. In ICML 2024 Workshop on Geometry-grounded Representation Learning and Generative Modeling.
  48. Hyperbolic graph neural networks: A review of methods and applications. arXiv preprint arXiv:2202.13852.
  49. Hyperbolic representation learning: Revisiting and advancing. In International Conference on Machine Learning, 39639–39659. PMLR.
  50. Hippocampal spatial representations exhibit a hyperbolic geometry that expands with experience. Nature Neuroscience, 26(1): 131–139.
  51. Hyperbolic geometry of the olfactory space. Science Advances, 4(8): eaaq1458.
  52. Zipf, G. K. 1949. The Principle of Least Effort. CH3.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now