Autonomous and Ubiquitous In-node Learning Algorithms of Active Directed Graphs and Its Storage Behavior (2307.05869v1)
Abstract: Memory is an important cognitive function for humans. How a brain with such a small power can complete such a complex memory function, the working mechanism behind this is undoubtedly fascinating. Engram theory views memory as the co-activation of specific neuronal clusters. From the perspective of graph theory, nodes represent neurons, and directed edges represent synapses. Then the memory engram is the connected subgraph formed between the activated nodes. In this paper, we use subgraphs as physical carriers of information and propose a parallel distributed information storage algorithm based on node scale in active-directed graphs. An active-directed graph is defined as a graph in which each node has autonomous and independent behavior and relies only on information obtained within the local field of view to make decisions. Unlike static directed graphs used for recording facts, active-directed graphs are decentralized like biological neuron networks and do not have a super manager who has a global view and can control the behavior of each node. Distinct from traditional algorithms with a global field of view, this algorithm is characterized by nodes collaborating globally on resource usage through their limited local field of view. While this strategy may not achieve global optimality as well as algorithms with a global field of view, it offers better robustness, concurrency, decentralization, and bioviability. Finally, it was tested in network capacity, fault tolerance, and robustness. It was found that the algorithm exhibits a larger network capacity in a more sparse network structure because the subgraph generated by a single sample is not a whole but consists of multiple weakly connected components. In this case, the network capacity can be understood as the number of permutations of several weakly connected components in the network.
- J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities.” Proceedings of the National Academy of Sciences, vol. 79, no. 8, pp. 2554–2558, Apr. 1982.
- B. Kosko, “Bidirectional associative memories,” IEEE Trans. Syst. Man Cybern., vol. 18, no. 1, pp. 49–60, 1988.
- G. Zlotnik and A. Vansintjan, “Memory: An Extended Definition,” Frontiers in Psychology, vol. 10, p. 2523, 2019.
- L. Luo, “Architectures of neuronal circuits,” Science (New York, N.Y.), vol. 373, no. 6559, p. eabg7285, Sep. 2021.
- N. Francis, A. Green, P. Guagliardo, L. Libkin, T. Lindaaker, V. Marsault, S. Plantikow, M. Rydberg, P. Selmer, and A. Taylor, “Cypher: An Evolving Query Language for Property Graphs,” in Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, June 10-15, 2018. ACM, 2018, pp. 1433–1445.
- S. Ji, S. Pan, E. Cambria, P. Marttinen, and P. S. Yu, “A Survey on Knowledge Graphs: Representation, Acquisition, and Applications,” IEEE Trans. Neural Networks Learn. Syst., vol. 33, no. 2, pp. 494–514, 2022.
- E. W. Dijkstra, “A note on two problems in connexion with graphs,” Numerische Mathematik, vol. 1, pp. 269–271, 1959.
- A. Dorri, S. S. Kanhere, and R. Jurdak, “Multi-Agent Systems: A Survey,” IEEE Access, vol. 6, pp. 28 573–28 593, 2018.
- C. D. Meliza and Y. Dan, “Receptive-field modification in rat visual cortex induced by paired visual stimulation and single-cell spiking,” Neuron, vol. 49, no. 2, pp. 183–189, Jan. 2006.
- D. Krotov and J. J. Hopfield, “Dense Associative Memory for Pattern Recognition,” in Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, 2016, pp. 1172–1180.
- M. Demircigil, J. Heusel, M. Löwe, S. Upgang, and F. Vermet, “On a Model of Associative Memory with Huge Storage Capacity,” Journal of Statistical Physics, vol. 168, no. 2, pp. 288–299, Jul. 2017.
- H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, L. Gruber, M. Holzleitner, T. Adler, D. P. Kreil, M. K. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter, “Hopfield Networks is All You Need,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
- B. Kosko, “Bidirectional Associative Memories: Unsupervised Hebbian Learning to Bidirectional Backpropagation,” IEEE Trans. Syst. Man Cybern. Syst., vol. 51, no. 1, pp. 103–115, 2021.
- R. Angles and C. Gutierrez, “Survey of graph database models,” ACM Computing Surveys, vol. 40, no. 1, pp. 1:1–1:39, Feb. 2008.
- I. Thiele, N. Swainston, R. M. T. Fleming, A. Hoppe, S. Sahoo, M. K. Aurich, H. Haraldsdottir, M. L. Mo, O. Rolfsson, M. D. Stobbe, S. G. Thorleifsson, R. Agren, C. Bölling, S. Bordel, A. K. Chavali, P. Dobson, W. B. Dunn, L. Endler, D. Hala, M. Hucka, D. Hull, D. Jameson, N. Jamshidi, J. J. Jonsson, N. Juty, S. Keating, I. Nookaew, N. Le Novère, N. Malys, A. Mazein, J. A. Papin, N. D. Price, E. Selkov, M. I. Sigurdsson, E. Simeonidis, N. Sonnenschein, K. Smallbone, A. Sorokin, J. H. G. M. van Beek, D. Weichart, I. Goryanin, J. Nielsen, H. V. Westerhoff, D. B. Kell, P. Mendes, and B. Ø. Palsson, “A community-driven global reconstruction of human metabolism,” Nature Biotechnology, vol. 31, no. 5, pp. 419–425, May 2013.
- I. Vastrik, P. D’Eustachio, E. Schmidt, G. Joshi-Tope, G. Gopinath, D. Croft, B. de Bono, M. Gillespie, B. Jassal, S. Lewis, L. Matthews, G. Wu, E. Birney, and L. Stein, “Reactome: A knowledge base of biologic pathways and processes,” Genome Biology, vol. 8, no. 3, p. R39, Mar. 2007.
- HoganAidan, BlomqvistEva, CochezMichael, D’amatoClaudia, M. De, GutierrezClaudio, KirraneSabrina, G. E. Labra, NavigliRoberto, NeumaierSebastian, N.-C. Ngonga, PolleresAxel, R. M, RulaAnisa, SchmelzeisenLukas, SequedaJuan, StaabSteffen, and ZimmermannAntoine, “Knowledge Graphs,” ACM Computing Surveys (CSUR), Jul. 2021.
- S. A. Josselyn and S. Tonegawa, “Memory engrams: Recalling the past and imagining the future,” Science (New York, N.Y.), vol. 367, no. 6473, Jan. 2020.
- K. Abdou, M. Shehata, K. Choko, H. Nishizono, M. Matsuo, S.-I. Muramatsu, and K. Inokuchi, “Synapse-specific representation of the identity of overlapping memory engrams,” Science (New York, N.Y.), vol. 360, no. 6394, pp. 1227–1231, Jun. 2018.
- N. Ohkawa, Y. Saitoh, A. Suzuki, S. Tsujimura, E. Murayama, S. Kosugi, H. Nishizono, M. Matsuo, Y. Takahashi, M. Nagase, Y. K. Sugimura, A. M. Watabe, F. Kato, and K. Inokuchi, “Artificial association of pre-stored information to generate a qualitatively new memory,” Cell Reports, vol. 11, no. 2, pp. 261–269, Apr. 2015.
- N. X. Tritsch, A. J. Granger, and B. L. Sabatini, “Mechanisms and functions of GABA co-release,” Nature Reviews. Neuroscience, vol. 17, no. 3, pp. 139–145, Mar. 2016.
- L. Chen, K. A. Cummings, W. Mau, Y. Zaki, Z. Dong, S. Rabinowitz, R. L. Clem, T. Shuman, and D. J. Cai, “The role of intrinsic excitability in the evolution of memory: Significance in memory allocation, consolidation, and updating,” Neurobiology of Learning and Memory, vol. 173, p. 107266, Sep. 2020.
- P. Erdös and A. Rényi, “On Random Graphs I,” Publicationes Mathematicae Debrecen, vol. 6, p. 290, 1959.
- ——, “On the evolution of random graphs,” in The Structure and Dynamics of Networks. Princeton University Press, 2006, pp. 38–82.
- D. J. Watts and S. H. Strogatz, “Collective dynamics of ‘small-world’ networks,” Nature, vol. 393, no. 6684, pp. 440–442, Jun. 1998.
- D. J. d. S. Price, “Networks of Scientific Papers,” Science, vol. 149, no. 3683, pp. 510–515, Jul. 1965.
- G. Fagiolo, “Clustering in complex directed networks,” Physical Review E, vol. 76, no. 2, p. 026107, 2007.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.