2000 character limit reached
Memory, Consciousness and Large Language Model (2401.02509v2)
Published 4 Jan 2024 in q-bio.NC, cs.AI, and cs.CL
Abstract: With the development in cognitive science and LLMs, increasing connections have come to light between these two distinct fields. Building upon these connections, we propose a conjecture suggesting the existence of a duality between LLMs and Tulving's theory of memory. We identify a potential correspondence between Tulving's synergistic ecphory model (SEM) of retrieval and the emergent abilities observed in LLMs, serving as supporting evidence for our conjecture. Furthermore, we speculate that consciousness may be considered a form of emergent ability based on this duality. We also discuss how other theories of consciousness intersect with our research.
- Long context prompting for claude 2.1. https://www.anthropic.com/index/claude-2-1-prompting. Accessed: 2023-12-12.
- P. W. Anderson. More is different: Broken symmetry and the nature of the hierarchical structure of science. Science, 177(4047):393–396, 1972.
- M. Atiyah. Duality in mathematics and physics. Conferències FME, 5:2007–2008, 2007.
- Human memory: A proposed system and its control processes. In Psychology of learning and motivation, volume 2, pages 89–195. Elsevier, 1968.
- Consciousness as a memory system. Cognitive and Behavioral Neurology, 35(4):263, 2022.
- Consciousness in artificial intelligence: Insights from the science of consciousness, 2023.
- Data distributional properties drive emergent in-context learning in transformers, 2022.
- Rethinking the distinction between episodic and semantic memory: Insights from the past, present, and future. Memory & Cognition, 50(3):459–463, 2022.
- T. E. Feinberg and J. Mallatt. Phenomenal consciousness and emergence: eliminating the explanatory gap. Frontiers in psychology, 11:1041, 2020.
- A. Gu and T. Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2023.
- J. Kagan. The nature of the child. Basic Books, 1984.
- Consciousness beyond the human case. Current Biology, 33(16):R832–R840, 2023.
- M. Lewis. Shame: The exposed self. Simon and Schuster, 1995.
- The cambridge declaration on consciousness. In Francis crick memorial conference, Cambridge, England, pages 1–2, 2012.
- Are emergent abilities in large language models just in-context learning? arXiv preprint arXiv:2309.01809, 2023.
- A. W. Melton. Implications of short-term memory for a general theory of memory. Journal of verbal Learning and verbal Behavior, 2(1):1–21, 1963.
- Rethinking the role of demonstrations: What makes in-context learning work?, 2022.
- Rwkv: Reinventing rnns for the transformer era, 2023.
- L. Peterson and M. J. Peterson. Short-term retention of individual verbal items. Journal of experimental psychology, 58(3):193, 1959.
- Are emergent abilities of large language models a mirage? arXiv preprint arXiv:2304.15004, 2023.
- A. K. Seth and T. Bayne. Theories of consciousness. Nature Reviews Neuroscience, 23(7):439–452, 2022.
- Organizing memories for generalization in complementary learning systems. Nature neuroscience, 26(8):1438–1448, 2023.
- R. Sutton. The bitter lesson. Incomplete Ideas (blog), 13(1), 2019.
- P. Thagard. Cognitive Science. In E. N. Zalta and U. Nodelman, editors, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter 2023 edition, 2023.
- E. Tulving. Synergistic ecphory in recall and recognition. Canadian Journal of Psychology/Revue canadienne de psychologie, 36(2):130, 1982.
- E. Tulving. Memory and consciousness. Canadian Psychology/Psychologie canadienne, 26(1):1, 1985.
- E. Tulving. Episodic memory: From mind to brain. Annual review of psychology, 53(1):1–25, 2002.
- E. Tulving et al. Episodic and semantic memory. Organization of memory, 1(381-403):1, 1972.
- Label words are anchors: An information flow perspective for understanding in-context learning, 2023.
- Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022.
- L. Weng. Llm-powered autonomous agents. lilianweng.github.io, Jun 2023. URL https://lilianweng.github.io/posts/2023-06-23-agent/.
- A contextual binding theory of episodic memory: systems consolidation reconsidered. Nature Reviews Neuroscience, 20(6):364–375, 2019.