2000 character limit reached
Meta-Referential Games to Learn Compositional Learning Behaviours
Published 16 Jul 2022 in cs.CL and cs.LG | (2207.08012v5)
Abstract: Human beings use compositionality to generalise from past experiences to novel experiences. We assume a separation of our experiences into fundamental atomic components that can be recombined in novel ways to support our ability to engage with novel experiences. We frame this as the ability to learn to generalise compositionally, and we will refer to behaviours making use of this ability as compositional learning behaviours (CLBs). A central problem to learning CLBs is the resolution of a binding problem (BP). While it is another feat of intelligence that human beings perform with ease, it is not the case for state-of-the-art artificial agents. Thus, in order to build artificial agents able to collaborate with human beings, we propose to develop a novel benchmark to investigate agents' abilities to exhibit CLBs by solving a domain-agnostic version of the BP. We take inspiration from the language emergence and grounding framework of referential games and propose a meta-learning extension of referential games, entitled Meta-Referential Games, and use this framework to build our benchmark, the Symbolic Behaviour Benchmark (S2B). We provide baseline results and error analysis showing that our benchmark is a compelling challenge that we hope will spur the research community towards developing more capable artificial agents.
- Relational inductive biases, deep learning, and graph networks. 2018. URL https://arxiv.org/pdf/1806.01261.pdf.
- A survey of meta-reinforcement learning. arXiv preprint arXiv:2301.08028, 2023.
- Y. Bengio. Deep learning of representations for unsupervised and transfer learning. Conf. Proc. IEEE Eng. Med. Biol. Soc., 27:17–37, 2012.
- L. Biewald. Experiment tracking with weights and biases, 2020. URL https://www.wandb.com/. Software available from wandb.com.
- N. Brandizzi. Towards more human-like AI communication: A review of emergent communication research. Aug. 2023.
- H. Brighton. Compositional syntax from cultural transmission. MIT Press, Artificial, 2002. URL https://www.mitpressjournals.org/doi/abs/10.1162/106454602753694756.
- H. Brighton and S. Kirby. The survival of the smallest: Stability conditions for the cultural evolution of compositional language. In European Conference on Artificial Life, pages 592–601. Springer, 2001.
- H. Brighton and S. Kirby. Understanding Linguistic Evolution by Visualizing the Emergence of Topographic Mappings. Artificial Life, 12(2):229–242, jan 2006. ISSN 1064-5462. doi: 10.1162/artl.2006.12.2.229. URL http://www.mitpressjournals.org/doi/10.1162/artl.2006.12.2.229.
- Compositionality and Generalization in Emergent Languages. apr 2020. URL http://arxiv.org/abs/2004.09124.
- Isolating sources of disentanglement in VAEs. https://papers.nips.cc/paper/2018/file/1ee3dfcd8a0645a25a35977997223d22-Paper.pdf. Accessed: 2021-3-17.
- Compositional Obverter Communication Learning From Raw Visual Input. apr 2018. URL http://arxiv.org/abs/1804.02341.
- F. Chollet. On the Measure of Intelligence. Technical report, 2019.
- D. Cope and N. Schoots. Learning to communicate with strangers via channel randomisation methods. arXiv preprint arXiv:2104.09557, 2021.
- K. Denamganaï and J. A. Walker. Referentialgym: A nomenclature and framework for language emergence & grounding in (visual) referential games. 4th NeurIPS Workshop on Emergent Communication, 2020a.
- K. Denamganaï and J. A. Walker. Referentialgym: A framework for language emergence & grounding in (visual) referential games. 4th NeurIPS Workshop on Emergent Communication, 2020b.
- Interpretable agent communication from scratch (with a generic visual processor emerging on the side). May 2021.
- Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1-2):3–71, 1988.
- Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471–476, 2016.
- On the binding problem in artificial neural networks. arXiv preprint arXiv:2012.05208, 2020.
- The emergence of compositional languages for numeric concepts through iterated learning in neural agents. arXiv preprint arXiv:1910.05291, 2019.
- Towards a Definition of Disentangled Representations. dec 2018. URL http://arxiv.org/abs/1812.02230.
- Environmental drivers of systematicity and generalization in a situated agent. Oct. 2019.
- Grounded language learning fast and slow. Technical report, 2020.
- S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
- Distributed prioritized experience replay. arXiv preprint arXiv:1803.00933, 2018.
- H. Hu and J. N. Foerster. Simplified action decoder for deep multi-agent reinforcement learning. In International Conference on Learning Representations, 2019.
- Compositionality decomposed: how do neural networks generalise? aug 2019. URL http://arxiv.org/abs/1908.08351.
- R. Jakobson. Linguistics and poetics. In Style in language, pages 350–377. MA: MIT Press, 1960.
- Recurrent experience replay in distributed reinforcement learning. In International conference on learning representations, 2018.
- H. Kim and A. Mnih. Disentangling by factorising. arXiv preprint arXiv:1802.05983, 2018.
- D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
- M. Krifka. Compositionality. The MIT encyclopedia of the cognitive sciences, pages 152–153, 2001.
- B. M. Lake. Compositional generalization through meta sequence-to-sequence learning. Advances in neural information processing systems, 32, 2019.
- B. M. Lake and M. Baroni. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. 35th International Conference on Machine Learning, ICML 2018, 7:4487–4499, oct 2018. URL http://arxiv.org/abs/1711.00350.
- B. M. Lake and M. Baroni. Human-like systematic generalization through a meta-learning neural network. Nature, pages 1–7, 2023.
- A. Lazaridou and M. Baroni. Emergent Multi-Agent communication in the deep learning era. June 2020.
- Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input. apr 2018. URL http://arxiv.org/abs/1804.03984.
- D. Lewis. Convention: A philosophical study. 1969.
- A sober look at the unsupervised learning of disentangled representations and their evaluation. Oct. 2020.
- Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks. jul 2018. URL http://arxiv.org/abs/1807.07545.
- A simple neural attentive meta-learner. In International Conference on Learning Representations, 2018.
- R. Montague. Universal grammar. Theoria, 36(3):373–398, 1970.
- J. Mu and N. Goodman. Emergent communication of generalizations. Advances in Neural Information Processing Systems, 34:17994–18007, 2021.
- Compositional Languages Emerge in a Neural Iterated Learning Model. feb 2020. URL http://arxiv.org/abs/2002.01365.
- K. Ridgeway and M. C. Mozer. Learning deep disentangled embeddings with the F-Statistic loss, 2018.
- A benchmark for systematic generalization in grounded language understanding. Mar. 2020.
- Symbolic behaviour in artificial intelligence. arXiv preprint arXiv:2102.03406, 2021.
- Exploring Structural Inductive Biases in Emergent Communication. feb 2020. URL http://arxiv.org/abs/2002.01335.
- Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296, 2017.
- Matching networks for one shot learning. volume 29, 2016.
- Dueling network architectures for deep reinforcement learning. In International conference on machine learning, pages 1995–2003. PMLR, 2016.
- Emergent symbols through binding in external memory. In International Conference on Learning Representations, 2020.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.