Learning Translations: Emergent Communication Pretraining for Cooperative Language Acquisition (2402.16247v1)
Abstract: In Emergent Communication (EC) agents learn to communicate with one another, but the protocols that they develop are specialised to their training community. This observation led to research into Zero-Shot Coordination (ZSC) for learning communication strategies that are robust to agents not encountered during training. However, ZSC typically assumes that no prior data is available about the agents that will be encountered in the zero-shot setting. In many cases, this presents an unnecessarily hard problem and rules out communication via preestablished conventions. We propose a novel AI challenge called a Cooperative Language Acquisition Problem (CLAP) in which the ZSC assumptions are relaxed by allowing a 'joiner' agent to learn from a dataset of interactions between agents in a target community. We propose and compare two methods for solving CLAPs: Imitation Learning (IL), and Emergent Communication pretraining and Translation Learning (ECTL), in which an agent is trained in self-play with EC and then learns from the data to translate between the emergent protocol and the target community's protocol.
- Communicating with unknown teammates. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems, AAMAS '14, pages 1433–1434, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems.
- Quasi-Equivalence Discovery for Zero-Shot Emergent Communication. arXiv:2103.08067 [cs].
- Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations. arXiv:2010.15896.
- Learning to Communicate with Strangers via Channel Randomisation Methods. In The Emergent Communication Workshop at NeurIPS.
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning. In D. D. Lee and M. Sugiyama and U. V. Luxburg and I. Guyon and R. Garnett, editor, Advances in Neural Information Processing Systems 29, pages 2137–2145. Curran Associates, Inc.
- Optimizing information exchange in cooperative multi-agent systems. In Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 03), AAMAS '03, pages 137–144, New York, NY, USA. Association for Computing Machinery.
- Decentralized control of cooperative systems: categorization and complexity analysis. Journal of Artificial Intelligence Research, 22(1):143–174.
- Communication-Based Decomposition Mechanisms for Decentralized MDPs. Journal of Artificial Intelligence Research, 32:169–202. arXiv:1111.0065 [cs].
- Off-Belief Learning. In the 38th International Conference on Machine Learning, PMLR 139. arXiv: 2103.04000.
- “Other-Play” for Zero-Shot Coordination. In Proceedings of the 37th International Conference on Machine Learning, pages 4399–4410. PMLR. ISSN: 2640-3498.
- Imitation Learning: A Survey of Learning Methods. ACM Computing Surveys, 50(2):21:1–21:35.
- Categorical Reparameterization with Gumbel-Softmax. In 5th International Conference on Learning Representations (ICLR 17).
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning. In Proceedings of the 36th International Conference on Machine Learning, pages 3040–3049. PMLR. ISSN: 2640-3498.
- Should I Run Offline Reinforcement Learning or Behavioral Cloning? In Proceedings of the International Conference on Learning Representations (ICLR 22).
- Emergent Multi-Agent Communication in the Deep Learning Era. arXiv:2006.02419 [cs].
- Cooperative Open-ended Learning Framework for Zero-Shot Coordination. In Proceedings of the 40th International Conference on Machine Learning, pages 20470–20484. PMLR. ISSN: 2640-3498.
- On the Pitfalls of Measuring Emergent Communication. In The 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS).
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. In 5th International Conference on Learning Representations (ICLR 17), Palais des Congrès Neptune, Toulon, France.
- A Penny for Your Thoughts: The Value of Communication in Ad Hoc Teamwork. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pages 254–260. International Joint Conferences on Artificial Intelligence Organization.
- A Concise Introduction to Decentralized POMDPs. Springer International Publishing, Cham. Series Title: SpringerBriefs in Intelligent Systems.
- Ossenkopf, M. (2020). CoMaze: A cooperative game for zero-shot coordination. In The Emergent Communication Workshop at NeurIPS.
- The Role of Models and Communication in the Ad Hoc Multiagent Team Decision Problem. In Proceedings of the Third Annual Conference on Advances in Cognitive Systems.
- Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination. In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence.
- Learning multiagent communication with backpropagation. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 2252–2260, Barcelona, Spain. Neural Information Processing Systems.
- Progress in the Simulation of Emergent Communication and Language. Adaptive Behavior, 11(1):37–69. Publisher: SAGE Publications Ltd STM.
- The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games. In 36th Conference on Neural Information Processing Systems (NeurIPS 2022) Track on Datasets and Benchmarks., New Orleans, USA.
- Dylan Cope (10 papers)
- Peter McBurney (13 papers)