Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
91 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
o3 Pro
5 tokens/sec
GPT-4.1 Pro
15 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
Gemini 2.5 Flash Deprecated
12 tokens/sec
2000 character limit reached

Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments (2311.04453v2)

Published 8 Nov 2023 in cs.CL

Abstract: As a sub-discipline of evolutionary and computational linguistics, emergent communication (EC) studies communication protocols, called emergent languages, arising in simulations where agents communicate. A key goal of EC is to give rise to languages that share statistical properties with natural languages. In this paper, we reinterpret Lewis's signaling game, a frequently used setting in EC, as beta-VAE and reformulate its objective function as ELBO. Consequently, we clarify the existence of prior distributions of emergent languages and show that the choice of the priors can influence their statistical properties. Specifically, we address the properties of word lengths and segmentation, known as Zipf's law of abbreviation (ZLA) and Harris's articulation scheme (HAS), respectively. It has been reported that the emergent languages do not follow them when using the conventional objective. We experimentally demonstrate that by selecting an appropriate prior distribution, more natural segments emerge, while suggesting that the conventional one prevents the languages from following ZLA and HAS.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (89)
  1. Deep variational information bottleneck. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum?id=HyxQzBceg.
  2. Fixing a broken ELBO. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp.  159–168. PMLR, 2018. URL http://proceedings.mlr.press/v80/alemi18a.html.
  3. Jacob Andreas. Measuring compositionality in representation learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL https://openreview.net/forum?id=HJz05o0qK7.
  4. Layer normalization. CoRR, abs/1607.06450, 2016. URL http://arxiv.org/abs/1607.06450.
  5. Text Compression. Prentice-Hall, Inc., 1990.
  6. Generating sentences from a continuous space. In Yoav Goldberg and Stefan Riezler (eds.), Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, August 11-12, 2016, pp.  10–21. ACL, 2016. doi: 10.18653/v1/k16-1002. URL https://doi.org/10.18653/v1/k16-1002.
  7. Nicolo’ Brandizzi. Towards more human-like AI communication: A review of emergent communication research. CoRR, abs/2308.02541, 2023. doi: 10.48550/arXiv.2308.02541. URL https://doi.org/10.48550/arXiv.2308.02541.
  8. Understanding linguistic evolution by visualizing the emergence of topographic mappings. Artif. Life, 12(2):229–242, 2006. URL https://doi.org/10.1162/artl.2006.12.2.229.
  9. Anti-efficient encoding in emergent communication. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp.  6290–6300, 2019. URL https://proceedings.neurips.cc/paper/2019/hash/31ca0ca71184bbdb3de7b20a51e88e90-Abstract.html.
  10. Compositionality and generalization in emergent languages. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp.  4427–4442. Association for Computational Linguistics, 2020. URL https://doi.org/10.18653/v1/2020.acl-main.407.
  11. Communicating artificial neural networks develop efficient color-naming systems. Proceedings of the National Academy of Sciences, 118(12):e2016569118, 2021. doi: 10.1073/pnas.2016569118. URL https://www.pnas.org/doi/abs/10.1073/pnas.2016569118.
  12. Emergent communication at scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https://openreview.net/forum?id=AUGBfDIV9rL.
  13. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp.  1724–1734. ACL, 2014. URL https://doi.org/10.3115/v1/d14-1179.
  14. Compositional obverter communication learning from raw visual input. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview.net/forum?id=rknt2Be0-.
  15. Power laws for monkeys typing randomly: the case of unequal probabilities. IEEE Transactions on Information Theory, 50(7):1403–1414, 2004. doi: 10.1109/TIT.2004.830752.
  16. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, 2006.
  17. Interpretable agent communication from scratch (with a generic visual processor emerging on the side). In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp.  26937–26949, 2021. URL https://proceedings.neurips.cc/paper/2021/hash/e250c59336b505ed411d455abaa30b4d-Abstract.html.
  18. Recurrent neural network grammars. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pp.  199–209. The Association for Computational Linguistics, 2016. doi: 10.18653/v1/n16-1024. URL https://doi.org/10.18653/v1/n16-1024.
  19. Learning to communicate with deep multi-agent reinforcement learning. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp.  2137–2145, 2016. URL https://proceedings.neurips.cc/paper/2016/hash/c7635bfd99248a2cdef8249ef7bfbef4-Abstract.html.
  20. Cyclical annealing schedule: A simple approach to mitigating KL vanishing. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pp.  240–250. Association for Computational Linguistics, 2019. doi: 10.18653/v1/n19-1021. URL https://doi.org/10.18653/v1/n19-1021.
  21. A theoretically grounded application of dropout in recurrent neural networks. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp.  1019–1027, 2016. URL https://proceedings.neurips.cc/paper/2016/hash/076a0c97d09cf1a0ec3e19c7f2529f2b-Abstract.html.
  22. Emergent communication for understanding human language evolution: What’s missing? CoRR, abs/2204.10590, 2022. doi: 10.48550/arXiv.2204.10590. URL https://doi.org/10.48550/arXiv.2204.10590.
  23. John Hale. A probabilistic earley parser as a psycholinguistic model. In Language Technologies 2001: The Second Meeting of the North American Chapter of the Association for Computational Linguistics, NAACL 2001, Pittsburgh, PA, USA, June 2-7, 2001. The Association for Computational Linguistics, 2001. URL https://aclanthology.org/N01-1021/.
  24. Zellig S. Harris. From phoneme to morpheme. Language, 31(2):190–222, 1955. URL http://www.jstor.org/stable/411036.
  25. Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp.  2149–2159, 2017. URL https://proceedings.neurips.cc/paper/2017/hash/70222949cc0db89ab32c9969754d4758-Abstract.html.
  26. beta-vae: Learning basic visual concepts with a constrained variational framework. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum?id=Sy2fzU9gl.
  27. Emergent communication of multimodal deep generative models based on metropolis-hastings naming game. Frontiers in Robotics and AI, 10, 2024. ISSN 2296-9144. doi: 10.3389/frobt.2023.1290604. URL https://www.frontiersin.org/articles/10.3389/frobt.2023.1290604.
  28. Long short-term memory. Neural Comput., 9(8):1735–1780, 1997. URL https://doi.org/10.1162/neco.1997.9.8.1735.
  29. Recursive metropolis-hastings naming game: Symbol emergence in a multi-agent system based on probabilistic generative models. CoRR, abs/2305.19761, 2023. doi: 10.48550/arXiv.2305.19761. URL https://doi.org/10.48550/arXiv.2305.19761.
  30. Categorical reparameterization with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum?id=rkE3y85ee.
  31. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp.  3040–3049. PMLR, 2019. URL http://proceedings.mlr.press/v97/jaques19a.html.
  32. Entropy minimization in emergent languages. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp.  5220–5230. PMLR, 2020. URL http://proceedings.mlr.press/v119/kharitonov20a.html.
  33. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6980.
  34. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http://arxiv.org/abs/1312.6114.
  35. Iterated learning and the evolution of language. Current Opinion in Neurobiology, 28:108–114, 2014. ISSN 0959-4388. doi: https://doi.org/10.1016/j.conb.2014.07.014. URL https://www.sciencedirect.com/science/article/pii/S0959438814001421. SI: Communication and language.
  36. Compression and communication in the cultural evolution of linguistic structure. Cognition, 141:87–102, 2015. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition.2015.03.016. URL https://www.sciencedirect.com/science/article/pii/S0010027715000815.
  37. Learning hierarchical priors in vaes. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp.  2866–2875, 2019. URL https://proceedings.neurips.cc/paper/2019/hash/7d12b66d3df6af8d429c1a357d8b9e1a-Abstract.html.
  38. Natural language does not emerge ’naturally’ in multi-agent dialog. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel (eds.), Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pp.  2962–2967. Association for Computational Linguistics, 2017. URL https://doi.org/10.18653/v1/d17-1321.
  39. Representational similarity analysis - connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 2008. ISSN 1662-5137. doi: 10.3389/neuro.06.004.2008. URL https://www.frontiersin.org/articles/10.3389/neuro.06.004.2008.
  40. Context limitations make neural language models more human-like. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp.  10421–10436. Association for Computational Linguistics, 2022. doi: 10.18653/v1/2022.emnlp-main.712. URL https://doi.org/10.18653/v1/2022.emnlp-main.712.
  41. Emergent multi-agent communication in the deep learning era. CoRR, abs/2006.02419, 2020. URL https://arxiv.org/abs/2006.02419.
  42. Multi-agent cooperation and the emergence of (natural) language. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum?id=Hk8N3Sclg.
  43. Emergence of linguistic communication from referential games with symbolic and pixel input. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview.net/forum?id=HJGv1Z-AW.
  44. Multi-agent communication meets natural language: Synergies between functional and structural language learning. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp.  7663–7674. Association for Computational Linguistics, 2020. URL https://doi.org/10.18653/v1/2020.acl-main.685.
  45. Sergey Levine. Reinforcement learning and control as probabilistic inference: Tutorial and review. CoRR, abs/1805.00909, 2018. URL http://arxiv.org/abs/1805.00909.
  46. Roger Levy. Expectation-based syntactic comprehension. Cognition, 106(3):1126–1177, 2008. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition.2007.05.006. URL https://www.sciencedirect.com/science/article/pii/S0010027707001436.
  47. David K. Lewis. Convention: A Philosophical Study. Wiley-Blackwell, 1969.
  48. Ease-of-teaching and language structure from emergent communication. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp.  15825–15835, 2019. URL https://proceedings.neurips.cc/paper/2019/hash/b0cf188d74589db9b23d5d277238a929-Abstract.html.
  49. Multi-agent actor-critic for mixed cooperative-competitive environments. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp.  6379–6390, 2017. URL https://proceedings.neurips.cc/paper/2017/hash/68a9750337a418a86fe06c1991a1d64c-Abstract.html.
  50. The concrete distribution: A continuous relaxation of discrete random variables. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum?id=S1jE5L5gl.
  51. André Martinet. Éléments de linguistique générale. Armand Colin, 1960.
  52. Revisiting populations in multi-agent communication. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/pdf?id=n-UHRIdPju.
  53. George A. Miller. Some effects of intermittent silence. The American Journal of Psychology, 70(2):311–314, 1957. URL http://www.jstor.org/stable/1419346.
  54. Neural variational inference and learning in belief networks. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014, volume 32 of JMLR Workshop and Conference Proceedings, pp.  1791–1799. JMLR.org, 2014. URL http://proceedings.mlr.press/v32/mnih14.html.
  55. Asynchronous methods for deep reinforcement learning. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pp.  1928–1937. JMLR.org, 2016. URL http://proceedings.mlr.press/v48/mniha16.html.
  56. Emergence of grounded compositional language in multi-agent populations. In Sheila A. McIlraith and Kilian Q. Weinberger (eds.), Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp.  1495–1502. AAAI Press, 2018. URL https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17007.
  57. Larger communities create more systematic languages. Proceedings of the Royal Society B: Biological Sciences, 286(1907):20191262, 2019. doi: 10.1098/rspb.2019.1262. URL https://royalsocietypublishing.org/doi/abs/10.1098/rspb.2019.1262.
  58. Compositional languages emerge in a neural iterated learning model. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openreview.net/forum?id=HkePNpVKPB.
  59. Capacity, bandwidth, and compositionality in emergent language learning. In Amal El Fallah Seghrouchni, Gita Sukthankar, Bo An, and Neil Yorke-Smith (eds.), Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’20, Auckland, New Zealand, May 9-13, 2020, pp.  1125–1133. International Foundation for Autonomous Agents and Multiagent Systems, 2020. doi: 10.5555/3398761.3398892. URL https://dl.acm.org/doi/10.5555/3398761.3398892.
  60. Emergent communication with attention. CoRR, abs/2305.10920, 2023. doi: 10.48550/arXiv.2305.10920. URL https://doi.org/10.48550/arXiv.2305.10920.
  61. ”lazimpa”: Lazy and impatient neural agents learn to communicate efficiently. In Raquel Fernández and Tal Linzen (eds.), Proceedings of the 24th Conference on Computational Natural Language Learning, CoNLL 2020, Online, November 19-20, 2020, pp.  335–343. Association for Computational Linguistics, 2020. URL https://doi.org/10.18653/v1/2020.conll-1.26.
  62. On the role of population heterogeneity in emergent communication. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022a. URL https://openreview.net/forum?id=5Qkd7-bZfI.
  63. Emergent communication: Generalization and overfitting in lewis games. In NeurIPS, 2022b. URL http://papers.nips.cc/paper_files/paper/2022/hash/093b08a7ad6e6dd8d34b9cc86bb5f07c-Abstract-Conference.html.
  64. Jason Tyler Rolfe. Discrete variational autoencoders. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum?id=ryMxXPFex.
  65. Gradient estimation using stochastic computation graphs. In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp.  3528–3536, 2015. URL https://proceedings.neurips.cc/paper/2015/hash/de03beffeed9da5f3639a621bcab5dd4-Abstract.html.
  66. Ordered neurons: Integrating tree structures into recurrent neural networks. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL https://openreview.net/forum?id=B1l6qiR5F7.
  67. Brian Skyrms. Signals: Evolution, Learning, and Information. Oxford University Press, Oxford, GB, 2010.
  68. The effect of word predictability on reading time is logarithmic. Cognition, 128(3):302–319, 2013. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition.2013.02.013. URL https://www.sciencedirect.com/science/article/pii/S0010027713000413.
  69. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1):1929–1958, 2014. doi: 10.5555/2627435.2670313. URL https://dl.acm.org/doi/10.5555/2627435.2670313.
  70. Modeling incremental language comprehension in the brain with combinatory categorial grammar. In Emmanuele Chersoni, Nora Hollenstein, Cassandra Jacobs, Yohei Oseki, Laurent Prévot, and Enrico Santus (eds.), Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, CMCL 2021, Online, June 10, 2021, pp.  23–38. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.cmcl-1.3. URL https://doi.org/10.18653/v1/2021.cmcl-1.3.
  71. A survey of multimodal deep generative models. Adv. Robotics, 36(5-6):261–278, 2022. doi: 10.1080/01691864.2022.2035253. URL https://doi.org/10.1080/01691864.2022.2035253.
  72. Kumiko Tanaka-Ishii. Entropy as an indicator of context boundaries: An experiment using a web search engine. In Robert Dale, Kam-Fai Wong, Jian Su, and Oi Yee Kwong (eds.), Natural Language Processing - IJCNLP 2005, Second International Joint Conference, Jeju Island, Korea, October 11-13, 2005, Proceedings, volume 3651 of Lecture Notes in Computer Science, pp.  93–105. Springer, 2005. URL https://doi.org/10.1007/11562214_9.
  73. Kumiko Tanaka-Ishii. Articulation of Elements, pp.  115–124. Springer International Publishing, Cham, 2021. URL https://doi.org/10.1007/978-3-030-59377-3_11.
  74. Multilingual phrase-based concordance generation in real-time. Inf. Retr., 10(3):275–295, 2007. URL https://doi.org/10.1007/s10791-006-9021-5.
  75. Emergent communication through metropolis-hastings naming game with deep generative models. CoRR, abs/2205.12392, 2022. doi: 10.48550/arXiv.2205.12392. URL https://doi.org/10.48550/arXiv.2205.12392.
  76. Trading off utility, informativeness, and complexity in emergent communication. In NeurIPS, 2022. URL http://papers.nips.cc/paper_files/paper/2022/hash/8bb5f66371c7e4cbf6c223162c62c0f4-Abstract-Conference.html.
  77. On the relationship between zipf’s law of abbreviation and interfering noise in emergent languages. In Proceedings of the ACL-IJCNLP 2021 Student Research Workshop, ACL 2021, Online, JUli 5-10, 2021, pp.  60–70. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.acl-srw.6. URL https://doi.org/10.18653/v1/2021.acl-srw.6.
  78. On the word boundaries of emergent languages based on harris’s articulation scheme. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/pdf?id=b4t9_XASt6G.
  79. Dvae#: Discrete variational autoencoders with relaxed boltzmann priors. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp.  1869–1878, 2018a. URL https://proceedings.neurips.cc/paper/2018/hash/9f53d83ec0691550f7d2507d57f4f5a2-Abstract.html.
  80. DVAE++: discrete variational autoencoders with overlapping transformations. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp.  5042–5051. PMLR, 2018b. URL http://proceedings.mlr.press/v80/vahdat18a.html.
  81. Representation learning with contrastive predictive coding. CoRR, abs/1807.03748, 2018. URL http://arxiv.org/abs/1807.03748.
  82. The grammar of emergent languages. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp.  3339–3359. Association for Computational Linguistics, 2020. URL https://doi.org/10.18653/v1/2020.emnlp-main.270.
  83. On the predictive power of neural language models for human real-time comprehension behavior. In Stephanie Denison, Michael Mack, Yang Xu, and Blair C. Armstrong (eds.), Proceedings of the 42th Annual Meeting of the Cognitive Science Society - Developing a Mind: Learning in Humans, Animals, and Machines, CogSci 2020, virtual, July 29 - August 1, 2020. cognitivesciencesociety.org, 2020. URL https://cogsci.mindmodeling.org/2020/papers/0375/index.html.
  84. Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn., 8:229–256, 1992. URL https://doi.org/10.1007/BF00992696.
  85. Function optimization using connectionist reinforcement learning algorithms. Connection Science, 3:241–268, 1991.
  86. Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language. In NeurIPS, 2022. URL http://papers.nips.cc/paper_files/paper/2022/hash/9f9ecbf4062842df17ec3f4ea3ad7f54-Abstract-Conference.html.
  87. The evolution of color naming reflects pressure for efficiency: Evidence from the recent past. Journal of Language Evolution, 7(2):184–199, 04 2022. ISSN 2058-458X. doi: 10.1093/jole/lzac001. URL https://doi.org/10.1093/jole/lzac001.
  88. George K. Zipf. The psycho-biology of language. Houghton Mifflin, 1935.
  89. George K. Zipf. Human Behaviour and the Principle of Least Effort. Addison-Wesley, 1949.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now