Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Social Learning through Interactions with Other Agents: A Survey (2407.21713v2)

Published 31 Jul 2024 in cs.LG and cs.AI

Abstract: Social learning plays an important role in the development of human intelligence. As children, we imitate our parents' speech patterns until we are able to produce sounds; we learn from them praising us and scolding us; and as adults, we learn by working with others. In this work, we survey the degree to which this paradigm -- social learning -- has been mirrored in machine learning. In particular, since learning socially requires interacting with others, we are interested in how embodied agents can and have utilised these techniques. This is especially in light of the degree to which recent advances in NLP enable us to perform new forms of social learning. We look at how behavioural cloning and next-token prediction mirror human imitation, how learning from human feedback mirrors human education, and how we can go further to enable fully communicative agents that learn from each other. We find that while individual social learning techniques have been used successfully, there has been little unifying work showing how to bring them together into socially embodied agents.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Rt-h: Action hierarchies using language. arXiv preprint arXiv:2403.01823, 2024.
  2. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623, 2021.
  3. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 41–48, 2009.
  4. Learning few-shot imitation as cultural transmission. Nature Communications, 14(1):7536, 2023.
  5. Social neuro AI: Social interaction as the “dark matter” of AI. Frontiers in Computer Science, 4:846440, 2022.
  6. Machine culture. Nature Human Behaviour, 7(11):1855–1868, 2023.
  7. The cultural niche: Why social learning is essential for human adaptation. Proceedings of the National Academy of Sciences, 108(supplement_2):10918–10925, 2011.
  8. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023.
  9. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR, 2018.
  10. Learning from natural language feedback. Transactions on Machine Learning Research, 2024.
  11. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In The Twelfth International Conference on Learning Representations, 2024.
  12. State2explanation: Concept-based explanations to benefit agent learning and user understanding. Advances in Neural Information Processing Systems, 36:67156–67182, 2023.
  13. A social path to human-like artificial intelligence. Nature Machine Intelligence, 5(11):1181–1188, 2023.
  14. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning, 110(9):2419–2468, 2021.
  15. Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science, 378(6624):1067–1074, 2022.
  16. Mindagent: Emergent gaming interaction. arXiv preprint arXiv:2309.09971, 2023.
  17. Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review, pages 1–49, 2022.
  18. Hyowon Gweon. Inferential social learning: Cognitive foundations of human social learning and teaching. Trends in Cognitive Sciences, 25(10):896–910, 2021.
  19. Contrastive preference learning: Learning from human feedback without reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024.
  20. Imitation learning: A survey of learning methods. ACM Computing Surveys (CSUR), 50(2):1–35, 2017.
  21. Michal Kosinski. Theory of mind may have spontaneously emerged in large language models. arXiv preprint arXiv:2302.02083, 2023.
  22. Theory of mind for multi-agent collaboration via large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 180–192, 2023.
  23. A review on interactive reinforcement learning from human social feedback. IEEE Access, 8:120757–120765, 2020.
  24. Training socially aligned language models on simulated social interactions. In The Twelfth International Conference on Learning Representations, 2023.
  25. Eureka: Human-level reward design via coding large language models. In Second Agent Learning in Open-Endedness Workshop, 2023.
  26. Teacher–student curriculum learning. IEEE Transactions on Neural Networks and Learning Systems, 31(9):3732–3740, 2019.
  27. Embodied visual navigation with automatic curriculum learning in real environments. IEEE Robotics and Automation Letters, 6(2):683–690, 2021.
  28. Embodiedgpt: Vision-language pre-training via embodied chain of thought. Advances in Neural Information Processing Systems, 36, 2024.
  29. A review of cooperative multi-agent deep reinforcement learning. Applied Intelligence, 53(11):13677–13722, 2023.
  30. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  31. Teach: Task-driven embodied agents that chat. Proceedings of the AAAI Conference on Artificial Intelligence, 36(2):2017–2025, Jun. 2022.
  32. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–22, 2023.
  33. Communicative agents for software development. arXiv preprint arXiv:2307.07924, 2023.
  34. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
  35. Habitat-web: Learning embodied object-search strategies from human demonstrations at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5173–5183, 2022.
  36. Curriculum learning via task selection for embodied navigation. CVPR 2023 Embodied AI Workshop, 2023.
  37. Robots that ask for help: Uncertainty alignment for large language model planners. In Conference on Robot Learning, pages 661–682. PMLR, 2023.
  38. Can language models teach? teacher explanations improve student performance via personalization. Advances in Neural Information Processing Systems, 36, 2024.
  39. Learning from natural language feedback. In ACL Workshop on Learning with Natural Language Supervision, 2022.
  40. ALFRED: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10740–10749, 2020.
  41. The development of embodied cognition: Six lessons from babies. Artificial Life, 11(1-2):13–29, 2005.
  42. Learning rewards from linguistic feedback. Proceedings of the AAAI Conference on Artificial Intelligence, 35(7):6002–6010, May 2021.
  43. Show or tell? exploring when (and why) teaching with language outperforms demonstration. Cognition, 232:105326, 2023.
  44. Socially assistive robotics [grand challenges of robotics]. IEEE robotics & automation magazine, 14(1):35–42, 2007.
  45. Cultural learning. Behavioral and Brain Sciences, 16(3):495–511, 1993.
  46. Michael Tomasello. The cultural origins of human cognition. Harvard University Press, 2009.
  47. Scaffolding in teacher–student interaction: A decade of research. Educational Psychology Review, 22:271–296, 2010.
  48. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782):350–354, 2019.
  49. Mind in society: Development of higher psychological processes. Harvard university press, 1978.
  50. JARVIS-1: Open-world multi-task agents with memory-augmented multimodal language models. In Second Agent Learning in Open-Endedness Workshop, 2023.
  51. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research, 2024.
  52. Findings of the BabyLM challenge: Sample-efficient pretraining on developmentally plausible corpora. In Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning, pages 1–34. Association for Computational Linguistics, 2023.
  53. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022.
  54. Social skill training with large language models. arXiv preprint arXiv:2404.04204, 2024.
  55. Transmission versus truth, imitation versus innovation: What children can do that large language and language-and-vision models cannot (yet). Perspectives on Psychological Science, page 17456916231201401, 2023.
  56. Language to rewards for robotic skill synthesis. In 7th Annual Conference on Robot Learning, 2023.
  57. Distilling and retrieving generalizable knowledge for robot manipulation via language corrections. In 2nd Workshop on Language and Robot Learning: Language as Grounding, 2023.
  58. Building cooperative embodied agents modularly with large language models. In The Twelfth International Conference on Learning Representations, 2024.
  59. Imitation learning: Progress, taxonomies and challenges. IEEE Transactions on Neural Networks and Learning Systems, 35(5):6322–6337, 2024.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Dylan Hillier (5 papers)
  2. Cheston Tan (49 papers)
  3. Jing Jiang (192 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube