Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
11 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
40 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
37 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Agents Thinking Fast and Slow: A Talker-Reasoner Architecture (2410.08328v1)

Published 10 Oct 2024 in cs.AI, cs.CL, and cs.LG

Abstract: LLMs have enabled agents of all kinds to interact with users through natural conversation. Consequently, agents now have two jobs: conversing and planning/reasoning. Their conversational responses must be informed by all available information, and their actions must help to achieve goals. This dichotomy between conversing with the user and doing multi-step reasoning and planning can be seen as analogous to the human systems of "thinking fast and slow" as introduced by Kahneman. Our approach is comprised of a "Talker" agent (System 1) that is fast and intuitive, and tasked with synthesizing the conversational response; and a "Reasoner" agent (System 2) that is slower, more deliberative, and more logical, and is tasked with multi-step reasoning and planning, calling tools, performing actions in the world, and thereby producing the new agent state. We describe the new Talker-Reasoner architecture and discuss its advantages, including modularity and decreased latency. We ground the discussion in the context of a sleep coaching agent, in order to demonstrate real-world relevance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023).
  2. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691 (2022).
  3. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 (2022).
  4. Large language models can implement policy iteration. Advances in Neural Information Processing Systems 36 (2024).
  5. Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020).
  6. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023).
  7. Jay W Forrester. 1971. Counterintuitive behavior of social systems. Theory and decision 2, 2 (1971), 109–140.
  8. Uta Frith and Christopher D Frith. 2003. Development and neurophysiology of mentalizing. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences 358, 1431 (2003), 459–473.
  9. Learning actionable representations with goal-conditioned policies. arXiv preprint arXiv:1811.07819 (2018).
  10. David Ha and Jürgen Schmidhuber. 2018. World models. arXiv preprint arXiv:1803.10122 (2018).
  11. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992 (2023).
  12. Zhiting Hu and Tianmin Shu. 2023. Language models, agent models, and world models: The law for machine reasoning and planning. arXiv preprint arXiv:2312.05230 (2023).
  13. Arianna Huffington. 2016. The sleep revolution: Transforming your life, one night at a time. Harmony.
  14. Daniel Kahneman. 2011. Thinking, fast and slow. Farrar, Straus and Giroux (2011).
  15. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199–22213.
  16. Robotic systems architectures and programming. Springer handbook of robotics (2016), 283–306.
  17. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33 (2020), 9459–9474.
  18. Theory of mind for multi-agent collaboration via large language models. arXiv preprint arXiv:2310.10701 (2023).
  19. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 9493–9500.
  20. Kevin P Murphy. 2000. A survey of POMDP solution techniques. environment 2, 10 (2000).
  21. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332 (2021).
  22. Generative agents: Interactive simulacra of human behavior. (2023), 1–22.
  23. David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind? Behavioral and brain sciences 1, 4 (1978), 515–526.
  24. Llm-assist: Enhancing closed-loop planning with language-based reasoning. arXiv preprint arXiv:2401.00125 (2023).
  25. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems 36 (2024).
  26. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 36 (2024).
  27. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 11523–11530.
  28. Interactive Planning Using Large Language Models for Partially Observable Robotic Tasks. In 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 14054–14061.
  29. Reinforcement learning. Journal of Cognitive Neuroscience 11, 1 (1999), 126–134.
  30. Large language models as generalizable policies for embodied tasks. In The Twelfth International Conference on Learning Representations.
  31. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023).
  32. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022).
  33. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).
  34. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023).
  35. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 6 (2024), 186345.
  36. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560 (2023).
  37. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837.
  38. Auto-gpt for online decision making: Benchmarks and additional opinions. arXiv preprint arXiv:2306.02224 (2023).
  39. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022).
  40. Empowering Large Language Model Agents through Action Learning. arXiv preprint arXiv:2402.15809 (2024).
  41. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023).
  42. How FaR Are Large Language Models From Agents with Theory-of-Mind? arXiv preprint arXiv:2310.03051 (2023).
  43. Self-discover: Large language models self-compose reasoning structures. arXiv preprint arXiv:2402.03620 (2024).
Citations (1)
View on Semantic Scholar

Summary

  • The paper presents a dual-agent framework that fuses rapid, intuitive dialogue from the Talker with methodical, multi-step reasoning from the Reasoner.
  • It employs large language models to drive both components, enabling fast conversational engagement alongside precise decision-making and belief updating.
  • Experimental evaluation using an AI sleep coaching agent highlights the architecture’s effectiveness in managing dialogue and executing complex planning tasks.

Exploring the Talker-Reasoner Architecture in Intelligent Agents

The paper "Agents Thinking Fast and Slow: A Talker-Reasoner Architecture" introduces an innovative dual-agent framework for conversational AI systems. It draws inspiration from Daniel Kahneman's dual-system theory of cognition, specifically the fast, intuitive System 1 and the slower, deliberative System 2. This architecture is engineered to enhance AI's capability to handle both conversational interactions and complex problem-solving tasks effectively.

Framework Overview

The proposed framework consists of two primary components: the Talker and the Reasoner. The Talker acts as System 1, providing rapid and intuitive responses during natural language interactions with users. It continuously engages with the environment, obtaining observations and maintaining dialogue coherence, albeit sometimes with outdated belief states. In contrast, the Reasoner embodies System 2, handling deliberate and methodical multi-step reasoning, planning, and belief updating.

The architecture employs LLMs to implement the Talker and Reasoner. This ensures the agents draw upon robust language priors and in-context learning capabilities, allowing them to excel in their respective roles. The Talker interacts directly with users, relying on stored beliefs and interaction history, whereas the Reasoner utilizes a modular approach to perform complex computations, make decisions, and update belief states.

Theoretical Implications

The Talker-Reasoner architecture offers a biologically inspired approach to AI agent design, mirroring human cognitive processes. This dual-agent system allows AI to parallelize the tasks of immediate response coordination and comprehensive problem analysis, reducing latency and improving efficiency. The modular separation further simplifies the design and deployment of intelligent agents, allowing individual components to be optimized independently.

Practical Application and Evaluation

The paper grounds its framework within the context of an AI sleep coaching agent, providing qualitative assessments of the Talker-Reasoner model. The evaluation demonstrated the system's proficiency in managing dialogue and executing sophisticated planning tasks related to sleep improvement strategies. Observations include:

  • The Talker efficiently managed straightforward conversational duties without waiting for updated belief states from the Reasoner, illustrating its capability for fast, intuitive processing.
  • In scenarios requiring detailed plans or problem-solving, the Reasoner's intervention was crucial, highlighting the necessity for occasional System 2 involvement.

Future Directions

Future work should focus on enhancing the dynamic interplay between the Talker and Reasoner, particularly in deciding when Talker should defer to Reasoner conclusions. Further exploration of multiple concurrent Reasoner modules, each specializing in distinct reasoning domains, presents another promising direction. Such advancements could allow each module to independently update a shared memory, enhancing the overall system's adaptability and responsiveness.

Conclusion

The Talker-Reasoner architecture presents a compelling step towards more sophisticated and human-like AI systems. By incorporating principles from cognitive science, this framework offers an effective paradigm for developing agents that can balance conversational engagement with complex decision-making processes. The practical implications in diverse domains signal a notable progression in the capabilities of intelligent conversational agents.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube