Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Dialogues for Joint Human-AI Reasoning and Value Alignment (2405.18073v1)

Published 28 May 2024 in cs.AI

Abstract: We argue that enabling human-AI dialogue, purposed to support joint reasoning (i.e., 'inquiry'), is important for ensuring that AI decision making is aligned with human values and preferences. In particular, we point to logic-based models of argumentation and dialogue, and suggest that the traditional focus on persuasion dialogues be replaced by a focus on inquiry dialogues, and the distinct challenges that joint inquiry raises. Given recent dramatic advances in the performance of LLMs, and the anticipated increase in their use for decision making, we provide a roadmap for research into inquiry dialogues for supporting joint human-LLM reasoning tasks that are ethically salient, and that thereby require that decisions are value aligned.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (82)
  1. 2024. Let the llms talk: Simulating human-to-human conversational qa via zero-shot llm-to-llm interactions. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, WSDM ’24, 8–17. New York, NY, USA: Association for Computing Machinery.
  2. 2024. Supporting student decisions on learning recommendations: An llm-based chatbot with knowledge graph contextualization for conversational explainability and mentoring. arXiv preprint arXiv:2401.08517.
  3. Araci, D. 2019. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063.
  4. 2007. Practical reasoning as presumptive argumentation using action based alternating transition systems. Artificial Intelligence 171(10):855–874.
  5. 2014. Introduction to structured argumentation. Argument and Computation 5.
  6. 2022. Towards an argument scheme classification for ethical reasoning. In Grasso, F.; Green, N. L.; Schneider, J.; and Wells, S., eds., Proceedings of the 22nd Workshop on Computational Models of Natural Argument, CMNA@COMMA 2022, Cardiff, Wales, September 12, 2022, volume 3205 of CEUR Workshop Proceedings, 13–17. CEUR-WS.org.
  7. 2009. An inquiry dialogue system. Autonomous Agents and Multi-agent Systems 19:173–209.
  8. 2022. Biomedlm: a domain-specific large language model for biomedical text. Stanford CRFM Blog.
  9. 1997. An abstract, argumentation–theoretic approach to default reasoning. Artificial Intelligence 93:63–101.
  10. Bostrom, N. 2014. Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
  11. 2020. Language models are few-shot learners. Advances in neural information processing systems 33:1877–1901.
  12. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712.
  13. 2023. Eqrbot: A chatbot delivering eqr argument-based explanations. Frontiers in Artificial Intelligence 6:1045614.
  14. 2024. Computational argumentation-based chatbots: a survey. arXiv preprint arXiv:2401.03454.
  15. Chang, E. Y. 2023. Prompting large language models with the socratic method. In 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), 0351–0360.
  16. 2023. Introspective tips: Large language model for in-context decision making. arXiv preprint arXiv:2305.11598.
  17. 2023. Agent instructs large language models to be general zero-shot reasoners. arXiv preprint arXiv:2310.03710.
  18. 2018. Classical logic, argument and dialectic. Artificial Intelligence 262:15–51.
  19. 2021. A fully rational account of structured argumentation under resource bounds. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI’20.
  20. 2023. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378.
  21. Dung, P. 1995. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n𝑛nitalic_n–person games. Artificial Intelligence 77:321–357.
  22. 2014. A general framework for sound assumption–based argumentation dialogues. Artificial. Intelligence. 216(0):20 – 54.
  23. 2004. Defeasible logic programming: an argumentative approach. Theory and Practice of Logic Programming 4(1–2):95–138.
  24. 2017. Argumentation mining in user-generated web discourse. Computational linguistics 43(1):125–179.
  25. 2016. Cooperative inverse reinforcement learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, 3916–3924. Red Hook, NY, USA: Curran Associates Inc.
  26. 2023. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–19.
  27. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556.
  28. 2019. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342.
  29. 2023. Instruct2act: Mapping multi-modality instructions to robotic actions with large language model. arXiv preprint arXiv:2305.11176.
  30. 2022. Maieutic prompting: Logically consistent reasoning with recursive explanations. arXiv preprint arXiv:2205.11822.
  31. 2023. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050.
  32. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35:22199–22213.
  33. 2021. Applying metalevel argumentation frameworks to support medical decision making. IEEE Intelligent Systems 36(2):64–71.
  34. 2024. Towards understanding counseling conversations: Domain knowledge and large language models. In Graham, Y., and Purver, M., eds., Findings of the Association for Computational Linguistics: EACL 2024, 2032–2047. St. Julian’s, Malta: Association for Computational Linguistics.
  35. 2022. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems 35:3843–3857.
  36. 2022. Design guidelines for prompt engineering text-to-image generative models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 1–23.
  37. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys 55(9):1–35.
  38. 2024. From llm to conversational agent: A memory enhanced architecture with fine-tuning of large language models. arXiv preprint arXiv:2401.02777.
  39. 2023. Memochat: Tuning llms to use memos for consistent long-range open-domain conversation. arXiv preprint arXiv:2308.08239.
  40. 2020. Moral uncertainty. Oxford University Press.
  41. 2009. Chapter 6 : Proof theories and algorithms for abstract argumentation frameworks. In Simari, G., and Rahwan, I., eds., Argumentation in AI. Springer. 105–129.
  42. 2010. Reasoning about preferences in structured extended argumentation frameworks. In Proceedings of the 2010 Conference on Computational Models of Argument: Proceedings of COMMA 2010, 347–358. NLD: IOS Press.
  43. 2013. A general account of argumentation with preferences. Artificial Intelligence 195:361–397.
  44. 2018. Abstract rule–based argumentation. In P. Baroni, D. Gabbay, M. G., and van der Torre, L., eds., Handbook of Formal Argumentation. 286–361.
  45. Modgil, S. 2017a. Dialogical scaffolding for human and artificial agent reasoning. In AIC, 58–71.
  46. Modgil, S. 2017b. Towards a general framework for dialogues that accommodate reasoning about preferences. In International Workshop on Theorie and Applications of Formal Argumentation, 175–191. Springer.
  47. 2023. Leveraging large language models for collective decision-making. arXiv preprint arXiv:2311.04928.
  48. Prakken, H. 2005. Coherence and flexibility in dialogue games for argumentation. Journal of logic and computation 15(6):1009–1040.
  49. Prakken, H. 2009. Models of Persuasion Dialogue. Boston, MA: Springer US. 281–300.
  50. 2019. Explain yourself! leveraging language models for commonsense reasoning. arXiv preprint arXiv:1906.02361.
  51. 2015. Research priorities for robust and beneficial artificial intelligence. AI magazine 36(4):105–114.
  52. Russell, S. 2020. Human Compatible: AI and the Problem of Control. Penguin Press.
  53. 2023. Cognitive architectures for language agents. arXiv preprint arXiv:2309.02427.
  54. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261.
  55. 2024. Moral uncertainty and the problem of fanaticism. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 19948–19955.
  56. 2020. Alignment for Advanced Machine Learning Systems. 342–382.
  57. 2014. Translating preferred subtheories into structured argumentation. Journal of Logic and Computation 74(4):831–849.
  58. 2012. Deliberation dialogues for reasoning about safety critical actions. Autonomous Agents and Multi-Agent Systems 25(2):209–259.
  59. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  60. 2024. Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems 36.
  61. 2021. Annotating argument schemes. Argumentation 35(1):101–139.
  62. Vlastos, G. 1982. The socratic elenchus. The Journal of Philosophy 79(11):711–714.
  63. 2023. Does gpt-3 grasp metaphors? identifying metaphor mappings with generative language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1018–1032.
  64. 2015. A classification system for argumentation schemes. Argument & Computation 6(3):219–245.
  65. 2008. Argumentation schemes. Cambridge University Press.
  66. Walton, D. 2009. Argumentation theory: A very short introduction. In Argumentation in artificial intelligence. Springer. 1–22.
  67. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  68. 2023. A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432.
  69. 2022a. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682.
  70. 2022b. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35:24824–24837.
  71. 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382.
  72. 2023. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564.
  73. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864.
  74. 2020. Enthymemes in dialogues. In COMMA, 355–369.
  75. 2022. A sound and complete dialogue system for handling misunderstandings. In SAFA@COMMA, 19–32.
  76. 2023. Foundation models for decision making: Problems, methods, and opportunities. arXiv preprint arXiv:2303.04129.
  77. 2016. Prioritised default logic as rational argumentation. In Proc. 15th Int. Conference on Autonomous Agents and Multiagent Systems (AAMAS16), 626–634.
  78. 2022. STar: Bootstrapping reasoning with reasoning. In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Processing Systems.
  79. 2022a. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068.
  80. 2022b. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493.
  81. 2024. A survey on the memory mechanism of large language model based agents. arXiv preprint arXiv:2404.13501.
  82. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Elfia Bezou-Vrakatseli (4 papers)
  2. Oana Cocarascu (14 papers)
  3. Sanjay Modgil (7 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets