Papers
Topics
Authors
Recent
Search
2000 character limit reached

Reverse Multi-Choice Dialogue Commonsense Inference with Graph-of-Thought

Published 23 Dec 2023 in cs.CL | (2312.15291v2)

Abstract: With the proliferation of dialogic data across the Internet, the Dialogue Commonsense Multi-choice Question Answering (DC-MCQ) task has emerged as a response to the challenge of comprehending user queries and intentions. Although prevailing methodologies exhibit effectiveness in addressing single-choice questions, they encounter difficulties in handling multi-choice queries due to the heightened intricacy and informational density. In this paper, inspired by the human cognitive process of progressively excluding options, we propose a three-step Reverse Exclusion Graph-of-Thought (ReX-GoT) framework, including Option Exclusion, Error Analysis, and Combine Information. Specifically, our ReX-GoT mimics human reasoning by gradually excluding irrelevant options and learning the reasons for option errors to choose the optimal path of the GoT and ultimately infer the correct answer. By progressively integrating intricate clues, our method effectively reduces the difficulty of multi-choice reasoning and provides a novel solution for DC-MCQ. Extensive experiments on the CICERO and CICERO$_{v2}$ datasets validate the significant improvement of our approach on DC-MCQ task. On zero-shot setting, our model outperform the best baseline by 17.67% in terms of F1 score for the multi-choice task. Most strikingly, our GPT3.5-based ReX-GoT framework achieves a remarkable 39.44% increase in F1 score.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Conversational Neuro-Symbolic Commonsense Reasoning. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI’21), 4902–4911.
  2. Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI’21), 4923–4931.
  3. Language Models are Few-Shot Learners. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS’20).
  4. Distinguish Before Answer: Generating Contrastive Explanation as Knowledge for Commonsense Question Answering. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Findings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 13207–13224. Association for Computational Linguistics.
  5. Jailbreaker: Automated Jailbreak Across Multiple Large Language Model Chatbots. CoRR, abs/2307.08715.
  6. Zero-Shot Commonsense Question Answering with Cloze Translation and Consistency Optimization. In Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI’22), 10572–10580. AAAI Press.
  7. Reasoning Implicit Sentiment with Chain-of-Thought Prompting. arXiv preprint arXiv:2305.11255.
  8. CIDER: Commonsense Inference for Dialogue Explanation and Reasoning. In Li, H.; Levow, G.; Yu, Z.; Gupta, C.; Sisman, B.; Cai, S.; Vandyke, D.; Dethlefs, N.; Wu, Y.; and Li, J. J., eds., Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial’21), 301–313. Association for Computational Linguistics.
  9. Two is Better than Many Binary? Classification as an Effective Approach to Multi-Choice Question Answering. arXiv preprint arXiv:2210.16495.
  10. CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL’22), 5010–5028.
  11. MathPrompter: Mathematical Reasoning using Large Language Models. In Sitaram, S.; Klebanov, B. B.; and Williams, J. D., eds., Proceedings of the The 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 37–42. Association for Computational Linguistics.
  12. Tab-CoT: Zero-shot Tabular Chain of Thought. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Findings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 10259–10277. Association for Computational Linguistics.
  13. Enhancing multiple-choice machine reading comprehension by punishing illogical interpretations. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’21), 3641–3652.
  14. Zero-Shot Prompting for Implicit Intent Prediction and Recommendation with Commonsense Reasoning. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Findings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 249–258. Association for Computational Linguistics.
  15. Generated Knowledge Prompting for Commonsense Reasoning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL’22), 3154–3169.
  16. Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI’21), 13507–13515.
  17. HybridPrompt: Bridging Language Models and Human Priors in Prompt Tuning for Visual Question Answering. In Williams, B.; Chen, Y.; and Neville, J., eds., Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI’23), 13371–13379.
  18. Training language models to follow instructions with human feedback. In Proceedings of the 36th International Conference on Neural Information Processing Systems (NeurIPS’22), 27730–27744.
  19. Prompting Contrastive Explanations for Commonsense Reasoning Tasks. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL-IJCNLP 2021 of Findings of ACL, 4179–4192. Association for Computational Linguistics.
  20. TIMEDIAL: Temporal Commonsense Reasoning in Dialog. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, 7066–7076. Association for Computational Linguistics.
  21. Commonsense Reasoning for Conversational AI: A Survey of the State of the Art. CoRR, abs/2302.07926.
  22. Multiview contextual commonsense inference: A new dataset and task. arXiv preprint arXiv:2210.02890.
  23. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 10014–10037. Association for Computational Linguistics.
  24. Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 2609–2634. Association for Computational Linguistics.
  25. SCOTT: Self-Consistent Chain-of-Thought Distillation. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 5546–5558. Association for Computational Linguistics.
  26. A Co-Matching Model for Multi-choice Reading Comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL’18), 746–751.
  27. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Proceedings of the 36th International Conference on Neural Information Processing Systems (NeurIPS’22).
  28. NExT-GPT: Any-to-Any Multimodal LLM. CoRR, abs/2309.05519.
  29. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. CoRR, abs/2305.10601.
  30. Synthesize, Prompt and Transfer: Zero-shot Conversational Question Generation with Pre-trained Language Model. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 8989–9010. Association for Computational Linguistics.
  31. DCMN+: Dual co-matching network for multi-choice reading comprehension. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI’20), 9563–9570.
  32. Automatic Chain of Thought Prompting in Large Language Models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
  33. Knowledgeable Parameter Efficient Tuning Network for Commonsense Question Answering. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), 9051–9063. Association for Computational Linguistics.
  34. ECQED: Emotion-Cause Quadruple Extraction in Dialogs. CoRR, abs/2306.03969.
  35. A Bi-directional Multi-hop Inference Model for Joint Dialog Sentiment Classification and Act Recognition. In Natural Language Processing and Chinese Computing - 12th National CCF Conference, NLPCC 2023, Foshan, China, October 12-15, 2023, Proceedings, Part I, volume 14302, 235–248. Springer.
Citations (6)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up