Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Self-Contained Answers: Entity-Based Answer Rewriting in Conversational Search (2403.01747v1)

Published 4 Mar 2024 in cs.IR and cs.CL

Abstract: Conversational information-seeking (CIS) is an emerging paradigm for knowledge acquisition and exploratory search. Traditional web search interfaces enable easy exploration of entities, but this is limited in conversational settings due to the limited-bandwidth interface. This paper explore ways to rewrite answers in CIS, so that users can understand them without having to resort to external services or sources. Specifically, we focus on salient entities -- entities that are central to understanding the answer. As our first contribution, we create a dataset of conversations annotated with entities for saliency. Our analysis of the collected data reveals that the majority of answers contain salient entities. As our second contribution, we propose two answer rewriting strategies aimed at improving the overall user experience in CIS. One approach expands answers with inline definitions of salient entities, making the answer self-contained. The other approach complements answers with follow-up questions, offering users the possibility to learn more about specific entities. Results of a crowdsourcing-based study indicate that rewritten answers are clearly preferred over the original ones. We also find that inline definitions tend to be favored over follow-up questions, but this choice is highly subjective, thereby providing a promising future direction for personalization.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Asking clarifying questions in open-domain information-seeking conversations. In Proceedings of the 42nd International ACM SIGIR conference on research and development in information retrieval (SIGIR ’19). 475–484.
  2. Mixed-initiative interaction. IEEE Intelligent Systems and their Applications 14, 5 (1999), 14–23.
  3. Conversational search (Dagstuhl seminar 19461). In Dagstuhl Reports, Vol. 9. Schloss Dagstuhl-Leibniz-Zentrum für Informatik.
  4. Open-Domain Question Answering Goes Conversational via Question Rewriting. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL ’21). 520–534.
  5. Krisztian Balog. 2018. Entity-oriented search. Springer.
  6. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS ’20). 1877–1901.
  7. Searching to Learn with Instructional Scaffolding. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval (CHIIR ’21). 209–218.
  8. QuAC: Question answering in context. arXiv preprint arXiv:1808.07036 (2018).
  9. Personalizing Web Search Results by Reading Level. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management (CIKM ’11). 403–412.
  10. TREC CAsT 2019: The conversational assistance track overview. In The Twenty-Eighth Text REtrieval Conference Proceedings (TREC’19).
  11. Lessons from the Journey: A Query Log Analysis of within-Session Learning. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM ’14). 223–232.
  12. Automatic Simplification of Scientific Texts: SimpleText Lab at CLEF-2022. In European Conference on Information Retrieval (ECIR ’22). 364–373.
  13. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin 76, 5 (1971), 378.
  14. Analyzing knowledge gain of users in informational search sessions on the web. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR ’18). 2–11.
  15. Identifying Salient Entities in Web Pages. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management (CIKM ’13). 2375–2380.
  16. Precise Zero-Shot Dense Retrieval without Relevance Labels. (2023), 1762–1777.
  17. Yasin Ghafourian. 2022. Relevance Models Based on the Knowledge Gap. In European Conference on Information Retrieval (ECIR ’22). Springer, 488–495.
  18. Collective entity linking in web text: a graph-based method. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval (SIGIR ’11). 765–774.
  19. Marti Hearst. 2009. Search user interfaces. Cambridge University Press.
  20. Survey of hallucination in natural language generation. Comput. Surveys 55, 12 (2023), 1–38.
  21. Hideaki Joko and Faegheh Hasibi. 2022. Personal Entity, Concept, and Named Entity Linking in Conversations. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (CIKM ’22’). 4099–4103.
  22. Conversational Entity Linking: Problem Definition and Datasets. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21). 2390–2397.
  23. Natural Questions: A Benchmark for Question Answering Research. Transactions of the Association for Computational Linguistics 7 (2019), 452–466.
  24. User evaluation of the effects of a text simplification algorithm using term familiarity on perception, understanding, learning, and information retention. Journal of medical Internet research 15, 7 (2013), e2569.
  25. Acute-eval: Improved dialogue evaluation with optimized questions and multi-turn comparisons. arXiv preprint arXiv:1909.03087 (2019).
  26. Gary Marchionini. 2006. Exploratory search: from finding to understanding. Commun. ACM 49, 4 (2006), 41–46.
  27. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 27730–27744.
  28. Deepa Paranjpe. 2009. Learning document aboutness from implicit user feedback and document structure. In Proceedings of the 18th ACM conference on Information and knowledge management (CIKM ’09). 365–374.
  29. Visconde: Multi-document QA with GPT-3 and Neural Reranking. In Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2–6, 2023, Proceedings, Part II (ECIR ’23’). 534–543.
  30. Francesco Piccinno and Paolo Ferragina. 2014. From TagME to WAT: A new entity annotator. In Proceedings of the first international workshop on Entity recognition & disambiguation (ERD ’14). 55–62.
  31. Swat: A system for detecting salient Wikipedia entities in texts. Computational Intelligence 35, 4 (2019), 858–890.
  32. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 140 (2020), 1–67.
  33. Tetsuya Sakai. 2021. Graded relevance. In Evaluating Information Retrieval and Access Tasks. 1–20.
  34. Kim Cheng Sheang and Horacio Saggion. 2021. Controllable Sentence Simplification with a Unified Text-to-Text Transfer Transformer. In Proceedings of the 14th International Conference on Natural Language Generation (INLG ’21). 341–352.
  35. Dynamic composition for conversational domain exploration. In Proceedings of The Web Conference 2020 (TheWebConf ’20). 872–883.
  36. A comparison of question rewriting methods for conversational passage retrieval. In European Conference on Information Retrieval (ECIR ’21). 418–424.
  37. Understanding interobserver agreement: the kappa statistic. Family Medicine 37, 5 (2005), 360–363.
  38. Ryen W White. 2016. Interactions with search systems. Cambridge University Press.
  39. It all starts with entities: A Salient entity topic model. Natural Language Engineering 26, 5 (2020), 531–549.
  40. Few-shot generative conversational query rewriting. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval (SIGIR ’20). 1933–1936.
  41. Few-shot conversational dense retrieval. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21). 829–838.
  42. Conversational information seeking. Foundations and Trends® in Information Retrieval 17, 3-4 (2023), 244–456.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Krisztian Balog (75 papers)
  2. Fabio Crestani (31 papers)
  3. Ivan Sekulić (12 papers)
Citations (4)
View on Semantic Scholar