Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unexplored Frontiers: A Review of Empirical Studies of Exploratory Search (2312.13695v2)

Published 21 Dec 2023 in cs.IR, cs.DL, and cs.HC

Abstract: This article reviews how empirical research of exploratory search is conducted. We investigated aspects of interdisciplinarity, study settings and evaluation methodologies from a systematically selected sample of 231 publications from 2010-2021, including a total of 172 articles with empirical studies. Our results show that exploratory search is highly interdisciplinary, with the most frequently occurring publication venues including high impact venues in information science, information systems and human-computer interaction. However, taken in aggregate, the breadth of study settings investigated was limited. We found that a majority of studies (77%) focused on evaluating novel retrieval systems as opposed to investigating users' search processes. Furthermore, a disproportionate number of studies were based on scientific literature search (20.7%), a majority of which only considered searching for Computer Science articles. Study participants were generally from convenience samples, with 75% of studies composed exclusively of students and other academics. The methodologies used for evaluation were mostly quantitative, but lacked consistency between studies and validated questionnaires were rarely used. In discussion, we offer a critical analysis of our findings and suggest potential improvements for future exploratory search studies.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Is exploratory search different? A comparison of information search behavior for exploratory and lookup tasks. Journal of the Association for Information Science and Technology 67, 11 (2016), 2635–2651.
  2. Beyond relevance: Adapting exploration/exploitation in information retrieval. In Proceedings of the 21st international conference on intelligent user interfaces. 359–369.
  3. Marcia J Bates. 1989. The design of browsing and berrypicking techniques for the online search interface. Online review (1989).
  4. ASK for information retrieval: Part I. Background and theory. Journal of documentation (1982).
  5. Pia Borlund. 2003. The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. Information research 8, 3 (2003), 8–3.
  6. John Brooke. 1996. SUS: A “quick and dirty” usability scale. Usability evaluation in industry 189, 3 (1996), 189–194.
  7. Supporting the discoverability of open educational resources. Education and Information Technologies 24 (2019), 3129–3161.
  8. Evaluation of a novel conjunctive exploratory navigation interface for consumer health information: a crowdsourced comparative study. Journal of medical Internet research 16, 2 (2014), e45.
  9. Brenda Dervin. 1998. Sense-making theory and practice: An overview of user interests in knowledge seeking and use. Journal of knowledge management (1998).
  10. A study on user-controllable social exploratory search. In 23rd International conference on intelligent user interfaces. 353–364.
  11. Demographics and dynamics of mechanical turk workers. In Proceedings of the eleventh ACM international conference on web search and data mining. 135–143.
  12. Revisiting exploratory search from the hci perspective. HCIR 2010 99 (2010).
  13. Using a concept map to evaluate exploratory search. In Proceedings of the third symposium on Information interaction in context. 175–184.
  14. How concept maps change if a user does search or not?. In Proceedings of the 5th Information Interaction in Context Symposium. 68–75.
  15. Directing exploratory search: Reinforcement learning from user interactions with keywords. In Proceedings of the 2013 international conference on Intelligent user interfaces. 117–128.
  16. Chathra Hendahewa and Chirag Shah. 2017. Evaluating user search trails in exploratory search tasks. Information Processing & Management 53, 4 (2017), 905–922.
  17. The weirdest people in the world? Behavioral and brain sciences 33, 2-3 (2010), 61–83.
  18. Diane Kelly et al. 2009. Methods for evaluating interactive information retrieval systems with users. Foundations and Trends® in Information Retrieval 3, 1–2 (2009), 1–224.
  19. Exploration or fact-finding: Inferring user’s search activity just in time. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. 87–96.
  20. Exploratory Search of GANs with Contextual Bandits. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 3157–3161.
  21. Bill Kules and Robert Capra. 2008. Creating exploratory tasks for a faceted search interface. Proc. of HCIR (2008), 18–21.
  22. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159–174.
  23. Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic web 6, 2 (2015), 167–195.
  24. Yuelin Li and Nicholas J Belkin. 2010. An exploration of the relationships between work task and interactive information search behavior. Journal of the American Society for information Science and Technology 61, 9 (2010), 1771–1789.
  25. Jingjing Liu and Nicholas J Belkin. 2010. Personalizing information retrieval for multi-session tasks: The roles of task stage and task type. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. 26–33.
  26. Rogue: A system for exploratory search of gans. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval. 3278–3282.
  27. Susan E MacMullin and Robert S Taylor. 1984. Problem dimensions and information traits. The information society 3, 1 (1984), 91–111.
  28. Gary Marchionini. 1993. Information seeking in full-text end-user-oriented search systems: The roles of domain and search expertise. Library & information science research 15, 1 (1993), 35–69.
  29. Gary Marchionini. 2006. Exploratory search: from finding to understanding. Commun. ACM 49, 4 (2006), 41–46.
  30. Alan Medlar and Dorota Glowacka. 2018. How consistent is relevance feedback in exploratory search?. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 1615–1618.
  31. PULP: A system for exploratory search of scientific literature. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 1133–1136.
  32. Query suggestions as summarization in exploratory search. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval. 119–128.
  33. Towards fine-grained adaptation of exploration/exploitation in information retrieval. In Proceedings of the 22nd International Conference on Intelligent User Interfaces. 623–627.
  34. How much is too much? Whole session vs. first query behaviors in task type prediction. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 1141–1144.
  35. Improving the Quality of Reports of Meta-Analyses of Randomised Controlled Trials: The QUOROM Statement. Lancet 354 (12 1999), 1896–900.
  36. Heather L O’brien and Elaine G Toms. 2013. Examining the generalizability of the User Engagement Scale (UES) in exploratory search. Information Processing & Management 49, 5 (2013), 1092–1107.
  37. Steven Pace. 2004. A grounded theory of the flow experiences of Web users. International journal of human-computer studies 60, 3 (2004), 327–363.
  38. A survey of definitions and models of exploratory search. In Proceedings of the 2017 ACM workshop on exploratory search and interactive data analytics. 3–8.
  39. Ruth A Palmquist and Kyung-Sun Kim. 2000. Cognitive style and on-line database search experience as predictors of Web search performance. Journal of the american society for information science 51, 6 (2000), 558–566.
  40. Designing health websites based on users’ web-based information-seeking behaviors: A mixed-method observational study. Journal of medical Internet research 18, 6 (2016), e145.
  41. A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems. 157–164.
  42. Looking ahead: Query preview in exploratory search. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. 243–252.
  43. Reijo Savolainen. 2018. Berrypicking and information foraging: Comparison of two theoretical frameworks for studying exploratory search. Journal of Information Science 44, 5 (2018), 580–593.
  44. Chirag Shah. 2010. Coagmento-a collaborative information seeking, synthesis and sense-making framework. Integrated demo at CSCW 2010 (2010).
  45. Assisting user browsing over linked data: Requirements elicitation with a user study. In Web Engineering: 13th International Conference, ICWE 2013, Aalborg, Denmark, July 8-12, 2013. Proceedings 13. Springer, 376–383.
  46. Task effects on interactive search: The query factor. In International Workshop of the Initiative for the Evaluation of XML Retrieval. Springer, 359–372.
  47. Jane Webster and Richard T Watson. 2002. Analyzing the past to prepare for the future: Writing a literature review. MIS quarterly (2002), xiii–xxiii.
  48. Ryen W White and Resa A Roth. 2009. Exploratory search: Beyond the query-response paradigm. Synthesis lectures on information concepts, retrieval, and services 1, 1 (2009), 1–98.
  49. Barbara M Wildemuth and Luanne Freund. 2012. Assigning search tasks designed to elicit exploratory search behaviors. In Proceedings of the symposium on human-computer interaction and information retrieval. 1–10.
  50. Grannies, tanning beds, tattoos and NASCAR: Evaluation of search tasks with varying levels of cognitive complexity. In Proceedings of the 4th information interaction in context symposium. 254–257.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Alan Medlar (5 papers)
  2. Denis Kotkov (1 paper)
  3. Dorota Glowacka (7 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now